Searching for Short-Timescale Transients in Gamma-ray Telescope Data
Round 1
Reviewer 1 Report
The method proposed for the analysis of multiplets is well described and tested. The paper is useful because can be a first step to develop a general method to identify and analyze multiplets in experiments with different characteristics.
Anyway, the proposed method can be a first step for a more evolved method that has to consider some other factors as the energy of the event and the angular resolution. Depending on the characteristics of each event collected by the considered detectors, the statistic error in the identification of the direction can change. This resolution is a relevant parameter if the angle between two events is the analyzed quantity. Just assuming the reconstructed direction of the event without considering the associate uncertainty can reduce the power of the method. Also, the energy of the events that are put together in a multiplet has to be taken into account. Another category of experiments needs the energy as a relevant ingredient in the algorithms for the identification of the multiplets (see as example HAWK or LHASOO or the proposed project SWGO). A general method for the identification of multiplets is estremely useful for them.
In details.
On page 8 line 278-282 you say that the method overestimates the probability of the multiplet. To me, this means that there is an intrinsic systematic in the method. I think that this sentence must be better supported to explain why this happened and the source of the systematic. If not, the overall impression is that the probability estimation is generally wrong, and the method apparently works in low probability conditions where the estimation of the probability is affected by too large fluctuations or the statistical uncertainty “hides” the systematics.
Typo: line 278 Sincce -> Since
Author Response
Dear Reviewer,
Thank you for taking the time to review our work. We appreciate your constructive comments, they have greatly contributed to improving the quality and clarity of our work. Below, we address each of the raised points:
Comment: Anyway, the proposed method can be a first step for a more evolved method that has to consider some other factors as the energy of the event and the angular resolution. Depending on the characteristics of each event collected by the considered detectors, the statistic error in the identification of the direction can change. This resolution is a relevant parameter if the angle between two events is the analyzed quantity. Just assuming the reconstructed direction of the event without considering the associate uncertainty can reduce the power of the method. Also, the energy of the events that are put together in a multiplet has to be taken into account. Another category of experiments needs the energy as a relevant ingredient in the algorithms for the identification of the multiplets (see as example HAWK or LHASOO or the proposed project SWGO). A general method for the identification of multiplets is estremely useful for them.
Response: We have clarified that the search tool is a first step and more useful features will be implemented in the future as work on it progresses. To motivate our use of average psf for the smallest enclosing circle and why angular resolution is not used is the search process in this iteration of the tool, we have added the following to the discussion:
"Note that the method presented here is only intended to search large datasets for burst-like clusters of events and find the candidates worth further study, using the probability of them originating from background fluctuations as a criterion. Systematic errors of the instruments themselves are not taken into account -- the analyses presented above used the average of the telescopes' point-spread function (PSF) as the diameter of the smallest circle enclosing all events. The PSF is optimal for this because (i) if the maximum diameter of the smallest enclosing circle is chosen to be much smaller then the PSF, only part of the burst's events will be contained in it, and (ii) if the maximum diameter is too big then we are effectively integrating over the background as well. Taking the exact PSF into account (e.g by convolution) however would slow down the search process due to increased computation complexity, which is intended to be as fast as possible. Furthermore, the PSF is not necessarily symmetric, in which case it cannot be simply parametrised as an analytical function as energy. A possible approach to taking systematics into account and obtaining a localization contour of the multiplet is presented in Ref.~\cite{Ajello2021_sc_mgf}."
Comment: On page 8 line 278-282 you say that the method overestimates the probability of the multiplet. To me, this means that there is an intrinsic systematic in the method. I think that this sentence must be better supported to explain why this happened and the source of the systematic. If not, the overall impression is that the probability estimation is generally wrong, and the method apparently works in low probability conditions where the estimation of the probability is affected by too large fluctuations or the statistical uncertainty “hides” the systematics.
Response: We clarify that the simple method presented here is applicable for weak sources. In case of strong sources more information about the source is required in order to model it properly and not overestimate the probabilities of multiplets from it. We have modified the paragraph to elaborate on this part as follows:
"To estimate the probability of detecting such multiplets from background fluctuations by random chance, a simple method for simulating background multiplets was developed: the real data's time distribution is estimated using an exponential fit and its spatial distribution approximated by rejection sampling of coordinates from the exposure map. Since this method is based on the exposure maps neglecting the spectrum and morphology of the source, it will have a tendency for overestimating multiplet probabilities when analysing especially strong gamma-ray sources. It is therefore only valid for regions with potential sources featuring low signal-to-noise ratios; both conditions are usually fulfilled in extra-galactic fields of view. Thus, the tool may already be used for the analyses of extra galactic regions in its current stage of development. In order to simulate background multiplets from a strong source, the sources themselves must be modelled using their known morphology and energy spectra; for example using the spatial and spectral models available with \texttt{Gammapy}\footnote{\url{https://docs.gammapy.org/dev/tutorials/api/models.html}}."
We hope this better explains our concept and addresses your comments. Thank you once again for your valuable input.
Sincerely, AJ on behalf of authors
Reviewer 2 Report
The authors present a versatile tool designed for searching gamma-ray data for transient phenomena within short time scales. The paper is well-written and the tool demonstrates potential for widespread usage within the community. Therefore, I recommend the publication of this paper in Galaxy after addressing the following minor modifications:
Firstly, I suggest that the authors carefully review their codes available on GitHub to ensure they are executable and, for the most part, free of bugs. During my examination, I encountered import errors when attempting to run the Example_Fermi_LAT.ipynb code. This issue can be easily resolved by either placing the "gamma_transient" folder under the notebook directory or adjusting the path settings accordingly. Similar comments apply to Example_HESS_msh15-52_gammapy.ipynb, where consideration should be given to the general settings regarding the database path.
In essence, it would be beneficial for the authors to document in their codes how to establish the relevant path settings. This will facilitate easier setup for users seeking to replicate the results.
Author Response
Dear Reviewer,
Thank you for taking the time to review our work. We appreciate your constructive comments, they have greatly contributed to improving the quality and clarity of our work. Below, we address the raised point:
Comment: Firstly, I suggest that the authors carefully review their codes available on GitHub to ensure they are executable and, for the most part, free of bugs. During my examination, I encountered import errors when attempting to run the Example_Fermi_LAT.ipynb code. This issue can be easily resolved by either placing the "gamma_transient" folder under the notebook directory or adjusting the path settings accordingly. Similar comments apply to Example_HESS_msh15-52_gammapy.ipynb, where consideration should be given to the general settings regarding the database path.
Response: We have cross-checked this issue one more time and found no problems in our environment. Our assumption is that reviewer has gammapy installed, but did not install our package following the instructions on github. Wer have also updated the text as follows:
"The appearance of bugs and issues cannot be avoided at the initial stages of a project like this, the issues tracker in github can be used to facilitate communication with the tool's first adopters. Future versions of the tool will aim to resolve some of the limitations mentioned above and better integrate the tool into the framework of \texttt{Gammapy}."
We hope this better explains our concept and addresses your comments. Thank you once again for your valuable input.
Sincerely, AJ on behalf of authors
Reviewer 3 Report
The manuscript develops a tool capable of searching for short timescale transients in gamma-ray telescope data. This tool was then used to perform a short timescale analysis of Public Fermi LAT data and H.E.S.S. DL3 DR1 data as a proof of concept.
The paper is easy to follow and clearly written, thus I recommend this paper for publication after addressing a comment that I describe below.
As the authors said, since their methodology is based on the exposure maps neglecting the spectrum and morphology of the source, it will have a tendency for overestimating multiplet probabilities when analysing especially strong gamma-ray sources. Is it possible for the authors to consider the spectrum and morphology of the source in their method? The authors should discuss it somewhere.
Author Response
Dear Reviewer,
Thank you for taking the time to review our work. We appreciate your constructive comments, they have greatly contributed to improving the quality and clarity of our work. Below, we address the raised point:
Comment:
As the authors said, since their methodology is based on the exposure maps neglecting the spectrum and morphology of the source, it will have a tendency for overestimating multiplet probabilities when analysing especially strong gamma-ray sources. Is it possible for the authors to consider the spectrum and morphology of the source in their method? The authors should discuss it somewhere.
Response:
We clarify that the simple method presented here is applicable for weak sources. In case of strong sources more information about the source is required in order to model it properly and not overestimate the probabilities of multiplets from it. We have modified the paragraph to elaborate on this part as follows: "To estimate the probability of detecting such multiplets from background fluctuations by random chance, a simple method for simulating background multiplets was developed: the real data's time distribution is estimated using an exponential fit and its spatial distribution approximated by rejection sampling of coordinates from the exposure map. Since this method is based on the exposure maps neglecting the spectrum and morphology of the source, it will have a tendency for overestimating multiplet probabilities when analysing especially strong gamma-ray sources. It is therefore only valid for regions with potential sources featuring low signal-to-noise ratios; both conditions are usually fulfilled in extra-galactic fields of view. Thus, the tool may already be used for the analyses of extra galactic regions in its current stage of development. In order to simulate background multiplets from a strong source, the sources themselves must be modelled using their known morphology and energy spectra; for example using the spatial and spectral models available with \texttt{Gammapy}\footnote{\url{https://docs.gammapy.org/dev/tutorials/api/models.html}}."
We hope this better explains our concept and addresses your comments. Thank you once again for your valuable input.
Sincerely, AJ on behalf of authors
