SNM Radiation Signature Classification Using Different Semi-Supervised Machine Learning Models

Round 1

Reviewer 1 Report

This paper presents the use of semi supervised learning algorithms to detect transportation of SNM.
The use of supervised learning algorithms cannot realistically be envisaged because of the large amount of labeled data that is required. A mix of labeled and unlabeled data are used to train four models:
pseudo-label are used for the training based on spectral shapes that give radiation events categories, thus providing a Noisy labeling.
The different algorithms performances are then compared.
This paper is interesting and can inspire other researches.
I have only a minor remark: For the reader unfamiliar with SSM, it would be useful to indicate that teacher is trained on labeled images and student is trained on labeled and unlabeled images.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Thank you for submitting your paper. I found it well written and interesting to read, if a tad bit long to get through. There's a choice to be made about explaining in detail ML to the reader vs. getting into the meat of the study. I think you've almost perfectly balanced it. I will provide below first some overarching comments on the paper in general, and then detailed comments about specific points.

Overarching:

I think you can in general describe the cost of labeling the data in more detail to convince the reader why semi-supervised machine learning is necessary. Does ORNL not log material transfers in a logbook? Then it's easy to say "In the past month there were N material transfers. Transfer n=1 took place at 10:00 on Monday the xx of yy month. Detectors will have detected the transfer then." There is manpower needed, but it isn't overwhelming and well suited for an intern. Figure 3 also shows a clear difference between a background and an event. Why isn't it as simple as doing the subtraction of a known background vs. events and then if it exceeds a threshold label it? The ML aspects are very interesting, but I think you could explain why ML is needed better vs. it just being interesting to apply ML to a data set. Are there transfers that aren't obvious in the data set where a simplistic approach doesn't work? Does ML detect these better? In summary: quantify why labeling data is so unfeasible, and show why simple approaches fail and ML is needed for detecting material transfers.

Individual comments:

- Line 29: Does using SNM in general need timely detection? Or does the illicit use need it?

- Line 45: Comment above, suggest giving an example of how and why it's too costly to label

- Line 52: 'when labeled data are'

- Line 86: change compute to computing

- Line 111: Is the presence or absence of SNM a continuous system? Seems like a binary classification to me.

- Line 115: Sentence starting with "the set of labels" reads strangely. Object of sentence missing predicate.

- Eq 2/3, lines 132-137: I found the indexing and variables hard to follow here. Recommend treating vectors as bold font here to help separate them from the scalars. The j index in in the feature vector x_i is reused to iterate in the model for multiplying by weights. Eq. 3 has w as a vector, while in text it is iterated as a scalar.

- Line 146: recommend defining decision boundary.

- Line 201: Are gain shifts of the detector over long time periods and over large temperature changes a problem? Does this effect how you do background subtraction of identify a photopeak?

- Figure 3: recommend indicating if y-axis is long. recommend cropping the y-axis to help zoom in on features more. Or giving a second y -axis that helps see the difference better.

- Figure 4: I'm not sure if this is truly a PDF. PDF is is a mathematical function to describe a random variable. This is more of a frequency histogram.

- Line 262: First time RadClass is mentioned. I'm not familiar with what this is.

- Paragraph at line 284: recommend clearly defining what a positive is in this case.

- caption fig 6: change which varies to which vary

-line 306: should analysis below be analysis above?

- fig 8: recommend defining what test accuracy is quantitatively.

- eq 10: what is alpha? the perturbation ?

- Line 434: would a different loss function change the noisy behavior?

- page 14: for understanding the importance of TP, FP, TN, FN what are the stakes for a FN? Is it unacceptable?

- For the confusion matrices of fig 11, recommend more clearly tying the 0 and 1 to TP, FP, TN, FN

- line 494: what is recall in this context?

-

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Congratulations for the work. I found the manuscript very well driven and clear for the reader. It combines the knowledge in different disciplines for the demonstration of use of ML in gamma-ray analysis.

All explanations, figures and examples help a lot to a much better understanding of the procedure. Results are clear and of high interest.

Although more detailed and complete work is advisable in the future, here you present a very serious work and I recommend the manuscript for publication as it is.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf