Preprocessing for Unintended Conducted Emissions Classification with ResNet

Round 1

Reviewer 1 Report

The topic seems interesting, I have the following concerns to enhance the quality of the work.

• Why Resnet gives the best performance?
• The research problem/ requirement is not elaborated properly.
• The Authors need to explain how to handle class imbalance. It must be added to the proposed method.
• Authors need to re-write the Abstract in a more meaningful way example (Problem definition=> How existing methods are lacking => proposed solution => Outcome
• All equations should be assigned numbers. And align with the text.
• All figures should be redrawn with high resolutions and different colors.
• Authors should give all experiment parameters. Experimental setup, still few experiments paraments are missing??

 Conclusion and Future work must be updated.

Authors should need to update the the abstract and introduction part as well. 

The major contribution of the paper is very confusing for the readers.

The conclusion part should need to add another comparison table .

Author Response

(x) I would not like to sign my review report
( ) I would like to sign my review report

English language and style

(x) Extensive editing of English language and style required
Please note: All the authors are native English speakers and reviewed the manuscript thoroughly for grammatical errors.  In addition, after this was accomplished, a staff technical writer from the Oak Ridge National Laboratory reviewed the manuscript for English language, grammar, and style and 99% of the feedback from that staff member specializing in the English language communication was accepted.  We are not sure what to do further to improve the English language and style in the paper.  More detailed feedback will be necessary to further address the concerns.

Comments and Suggestions for Authors

The topic seems interesting, I have the following concerns to enhance the quality of the work.

• Why Resnet gives the best performance?

Please note that we do not state in the paper that the ResNet yields the best performance for this application.  It was selected as a representative deep network to facilitate the research and experimentation regarding the preprocessing techniques and the overarching paradigm of utilizing methodical preprocessing modifications and the output of a deep network to make hypotheses regarding the feature selection of the deep network for this application.  It also facilitates the paradigm of experimentation and hypothesis generation concerning what the deep network may see as noise, and something regarding its characteristics, providing a measure of explainability or guide or supplement further explainability studies into the use of the network for the types of data sets studied, while optimizing the performance of the overall signal processing and machine learning system.  We have attempted to explain this more clearly in a change to the introduction section of the paper.

• The research problem/ requirement is not elaborated properly.

An attempt was made to try to clarify this in the abstract and the introduction.

• The Authors need to explain how to handle class imbalance. It must be added to the proposed method.

We are not too sure we can improve upon what was in the paper.  Please advise concerning improvements needed more specifically.  From the paper:

“A total of about 3.7~s of data for each of the 18 devices was used for training across the 18 devices. The same amount of data was also used for validation purposes. Approximately 12~s of data per device were used for testing after training epochs were completed.”

As we show in the paper, each device is a class and we are stating that we are using exactly the same amount of data for each class.

• Authors need to re-write the Abstract in a more meaningful way example (Problem definition=> How existing methods are lacking => proposed solution => Outcome

We attempted to structure the Abstract closer to this suggested form.

• All equations should be assigned numbers. And align with the text. – All of the equations that we could find in the manuscript have been assigned numbers.  Please let us know where the equations are that don’t have the numbers.  Also, we have attempted to “left justify” the equations, which is how we interpreted the “align with the text” suggestion, if this is wrong, please let us know.
• All figures should be redrawn with high resolutions and different colors. – There is one figure that appeared to have colors that were close together for lines drawn close together (Figure 6).  We have attempted to replot that one and replace it in the document with what we perceive to be a better color scheme.  If there are more graphs of concern or the graph is still insufficient in some way, we will need to know how much resolution and what colors would be preferred for those graphs.  Thanks for the suggestion.  We think Figure 6 looks much better now.
• Authors should give all experiment parameters. Experimental setup, still few experiments paraments are missing??

The data was collected under another project that we reference in the paper.  We are limited in the data acquisition description by the specifications and parameters they provide in their paper. An attempt was made to point the reader to the referenced paper for details. Please let us know if there are specific additional parameters that are needed.

 Conclusion and Future work must be updated. – We need more details to understand exactly what is deficient so that we can correct it.  It is updated in that it is the most recent version that we had when we submitted the paper.  None of the suggested changes appear to impact the Conclusion and Future work section.

Authors should need to update the the abstract and introduction part as well. – We have attempted to modify the document to meet the above suggestions as they affect both the abstract and introduction.  Hopefully it is sufficient.

The major contribution of the paper is very confusing for the readers.

An attempt was made to clarify this in the abstract and introduction.

The conclusion part should need to add another comparison table . – We need more details.  There are no additional comparisons being made in the conclusion part.  So, we are uncertain what other table to include.  Comparing what with what?

Thank you for your review and feedback.

Author Response File: Author Response.docx

Reviewer 2 Report

The authors studied an interesting project, i.e. characterizing UCE from electronic devices. The authors claimed to present a novel framework of preprocessing plus deep neural network to classify UCE data. I have no question about the organization and contents of this paper. However, I did not see much novelty from this paper, especially compared to the cited paper by Vann et al. [5]. 

Author Response

The authors studied an interesting project, i.e. characterizing UCE from electronic devices. The authors claimed to present a novel framework of preprocessing plus deep neural network to classify UCE data. I have no question about the organization and contents of this paper. However, I did not see much novelty from this paper, especially compared to the cited paper by Vann et al. [5].  While we used novel preprocessing plus deep network combinations that did not exhibit a high degree of originality (variations of a theme), we believe that the methodical methods and description of hypothesis generation for explaining the feature selection and classification of the deep network classifier is more novel in the way it was performed, especially given the fact that we can generate hypotheses regarding the relationships between the features and the noise.  Thank you for your review and feedback.

Author Response File: Author Response.docx

Reviewer 3 Report

The authors investigate the Unintended Conducted Emissions analysis with a pre-processing and demonstrated that an FFT preprocessing increases the classification by 62%. A further optimized preprocessing yields an accuracy of more than 90%. I believe the research would be interesting to scholars from multiple research fields. The data is convincing and the approach is clear.

I would recommend the authors to discuss a little bit deeper why a neural network cannot implement FFT with extensive training. Is it possible to compare FFT preprocessing (optimized) + neural network vs. Fully optimized neural network with extensive training? It might be out of the scope of this study but would enhance the importance is this paper.

Author Response

The authors investigate the Unintended Conducted Emissions analysis with a pre-processing and demonstrated that an FFT preprocessing increases the classification by 62%. A further optimized preprocessing yields an accuracy of more than 90%. I believe the research would be interesting to scholars from multiple research fields. The data is convincing and the approach is clear.

I would recommend the authors to discuss a little bit deeper why a neural network cannot implement FFT with extensive training. Is it possible to compare FFT preprocessing (optimized) + neural network vs. Fully optimized neural network with extensive training? It might be out of the scope of this study but would enhance the importance is this paper. We are not claiming that a neural network cannot learn and implement an FFT with extensive training. We are merely stating that for applications with reduced training data, our approach is a good way to increase performance over merely using a deep network and additionally state that it is theoretically possible to generate hypothesis concerning the feature selection and classification of the deep network.  We do believe that your suggestion would enhance the importance of this paper. However, the additional work that you are proposing (while making for another good paper) is too much to include in this study.  We have attempted to add a few words to clarify and explain some of the wording in the paper surrounding this topic.  Thank you for your review and feedback.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

I might miss something from this manuscript when I reviewed this paper for the first time. Now I think the paper may be helpful for the related research community.