Multi-Classifier Fusion for Open-Set Specific Emitter Identification

Round 1

Reviewer 1 Report

REVIEW_2

 

 

Article titled

“Multi-classifier Fusion for Open-set Specific Emitter  Identification”

 

 

Remote Sensing no: 1691224

 

List of Authors:

 

Yurui Zhao, Xiang Wang, Ziyu Lin, Zhitao Huang

 

 

 

The article MDPI Remote Sensing no: 1691224 titled “Multi-classifier Fusion for Open-set Specific Emitter  Identification” has been carefully modified and well revised. The present version of the article includes all remarks found in the reviews. In this way, present version of this article may be finally accepted for publication in MDPI Remote Sensing.

 

Author Response

Thanks for your approval of our work. We really appreciate your previous comments and suggestions. Those positive and constructive comments contribute to improving the quality of our manuscript. Also, we have learned a lot about how to write an article and present our work. Thanks again for your efforts.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors have successfully implemented my former comments in the revised manuscript.

Author Response

Thanks for your positive comments. We really appreciate your previous comments and suggestions which help a lot in improving the quality of our manuscript. Also, we have learned a lot about how to write an article and present our work. Thanks again for your efforts.

Author Response File: Author Response.pdf

Reviewer 3 Report

• Moderate English changes required
• In the Background session, authors should at least write a paragraph.
• Every table or figure should be mentioned in the text. The case in table 1; the authors have not mentioned anywhere in the text what they have presented in table 1

Author Response

We really appreciate your constructive comments. Please see the attachment.

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Multi-classifier Fusion for Open-set Specific Emitter Identification

In this manuscript, authors discussed about open-set specific emitter identification. They adopted three different inputs and fusion methods using multi-classifier fusion. The main aim of the authors is to confirm that the proposed method can avoid the coincidence of feature space, achieve higher accuracy, and recall ratio. As a case study, they used Huanghua Airport

 

Comments

• The authors should include separate session regarding the literature review (preferred immediately after Introduction), using the latest literature on the topic
• Except for 5-6 references, the other references used are old
• Rows 206-2010 are not clear how they flow and there is no detailed explanation
• Abstract; the issue is not clearly stated
• Introduction; introduction should present the thesis statement, and get readers interested in the subject of the essay. Besides this, the introduction does provide sufficient background
• The results are not clearly presented
• The conclusions are not fully supported by the presented results

Final decision: Reject

Reviewer 2 Report

In this paper, the authors proposed a multi-classifier fusion method for the open-set specific emitter identification. The paper is well-written and the results are interesting. However, to make this paper publishable in this journal a few modifications are highly recommended as follows.

1. The problem formulation (page 2) needs more explanation. You can briefly introduce the problem and goals, and then you can start to explain the details.
2. A table of annotations and parameters in necessary.
3. Non-trainable classifier combiners is not described as much as needed. Besides Equations 20-23 have to be explained.
4. The number of references in this manuscript are not sufficient. There many recent works regarding the specific emitter, multiple feature detectors, and multi-classifier fusion. Some papers that I suggest that the authors consider are

[1] Specific emitter identification using IMF-DNA with a joint feature selection algorithm, 2019.

[2] Are covert DDoS attacks facing multi-feature detectors feasible?, 2021.

[3] A lightweight spatial and temporal multi-feature fusion network for defect detection, 2020.

5. Minor: correct the caption of Figure 6.

Reviewer 3 Report

The paper presented a signal emitter classification method that can have potential military and civilian applications. 

The data set for 20 devices isn't sufficient, but it can be ignored at the moment due to the novelty of the work. However, the authors are advised to share the dataset and relevant code in the paper for other researchers to validate the results independently.


The authors are advised to review the relevant literature and add a separate section for related work

some of the related work which I found is given here
https://journals.pan.pl/Content/119421/PDF/32_01952_Bpast.No.69(2)_23.04.21_K1_G_TeX_OK.pdf

https://ieeexplore.ieee.org/abstract/document/9674605 


Results presented in table 1 does not match with figure 13 and 14 for accuracy and recall in softmax. Further explanation is needed

The conclusion and discussion section can be expanded with future research directions

Reviewer 4 Report

REVIEW

 

Article titled

“Multi-classifier Fusion for Open-set Specific Emitter  Identification”

 

 

Remote Sensing no: 1637684

 

List of Authors:

 

Yurui Zhao, Xiang Wang, Ziyu Lin, Zhitao Huang

 

 

1. In this paper the Authors show that specific emitter identification (SEI method) is utilized to recognize device identity with hardware characteristics. Considering the demand for identifying unknown devices, it seems to be very important. As the Authors claim, this method based on multi-classifier fusion is initialized by adopting three different inputs, and four different fusion methods. The proposed method can avoid the coincidence of feature space and achieve higher accuracy and recall ratio.

 

2. The Authors in Introduction did extensive literature review dealing with the most important aspects about specific emitter identification and verification, identification based on bi-spectra, specific emitter identification via convolutional neural networks and probabilistic neural networks, technology of communication emitters fingerprint features extracting and classifying, deep residual learning for image recognition, and radio frequency fingerprint extraction based on feature inhomogeneity. Just more recent papers from national research community instead of the original international works are cited. I am sorry to say that the list of cited publications is not very extensive.

There has been plenty of work on SEI methods since 2000, including surveys, applications of modern SEI models and methods (fractal features extraction, out-of-band emission and so,  to the problem at hand and performing signal classification and identification in shared spectrum and real-world data . The Authors are not citing any of these prior art or state of the art works.

 

3. I am forced to draw attention to a very important substantive aspect. Namely, the Authors of this article write about Specific Emitter Identification in the aspect of multi classifier fusion. As it is commonly known (also taking into consideration the pattern recognition, classification, and identification), in hardware characteristics classification there is a process of hardware classification and one more process, which is more complicated, namely a process of recognition called the “identification” process. Recognition concerns hardware type classification, whereas identification means recognition of particular copies of the same hardware types. Such an approach is called Specific Emitter Identification (SEI). SEI is based on intrapulse-analysis, out-of-band radiation, fractal theory and other methods by which it is possible to extract additional features from the IoT hardware. The aforementioned features are good separation measures in the identification process. The presented theory (SEI, classification, identification) must be precisely described in this article.

In my opinion, considering the different SEI, and classification by the example of occupancy detection, the following articles “The utilization of unintentional radiation for identification of the radiation sources” IEEE Proc. 34 European Microwave Conference EuMC 2004, vol. 2, pp. 777-780, and “Variant of data particle geometrical divide for imbalanced data sets classification by the example of occupancy detection”, Applied Sciences, 2021, 11(11) are also supposed to be listed in the References.

 

 

4. In Section no. 2, the proposed method of SEI is described. The proposed method, presented in Fig. 1 (Workflow of SEI), consists of extracting features and classification. In Section no.3 the framework of classifier fusion for open-set SEI is presented.

What was the criterion for selecting the ReLu and Softmax activation functions in the ResNet Neural Network?

How do the above-mentioned activation functions influence ResNet Neural Network?

 

5. In Section no. 3, full steps of the proposed framework SEI are presented. I wanted to stop for a moment, and think about algorithms (classifier confusion, input data, network, classifier, and combiner).

My question is as follows: What is the computational complexity of these method/algorithm?

Computational complexity calculations should be presented in this article.

 

6. How the “accuracy” in the identification process is calculated?

 

 

In Conclusion, the Authors wrote: “…Also, we just discussed the condition where combiners use the same type of classifiers. For further work, it deserves to investigating whether more classifiers or mixed classifiers can achieve better performance...”.

So my question is: How to compare the obtained results with other known methods?

 

The Authors considered a problem which is relevant and appealing for this journal. However, I cannot recommend the current manuscript for publication unless the current version is corrected. After providing the modifications to the article, the work is supposed to be reviewed once again.