Pulsar Candidate Recognition Using Deep Neural Network Model
Round 1
Reviewer 1 Report
This paper is submitted as a contribution to computer science and machine learning. The authors employ a set of existing computer vision tools and apply it to the pulsar classification problem. This is not identified as a contribution to the astronomy literature and the novelty of this work w.r.t. machine learning literature is not clear to me. I suggest the authors elaborate on which ways they have contributed to the field of computer vision.
Pulsar classification, in particular, is an easy problem and might not be the best candidate to demonstrate the application of this model (if this is the aim of this work). If the aim is to develop a novel classification algorithm for astronomers, then the comparisons are not appropriate. The baseline candidates are selected randomly. It is important to compare this model with the model that is already established in the astronomy literature and used by astronomers. I am wondering how much improvement they get upon the astronomer's baseline?
Since this work requires reperforming the experiments and major changes in the experimental design, I recommend rejection. The authors should consider resubmitting the work as a new manuscript.
Author Response
Response to Reviewer 1 Comments
Point 1: This paper is submitted as a contribution to computer science and machine learning. The authors employ a set of existing computer vision tools and apply it to the pulsar classification problem. This is not identified as a contribution to the astronomy literature and the novelty of this work w.r.t. machine learning literature is not clear to me. I suggest the authors elaborate on which ways they have contributed to the field of computer vision. Pulsar classification, in particular, is an easy problem and might not be the best candidate to demonstrate the application of this model (if this is the aim of this work). If the aim is to develop a novel classification algorithm for astronomers, then the comparisons are not appropriate. The baseline candidates are selected randomly. It is important to compare this model with the model that is already established in the astronomy literature and used by astronomers. I am wondering how much improvement they get upon the astronomer's baseline? Since this work requires reperforming the experiments and major changes in the experimental design, I recommend rejection. The authors should consider resubmitting the work as a new manuscript.
Response 1: We sincerely thank the reviewer for very valuable comments, which helped us improve the quality of our paper. We feel apologetic about that we didn't explain our research motivation clearly. We have taken the critiques very seriously and reworked the manuscript by checking more literature and studying relevant articles. We agree that we should provide enough explanation. We have now gave the explanation as follows. Your interpretation of the paper is correct. We designed a new network model for pulsar recognition task based on neural network technology, which aims to improve the accuracy of pulsar recognition and help astronomers screen pulsars efficiently. But the baselines are not selected randomly. The manual matching method is used to screen pulsars at an early stage. The database matching method is used in the existing processing software that already established in the astronomy literature and used by astronomers [1]. The recognition efficiency is very low, and there is no exact accuracy to prove its performance. Previous work has proved that pulsar recognition methods are constantly evolving to improve efficiency and ensure accuracy. The pulsar recognition methods in recent years are listed in Table 1 of this article [2]. Compared with other models, the methods based on neural network have advantages because these methods avoid being influenced by human factor and reduces the number of features. And then, neural network for images processing has gradually become a hot one in the field of pulsar recognition, and it is also the focus of research in this paper. However, the data set of PMPS is difficult to handle, such methods are still in the process of development, and the performance is slightly insufficient. Thus, we focus on comparing the excellent pulsar recognition methods based on neural network proposed in recent years. Especially, the experimental results show that our method performs better. In addition, we have added the explanation from line 99 to line 103 in Section 1 on page 2 to page 3, as follows.
Compared with the other models, the methods based on neural network have advantages because these methods avoid being influenced by human factor and reduces the number of features. And then, neural network for images processing has gradually become a hot one in the field of pulsar recognition, and it is also the focus of research in this paper.
[1] Wang Y, Zheng J, Pan Z, et al. An overview of pulsar candidate classification methods[J]. Journal of Deep Space Exploration, 2018,5(03):203-211+218.DOI:10.15982/j.issn.2095-7777.2018.3.001. (in chinese)
https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2018&filename=SKTC201803001&uniplatform=NZKPT&v=ZtEfR74XuSIzSYdffrtu2_NAK5qKaWbIchDzeotsZXeX85f2ZeCxrV_7GkN1pqkd
[2] Xu Y, Zhang X, Liu Z, et al. Implementation of a Pulsar Candidate Verification Method[J]. Value Engineering, 2017,36(16): 172-175.DOI:10.14018/j.cnki.cn13-1085/n.2017.16.071. (in chinese)
https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2017&filename=JZGC201716071&uniplatform=NZKPT&v=ckXha3Gmja0gocxjRPP0Ca0iFgdHwfNB395aILccqxweRv-AmMB1DmwEOFPGv6dL
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The authors proposed AR net-based pulsar candidate recognition method. The manuscript is adequately written and must be improved for publication in this Journal
1. How novel is the information on binary classification? Do the authors think it requires a table for the audience of electronics to understand? Please clarify and improve.
2. The authors did not provide any information on hyperparameter tuning? is there any special reasons for it? Please clarify.
3. Technically, it is debatable to introduce a new method when ResNet is providing us with fair results. The authors should provide a brief commentary on it and how their proposed method helps the scientific community in comparison to ResNets?
4. Authors should introduce a brief paragraph on the drawbacks of the proposed method.
Author Response
Response to Reviewer 2 Comments
Point 1: How novel is the information on binary classification? Do the authors think it requires a table for the audience of electronics to understand? Please clarify and improve.
Response 1: We sincerely thank the reviewers for very valuable comments/feedback, which helped us improve the quality of our paper. We agree that we did not provide enough explanation. We have now added one small paragraph in Section 3.1 on page 6 and we also added a table in Section 3.1 on page 7, as follows.
The features used in the existing pulsar recognition methods are shown in the table 1. The subjective factors and limitations exist in the empirical features and statistic features, and numerous features were required. The image features are automatically extracted by convolution neural network when we solve the problem on angle of intelligent image processing, which can avoid being influenced by human factor and reduces the number of features. At the initial stage of development, the researchers designed integrated model to extract the features of four subgraphs. The integrated model is complex. After that, the model is continuously optimized, and only a single subgraph is used as the judgment basis. The model is simpler and the number of features is smaller, which has more advantages for the task of processing massive data.
Table 1. The features used in different Classifier.
|
Classifier |
Number of features |
Type of features |
|
ANN(Bates et al.) |
22 |
Empirical feature |
|
ANN(Eatough et al.) |
12 |
Empirical feature |
|
SPINN |
6 |
Empirical feature |
|
GH-VFDT、Fuzzy KNN |
8 |
Statistic feature |
|
PEACE |
6 |
Statistic feature |
|
PICS、PICS-ResNet |
4 |
Image features |
|
DCGAN-SVM、ResNet、AR-Net |
1 |
Image features |
Point 2: The authors did not provide any information on hyperparameter tuning? is there any special reasons for it? Please clarify.
Response 2: We thank the reviewer for mentioning this important point. We agree that we did not make it clarify and should provide enough explanation in the manuscript. We have now made modification as follows.
1. We have added the explanation from line 237 to line 238 in Section 3.3 on page 7, as follows.
Due to the size of input feature map is small, the size of all convolution kernel was set as 3×3.
2. We have added the explanation from line 242 to line 245 in Section 3.3 on page 8, as follows.
The purpose of introducing pooling layer is to retain larger response values and decrease resolution. Generally, the size of the poolling kernel is also set to a smaller value, 2x2 is in common use.
3. We have added the explanation from line 248 to line 259 in Section 3.3 on page 8, as follows.
Experimental parameter setting are as follows. If the value of Batch_Size is large, the model is easy to encounter local minimization, If the value of Batch_Size is small, the randomness will be greater and the training effect will be better. It is necessary to make adaptation adjustment according to the current task to obtain the best value, and it is usually set as the N-th power of 2. After many experiments, it is found that if Batch_Size is set as 16, the result is the best. The value of learning rate is crucial, the parameters are initialized randomly at the beginning of training, and the learning rate is set to 0.001, model can regulate the parameters rapidly. During the training process, the learning rate is dynamically adjusted, it changes to one tenth of the original value each time, and the optimal parameters are gradually found. The training results of the model show that when the training iterations reach 100, the fitting effect is well and almost have no change in later training. Consequently, the value of epochs is set as 100.
Point 3: Technically, it is debatable to introduce a new method when ResNet is providing us with fair results. The authors should provide a brief commentary on it and how their proposed method helps the scientific community in comparison to ResNets?
Response 3: We sincerely thank the reviewer for the suggestion. We agree that we should better explain this point. We have added a brief commentary from line 328 to line 333 in Section 4 on page 10, as follows.
In addition, network structure of ResNet has indeed achieved amazing results, but by comparison, AR_Net model is more helpful to extract crucial information. For image data with fuzzy features and prominent local features, such as pulsar subgraphs, network structure of AR_Net has more advantages, and the accuracy is higher, so it is worth popularizing to similar image processing tasks.
Point 4: Authors should introduce a brief paragraph on the drawbacks of the proposed method.
Response 4: We thank the reviewer for this thoughtful and reasonable suggestion. We have now revised and added the words from line 334 to line 339 in Section 4 on page 10 to elaborate the limitation of the proposed method, as follows.
However, the feature image was required in the model training process needs to be normalized, and the parameters of the model need to be further reduced. Moreover, to a certain extent, the efficiency of neural network recognition model depends on the integrity of labeled data set. The existing data set lacks a large number of labeled posi-tive sample data. In the future, unsupervised learning algorithm should be designed to alleviate the dependence of the model on label data.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
The authors clarified all the suggestions. I recommend it to accept for publication.
