Sub-Health Identification of Reciprocating Machinery Based on Sound Feature and OOD Detection

Round 1

Reviewer 1 Report

The authors should make revise on some points as follows:

1-The proposed method should add more experimental result on publicly dataset on the same field.

2-The comparison of the proposed method and other method should provide 

3-Where is relevant of Fourier transformation in the proposed method ?

Author Response

Dear Editors and Reviewers:

On behalf of my co-authors, we thank you very much for giving us an opportunity to revise our manuscript, we appreciate editor and reviewers very much for their positive and constructive comments and suggestions on our manuscript entitled “Sub-health Identification of Reciprocating Machinery Based on Sound Feature and OOD Detection” (ID: machines-1337515). Those comments are all valuable and very helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied comments carefully and have made correction which we hope meet with approval. Revised portion are marked in red in the revised manuscript. Our hope in the current revision is that both editors and reviewers may find the responses convincing and substantial. The main corrections in the paper and the responds to the reviewer’s comments are as follows:

Detailed Comments

• The proposed method should add more experimental result on publicly dataset on the same field.

Response:

Point taken and thanks. Thanks for reviewer’s insight comments. Many research institutions have shared their mechanical fault diagnosis data sets, such as Case Western Reserve University, Universität Paderborn, University of Connecticut, National Aeronautics and Space Administration, etc. But these data sets are all data sets for bearing fault detection based on vibration characteristics. The data set based on sound characteristics is only the data set provided by DCASE 2020 Task 2. This data set only contains the sound of the machine working normally and the sound after damage, so this task is just a simple fault identification. At present, there is no corresponding research on the sub-health state recognition we proposed, but the network we proposed can perform fault recognition. This is only a part of the function of our entire network. We can separate this part and compare with the baseline of DCASE 2020 Task 2.

Table 1. Comparison of mechanical fault identification module and baseline system.

• The comparison of the proposed method and other method should provide 

Response:

Thank you for your valuable and constructive comments. In order to further prove the performance of our proposed method, we added a comparison with two classic neural network models, which are AlexNet [1] and ResNet [2] respectively. The comparison results are shown in Table 2. PS: There are many classic neural networks, but each type of network needs to be debugged for too long, and 10 days is not enough to complete it. Therefore, we selected two models that are similar to ours and compared them. We hope to get you understanding.

Table 2. Comparison of the proposed method with other methods.

Thanks again for the reviewer’s good question.

[1] A. Krizhevsky, I. Sutskever and G. E. Hinton, "ImageNet classification with deep convolutional neural networks", Commun. ACM, vol. 60, no. 6, pp. 84-90, 2017. doi: 10.1145/3065386.

[2] K. He, X. Zhang, S. Ren and J. Sun, "Deep residual learning for image recognition", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 770-778, 2016. doi: 10.1109/CVPR.2016.90

• Where is relevant of Fourier transformation in the proposed method ?

Response:

Thank you for your valuable and constructive comments. We mentioned it briefly on line 163. I'm very sorry that I didn't explain it clearly. The entire model can be divided into two parts: audio preprocessing and deep learning network. Audio preprocessing converts the audio signal into Mel Frequency Cepstral Coefficient. The process includes pre-emphasis, sub-frame, add a window and fast Fourier transform. Since the transformation of the signal in the time domain is usually difficult to see the characteristics of the signal, it is usually converted to the energy distribution in the frequency domain for observation. Different energy distributions can represent the characteristics of different voices. Therefore, after the a window, each frame must be subjected to fast Fourier transform to obtain the energy distribution on the frequency spectrum, and then further features can be extracted. In summary, Fourier transformation is a step of the entire model, the purpose is to obtain the energy distribution on the frequency spectrum.

The description of the role of Fourier transformation in the model have been added to the resubmission manuscript.

Special thanks to your comments and suggestions, I hope you can recognize our responses and corrections.

We appreciate for Editors/Reviewers’ warm work earnestly, and hope that the correction will meet with approval.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper proposes to use deep convolutional neural networks to extract sound features and use Out-of-Distribution (OOD) detection technique to recognize mechanical part or equipment sub-health state. The sound signals recorded from a machine using an array of four microphones are used for model training and machine sub-health state detection.  

The review comments are given below: 

1. Review comments in detail are provided in the attached pdf file. The authors are required to make careful revisions by following the comments given.
2. Since fault detection techniques using vibration and acoustic signals are mature, it is suggested to give a brief review of these available techniques and make a comparison among them in Introduction. From this brief review of the existing techniques and approaches, the readers can easily understand why the proposed approach is utilized in this paper.
3. It is suggested to give the machine sub-health state detection performance indicator values using the baseline models so that the readers can make a better comparison between the proposed one and the baseline model.

Comments for author File: Comments.pdf

Author Response

Dear Editors and Reviewers:

On behalf of my co-authors, we thank you very much for giving us an opportunity to revise our manuscript, we appreciate editor and reviewers very much for their positive and constructive comments and suggestions on our manuscript entitled “Sub-health Identification of Reciprocating Machinery Based on Sound Feature and OOD Detection” (ID: machines-1337515). Those comments are all valuable and very helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied comments carefully and have made correction which we hope meet with approval. Revised portion are marked in red in the revised manuscript. Our hope in the current revision is that both editors and reviewers may find the responses convincing and substantial. The main corrections in the paper and the responds to the reviewer’s comments are as follows:

Detailed Comments

• Review comments in detail are provided in the attached pdf file. The authors are required to make careful revisions by following the comments given.

Response:

Point taken and thanks. Please accept the most sincere thanks from authors from non-English speaking country. We have modified them one by one according to the annotations in the PDF, and you can check the attachments for details.

• Since fault detection techniques using vibration and acoustic signals are mature, it is suggested to give a brief review of these available techniques and make a comparison among them in Introduction. From this brief review of the existing techniques and approaches, the readers can easily understand why the proposed approach is utilized in this paper.

Response:

Thank you for your valuable and constructive comments. The second paragraph of the introduction has been rewritten by us. We first reviewed the fault detection methods based on the characteristics of vibration signals, then reviewed the development of sound detection, and finally proposed the applicable environment of the two methods in fault detection. After rewriting in this way, readers can easily understand that heavy-duty reciprocating equipment has the characteristics of high stability and relatively loud sound, and the use of sound signal characteristics can achieve better fault recognition results. The specific description can be viewed in the attachment.

Thanks again for the reviewer’s good question.

• It is suggested to give the machine sub-health state detection performance indicator values using the baseline models so that the readers can make a better comparison between the proposed one and the baseline model.

Response:

Thank you for your valuable and constructive comments. We added the description of two key indicators of the machine sub-health state detection performance in line 304. These two indicators are AUC and pAUC.

Special thanks to your comments and suggestions, I hope you can recognize our responses and corrections.

We appreciate for Editors/Reviewers’ warm work earnestly, and hope that the correction will meet with approval.

Author Response File: Author Response.pdf

Reviewer 3 Report

The article presents a method for detecting failure and sub-failure modes in machinery using sound data. The authors propose to build a system of convolutional neural networks that can capture these states from data and the learned behaviors can be tested for state assignment. The concept is novel and the methodology appears to be sound.

While I am generally positive about the study, the paper need MAJOR revision simply based on language. The language is substandard and at times confusing and unclear.

In addition, the paper can be vastly improved if more information is presented in the analysis of the experimental results. I find the Table with AUC values to be inadequate representation of the performance. TPR and FPR can be provided? Time of detection?

Only one channel is used for this study. Is there an inherent limitation to use multiple channels?

Is there any possibility of interference of correlation coming from using multiple microphones located around the machine? How are they placed?

What is the rationale behind choosing the filter parameters?

Line 201, reference for Hawkins?

Use of Mahalanobis distance. Can correlation be used instead?

The paper is not publishable at this point.

Author Response

Dear Editors and Reviewers:

On behalf of my co-authors, we thank you very much for giving us an opportunity to revise our manuscript, we appreciate editor and reviewers very much for their positive and constructive comments and suggestions on our manuscript entitled “Sub-health Identification of Reciprocating Machinery Based on Sound Feature and OOD Detection” (ID: machines-1337515). Those comments are all valuable and very helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied comments carefully and have made correction which we hope meet with approval. Revised portion are marked in red in the revised manuscript. Our hope in the current revision is that both editors and reviewers may find the responses convincing and substantial. The main corrections in the paper and the responds to the reviewer’s comments are as follows:

Detailed Comments

• While I am generally positive about the study, the paper need MAJOR revision simply based on language. The language is substandard and at times confusing and unclear.

Response:

Point taken and thanks. We are very sorry that our English writing ability is terrible. We corrected 52 errors in the paper with the help of enthusiasts, and the second paragraph of the introduction was rewritten. For details, please refer to the red part of the attachment. Hope to get your understanding again, thank you.

• In addition, the paper can be vastly improved if more information is presented in the analysis of the experimental results. I find the Table with AUC values to be inadequate representation of the performance. TPR and FPR can be provided? Time of detection?

Response:

Thank you for your valuable and constructive comments. Table 1,2and3 were added to the resubmitted manuscript.

Table 1. TPR and FPR results of three experiments. From top to bottom are VGG16 network, VGG16+OOD network, VGG16+OOD+ auxiliary network. All values are in %.

Table 2. Comparison of mechanical fault identification module and baseline system of DCASE 2020 Task 2. All values are in %.

Table 3. Comparison of the proposed method with other methods. All values are in %.

Thanks again for the reviewer’s good question.

• Only one channel is used for this study. Is there an inherent limitation to use multiple channels?

Response:

Thank you for your valuable and constructive comments. There is no an inherent limitation to use multiple channels. We collect data in a multi-channel way, mainly for future research. The multi-channel can filter the sound waves by the difference between the phases of the sound waves, which can eliminate the environmental background sound to the greatest extent, leaving only the required sound waves. Our experiment needs to retain part of the environmental noise, so as to restore the real mechanical operation scene. So we only use single channel data.

 Is there any possibility of interference of correlation coming from using multiple microphones located around the machine? How are they placed?

Response:

Thank you for your valuable and constructive comments. The data set was collected using the TAMAGO-03 microphone manufactured by System In Frontier Inc. The performance of the microphone ensures that the use of multiple microphones around the machine is unlikely to cause related interference.

Figure 1 in the paper is the placement and distribution of microphones. I'm sorry for not explaining very clearly, so I added a detailed description. The four microphones form a square with a side length of 28 cm. The sound source is located at the center of the square. The distance from each microphone to the sound source is about 20 cm.

• What is the rationale behind choosing the filter parameters?

Response:

Point taken and thanks. The first-order FIR high-pass filter is to emphasize and increase the high-frequency part of the signal and eliminate the influence parts such as "lip radiation". The default range of the pre-emphasis coefficient is 0.9~1. The specific value is adjusted through continuous parameter adjustment, and finally it is determined that 0.98 is the most effective.

• Line 201, reference for Hawkins?

Response:

Thanks for your professional comments. Line 201, reference for Hawkins. I'm sorry I didn't mark it. It was marked in the resubmitted manuscript.

• Use of Mahalanobis distance. Can correlation be used instead?

Response:

Point taken and thanks. It is a good idea. But after experimentation, it was found that the effect was very bad, and the results are shown in Table 4.

Table 4. Summary of results. Only show the recognition results of sub-health data.

Special thanks to your comments and suggestions, I hope you can recognize our responses and corrections.

We appreciate for Editors/Reviewers’ warm work earnestly, and hope that the correction will meet with approval.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The revision manuscript satisfied my comments.  

Reviewer 3 Report

Thank you for responding to my comments.