A Modeling Design Method for Complex Products Based on LSTM Neural Network and Kansei Engineering
Round 1
Reviewer 1 Report
The manuscript presents an interesting approach for combining a CP modeling method based on Long Short Term Memory (LSTM) neural network and KE(CP-KEDL).
General: There is a space after the comma in a 35 paragraph " complex development,_" and before the Genetic Algorithm_(GA) in a 117 paragraph
Figure 1. Framework diagram of the CP-KEDL method.
It is difficult to see where the beginning of the framework is. It would be clearer if the framework will have a start and an end.
Paragraph 312: The Kansei pair "Technological-Traditional" is confusing. It needs to be described what was meant by "traditional," a "non-technological aspect, not so advanced or what?
Paragraph 315: 3.3. Acquisition of Perceptual Evaluation Data
According to your report, 12 participants (sample size) participated in a focus group study. This is a small group of participants from a statistical perspective to perform quantitative statistics such as reliability and validity tests. You have 12 participants x 205 representative samples x 3 Kansei pairs. Even if you get a good Cronbach's alpha of 0.987, this could be due to statistical error as your statistical power is low. This approach may only be valid as a confirmatory study after you have already conducted an experimental design with a larger number of subjects. Justify the results of your study with references to similar studies where the sample size was also small.
In a section 5. Conclusions, I miss the mention of study limitations. The small sample size is on of the limitation of this study.
Author Response
Thank you very much for your professional and enlightening comments concerning our manuscript entitled “A Modeling Design Method for Complex Products Based on LSTM Neural Network and Kansei Engineering” (ID: applsci-2010106). Those comments are all valuable and very helpful in revising and improving our article, as well as providing important guidance for our research. We have studied the comments carefully and have made corrections which we hope could meet with approval. Revised portions are marked in “Track Changes” in the manuscript.
Comment 1: General: There is a space after the comma in a 35 paragraph " complex development,_" and before the Genetic Algorithm_(GA) in a 117 paragraph.
Response: Thank you very much for your careful review and sorry for our incorrect writing. It has been corrected, and the full manuscript has been scanned again to find and correct similar writing errors and language errors.
Comment 2: Figure 1. Framework diagram of the CP-KEDL method. It is difficult to see where the beginning of the framework is. It would be clearer if the framework will have a start and an end.
Response: Thank you very much for this helpful comment. We have optimized the frame diagram and added title guidance to make it clearer and easier to read.
Comment 3: Paragraph 312: The Kansei pair "Technological-Traditional" is confusing. It needs to be described what was meant by "traditional," a "non-technological aspect, not so advanced or what?
Response: Just as your professional suggestion pointed, the semantic expression of Kansei pair is inaccurate and may make our reader confusing. Therefore, at the beginning of the experiment, the meaning of perceptual vocabularies had already been further explained to participants to promote them for a more consistent judgment standard of perceptual images. For example, Steady refers to the visual sense of stability, heaviness, and the feeling of not tipping, while Light means the opposite meaning of Steady; Technological refers to the image of future science and technology, while Traditional means the opposite meaning of Technological. The relevant description is also presented in the manuscript, see Section 3.3, lines 350-357.
Comment 4: Paragraph 315: 3.3. Acquisition of Perceptual Evaluation Data. According to your report, 12 participants (sample size) participated in a focus group study. This is a small group of participants from a statistical perspective to perform quantitative statistics such as reliability and validity tests. You have 12 participants x 205 representative samples x 3 Kansei pairs. Even if you get a good Cronbach's alpha of 0.987, this could be due to statistical error as your statistical power is low. This approach may only be valid as a confirmatory study after you have already conducted an experimental design with a larger number of subjects. Justify the results of your study with references to similar studies where the sample size was also small.)
Response: This is really a very professional and insightful comment, which helps us better improve the experimental design and optimize the data validation. Thank you very much.
Although the participants of the evaluation experiment were recruited from experts or expert users, and varied from designers, managers, engineers, and operators, which could help us collect data from different perspectives and avoid the limitations of a single perspective, there may be statistical reliability errors due to the limitation of the small size of participants and the large cost of the questions. Inspired by your suggestion, we refer to the relevant researches with small sample tests, and use the Kendall concordance coefficient (W) to test the consistency of the evaluation standard when experts scored, and to test the reliability of the data collected; At the same time, the retest method is further applied to double verify the reliability of the data. Based on the above work, we revised Section 3.3. Please see Section 3.3, lines 345-349, line 361-390, and Table 3.
Comment 5: In a section 5. Conclusions, It miss the mention of study limitations. The small sample size is one of the limitation of this study.
Response: Thank you very much for your professional reminder and suggestion. We have rewritten this part according to your suggestion. In this section, we discussed the limitations of the article, and pointed out the limitations of the small sample size of experts, together with the discussion of the future work. Please see lines 615-621.
Special thanks for your professional and enlightening comments.
Reviewer 2 Report
What is UAV? and KJ method? We have to understand it without going to reference [34]
In my opinion, a classical book dealing with Neural Networks is necessary in the references.
Reference [41] deals with Genetic Algorithm and not LSTM neural network. Therefore, at page 6, line 241, the reference must be other one.
At page 8, lines 327-329, when dealing with those tests and measures, I think that a reference is necessary.
At page 17, lines 488-489, when dealing with DNN and CNN, I think that a reference is necessary. If the authors accept the suggestion to cite a classical book of Neural Networks, this reference can be useful for every neural network architecture used in the proposal. I suppose that DNN and CNN are from the same packet used as that for LSTM. Please, improve the information about them. According to the literature, DNN is recommended for image processing. Therefore, I ask to the authors if they use the data with images or with numbers according to table 7. The same question is for CNN.
Although RMSE and MSE are very known metrics in the literature, I think that is necessary to cite one reference for those.
Author Response
Thank you very much for your professional and enlightening comments concerning our manuscript entitled “A Modeling Design Method for Complex Products Based on LSTM Neural Network and Kansei Engineering” (ID: applsci-2010106). Those comments are all valuable and very helpful in revising and improving our article, as well as providing important guidance for our research. We have studied the comments carefully and have made corrections which we hope could meet with approval. Revised portions are marked in “Track Changes” in the manuscript.
Comment 1: What is UAV? and KJ method? We have to understand it without going to reference [34].
Response: Thank you for your professional and patient suggestion. We are very sorry for our negligence in this. We have made corrections according to this comment. The full names of the relevant abbreviations have been added to make the meaning clearer.
Comment 2: In my opinion, a classical book dealing with Neural Networks is necessary in the references.
Response: Thank you again for your suggestion. Relevant classical books on neural networks have been quoted for the manuscript, such as Neural networks: a classroom approach[43] in line 265 and Neural networks and learning machines[58] in line 507.
Comment 3: Reference [41] deals with Genetic Algorithm and not LSTM neural network. Therefore, at page 6, line 241, the reference must be other one.
Response: Thank you for your careful and keen correction. We are very sorry for our negligence in the incorrect literature annotation. We have corrected it and scanned the full article to avoid similar errors.
Comment 4: At page 8, lines 327-329, when dealing with those tests and measures, I think that a reference is necessary.
Response: Considering the comments of the other reviewer, Cronbach's alpha is not suitable for testing the reliability and validity of a small number of subjects. Therefore, we refer to the relevant researches of small sample tests, and use the Kendall concordance coefficient (W) to test the consistency of the evaluation standard when experts scored, and to test the reliability of the collected data; At the same time, the test-retest method is further applied to double verify the reliability of the data. In addition, inspired by your suggestion, we have added relevant references for the tests and measures. Based on the above work, we revised Section 3.3. Please see Section 3.3, lines 345-349, line 361-390, and Table 3.
Comment 5: At page 17, lines 488-489, when dealing with DNN and CNN, I think that a reference is necessary. If the authors accept the suggestion to cite a classical book of Neural Networks, this reference can be useful for every neural network architecture used in the proposal. I suppose that DNN and CNN are from the same packet used as that for LSTM. Please, improve the information about them. According to the literature, DNN is recommended for image processing. Therefore, I ask to the authors if they use the data with images or with numbers according to table 7. The same question is for CNN.
Response: Thank you very much for your professional comment In Section 4, we have cited the necessary classical literature for CNN and DNN networks. Both CNN and DNN use the same coded number data in Table 7 as that for LSTM, and we have improved the information about them. At the same time, referring to the relevant researches, the article presents the reasons for choosing CNN and DNN networks as comparative experiments, together with the implementation approaches. As discribed in the ariticle, although CNN is considered to have good image processing ability, it has been pointed out that CNN also performs excellent results in processing matrix data[62], and has been verified and applied by relevant researches[63]. As a well-known basic traditional machine learning tool[59], DNN has been applied in many controlled experiments to validate the efficiency of researches[59-61]. Therefore, to verify the effectiveness of the proposed KE-LSTM model, DNN and CNN were chosen to be the compared models.
Comment 6: Although RMSE and MSE are very known metrics in the literature, I think that is necessary to cite one reference for those.
Response: Thank you very much for your suggestion. We have added related references for the relevant content.
Special thanks for your professional and enlightening comments.
Round 2
Reviewer 1 Report
Dear authors,
It is obvious that a manuscript has been improved. The methods are now clearly presented and with the help of the pictures the research process is easier to understand. Nevertheless, I notice some spelling mistakes (check words in bold):
Paragraph 269 and 270: In a sentence "The training process attempts to establish a deep learning model between the modeling feature data extracted by the designer in Section 2.5 and the perceptual evaluation values of CPs by learning the training set data". You intend to delete the word "designer." Be careful how this would affect the meaning of the sentence.
Paragraph 368-370: Sentence: "Although an expert evaluation method was used for data acquisition, there may be statistical relia bility errors due to the limitation of the small size of participants and the large cost of the questions". Did you mean questionnaires?
Author Response
Thank you very much for your letter and for the quick response to our manuscript entitled “A Modeling Design Method for Complex Products Based on LSTM Neural Network and Kansei Engineering” (ID: applsci-2010106). According to the valuable comments, we have made corrections which we hope could meet with approval. Revised portions are marked in “Track Changes” in the manuscript.
Reviewer :
Comment 1: Paragraph 269 and 270: In a sentence "The training process attempts to establish a deep learning model between the modeling feature data extracted by the designer in Section 2.5 and the perceptual evaluation values of CPs by learning the training set data". You intend to delete the word "designer." Be careful how this would affect the meaning of the sentence.
Response: Thank you very much for this helpful comment. We are very sorry for our incorrect writing. Here, we refer to the experienced designers mentioned in Section 2.5 and relevant contents have been corrected. Please see line 271.
Comment 2: Paragraph 368-370: Sentence: "Although an expert evaluation method was used for data acquisition, there may be statistical reliability errors due to the limitation of the small size of participants and the large cost of the questions". Did you mean questionnaires?
Response: Thank you very much for your careful review and sorry for our negligence in this. The question here actually means that experts need large cost of time and energy to fill in the questionnaire. We have modified the expression to make its meaning more clear. Please see line 369-370.
Round 3
Reviewer 1 Report
Dear authors,
Both comments have been considered. I have no further comments. I wish you success in your further research.
Author Response
Dear Reviewer:
Thank you very much for your professional and enlightening comments.
