Diagnosis of Autism Spectrum Disorder Using Convolutional Neural Networks
Round 1
Reviewer 1 Report (Previous Reviewer 1)
The manuscript has significantly improved with respect to the first version.
A few more minor comments:
-Figure 1 should be improved in quality as it is a bit blurred and not very readable
-Equations 1/2/3/4 should be rewritten in maths form
-Conclusions should be enlarged with further description and information concerning the conclusions drawn
Author Response
Reviewer 1
The manuscript has significantly improved with respect to the first version.
A few more minor comments:
- Figure 1 should be improved in quality as it is a bit blurred and not very readable
Response: Figure 1 has been updated accordingly with a better resolution for improved readability
-Equations 1/2/3/4 should be rewritten in math form
Response: Equations 1,2,3, and 4 have been rewritten in math format
-Conclusions should be enlarged with further description and information concerning the conclusions drawn
Response: The conclusion has been rewritten to be more informative based on the drawn conclusion
Author Response File: 
Author Response.pdf
Reviewer 2 Report (New Reviewer)
This is an interesting research related to ASD. The idea of using deep learning to diagnose ASD based on handwriting is considered as the novelty of current study. However, there are some issues to be addressed. Please find my comments as follows:
1. Line 17 - It seems more appropriate to replace "machine learning" with "deep learning".
2. Lines 29-30 - Result of F1 score is missing
3. Please ensure all keywords provided are reflected in the abstract.
4. Section I - First paragraph is too lengthy as compared with other paragraphs. Please reorganize the contents into more than one paragraphs with proper length.
5. GoogleNet is not something new. Why it is chosen to solve this problem? Why not more sophisticated pretrained network such as XceptionNet, InceptionNet, DarkNet and etc? More justifications are needed.
6. Section 4 Dataset and Preprocessing should be presented before the deep learning part.
7. The datasets created seem too small. How the authors ensure there will be no overfitting issue on the network?
8. How the authors can ensure the hyperparameter settings in Lines 358 and 359 are optimal for their task?
9. Why only accuracy values are reported in Table 1? How about other metrics such as sensitivity, specificity and F1 score? These results should be reported for more comprehensive comparisons.
10. It is quite surprise to see GoogleNet can outperform other more advanced pretrained networks such as ResNet, VGG and SqeezeNet. Authors should provide further elaborations to justify this finding.
11. In my opinion, comparison in Table 2 seems not relevant because different datasets and different patients are considered in different studies. How to ensure the comparisons are fair and relevant?
Author Response
Reviewer 3
- Line 17 - It seems more appropriate to replace "machine learning" with "deep learning".
Response: The term has been replaced as advised
- Lines 29-30 - Result of F1 score is missing
Response: The F1 score has been included in the abstract.
- Please ensure all keywords provided are reflected in the abstract.
Response: Al keywords have been reviewed and ensured they are reflected in the abstract
- Section I - The first paragraph is too lengthy compared with other paragraphs. Please reorganize the contents into more than one paragraphs with proper length.
Response: The first paragraph has been reformatted accordingly
- GoogleNet is not something new. Why it is chosen to solve this problem? Why not more sophisticated pretrained network such as XceptionNet, InceptionNet, DarkNet and etc? More justifications are needed.
Response: In this paper, we tried to test some of the well-known and robust models such as GoogleNet, Squeeze net, Resnet and etc. While there can and will be more accurate models than tested, it is shown that detecting autism spectrum disorder is possible with high accuracy using the handwriting of subjects which is a novel and practical approach.
- Section 4 Dataset and Preprocessing should be presented before the deep learning part.
Response: The section has been presented as advised before the deep learning part.
- The datasets created seem too small. How do the authors ensure there will be no overfitting issue on the network?
Response: The study acknowledges the data limitation of the research, hence the proposed use of transfer learning to enable optimal learning, and also the hyperparameters of the experimental setup were carefully chosen following preliminary experiments.
- How the authors can ensure the hyperparameter settings in Lines 358 and 359 are optimal for their task?
Response: The specified hyperparameters settings in this manuscript have been deemed optimal through variations of hyperparameters settings during preliminary experiments to ensure overfitting is avoided.
- Why only accuracy values are reported in Table 1? How about other metrics such as sensitivity, specificity, and F1 score? These results should be reported for more comprehensive comparisons.
Response:
- It is quite surprise to see GoogleNet can outperform other more advanced pretrained networks such as ResNet, VGG and SqeezeNet. Authors should provide further elaborations to justify this finding.
Response: In the literature, a lot of well-known models are trained and reported results using large datasets such as ImageNet which includes more than a million images. We believe that the more advanced pre-trained networks such as ResNet would perform better when larger datasets are available. But in the case of small datasets, they may be more prone to overfitting since they tend to have more trainable parameters.
- In my opinion, the comparison in Table 2 seems not relevant because different datasets and different patients are considered in different studies. How to ensure the comparisons are fair and relevant?
Response: The comparison table was requested by other reviewers, hence to make it objective we included the methodology of each research in the comparison table. The proposed approach is novel and there is no other paper/approach using handwriting as input to detect ASD.
Author Response File: Author Response.pdf
Reviewer 3 Report (New Reviewer)
1. The abstract should be broadened to give additional quantitative results.
2. As your abstract's final sentence, include a "take-home" message.
3. Reorder keywords based on alphabetical order.
4. Novelty in the current study's is too weak. The past has seen an extensive study of a lot of written material. It is required to provide more details for more explanation about the present novel in the introductory section.
5. To underline the study gaps that the newest research tries to fill, it is crucial to explain the merits, novelty, and limits of earlier studies in the introduction.
6. In the last paragraph of the introduction, please explain the objective of the present article.
7. Previous study of autism spectrum disorder sensor based, one of the is from Afif et al., needs to be explained. Furthermore, the MDPI's suggested reverence should be applied in the manner described below to further support this description as follows: Physiological Effect of Deep Pressure in Reducing Anxiety of Children with ASD during Traveling: A Public Transportation Setting. Bioengineering 2022, 9, 157. https://doi.org/10.3390/bioengineering9040157
8. Instead of only using the dominating text as a present form, the authors should also include extra illustrations in the form of figures that clarify the workflow of the current study to make the reader's understanding simpler.
9. More detail in tools information such as manufacturer, country, and specification needs to be stated.
10. Valuable information that must be included in the publication refers to the inaccuracy and intolerance of the experimental setup used in this inquiry.
11. A comparative assessment with similar previous research is required.
12. The authors need to improve the discussion in the present article become more comprehensive. The present form was insufficient.
13. Please include the limitation of the present study, it is missing.
14. Please discuss the further research in the conclusion section.
15. The authors should give additional references from the five-years back. MDPI reference is strongly recommended.
16. English needs to be proofread due to grammatical errors and English style.
17. Graphical abstract is encouraged to provide in submission after review.
Author Response
Reviewer 4
- The abstract should be broadened to give additional quantitative results.
Response: The abstract has been edited to provide better information about the research paper
- As your abstract's final sentence, include a "take-home" message.
Response: The final take-home message has been added accordingly
- Reorder keywords based on alphabetical order.
Response: The keywords have been reordered alphabetically
- Novelty in the current study's is too weak. The past has seen an extensive study of a lot of written material. It is required to provide more details for more explanation about the present novel in the introductory section.
Response: The novelty of the work has been reemphasized in the introductory section in the study contribution section.
- To underline the study gaps that the newest research tries to fill, it is crucial to explain the merits, novelty, and limits of earlier studies in the introduction.
Response: The research gaps have been outlined in the study contribution section.
- In the last paragraph of the introduction, please explain the objective of the present article.
Response: The objectives of the article have been added as suggested
- Previous study of autism spectrum disorder sensor based, one of the is from Afif et al., needs to be explained. Furthermore, the MDPI's suggested reverence should be applied in the manner described below to further support this description as follows: Physiological Effect of Deep Pressure in Reducing Anxiety of Children with ASD during Traveling: A Public Transportation Setting. Bioengineering 2022, 9, 157. https://doi.org/10.3390/bioengineering9040157.
Response: This has been treated as advised.
- Instead of only using the dominating text as a present form, the authors should also include extra illustrations in the form of figures that clarify the workflow of the current study to make the reader's understanding simpler.
Response: A diagram illustrating the workflow of the study has been provided with an explanation.
- More detail in tools information such as manufacturer, country, and specification needs to be stated.
Response: The information of the data collection tool has been added
- Valuable information that must be included in the publication refers to the inaccuracy and intolerance of the experimental setup used in this inquiry.
Response:
- A comparative assessment with similar previous research is required.
Response: A comparative assessment with previous works has been provided in the discussion section
- The authors need to improve the discussion in the present article to become more comprehensive. The present form was insufficient.
Response: Discussion section has been enriched with more objective points relevant to the study.
- Please include the limitation of the present study, it is missing.
Response: Limitation of the study has been included
- Please discuss further research in the conclusion section.
Response: The conclusion section has been enriched with further discussion.
- The authors should give additional references from the five-years back. MDPI reference is strongly recommended.
Response: More relevant references has been added to enrich the theoretical background of the sudy.
- English needs to be proofread due to grammatical errors and English style.
Response: The manuscript has been edited for grammatical errors and english style
- Graphical abstract is encouraged to provide in submission after review.
Response: A graphical abstract will be provided in submission after review as advised.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report (New Reviewer)
The authors have addressed most of the comments. Only the response for Comment 9 is not provided. No other further comments. However, authors should highlight all changes made in their revised manuscript so that reviewer can trace the changes made easily.
Reviewer 3 Report (New Reviewer)
I am nothing have any further comments for this manuscript.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
- in the abstract you mention transfer learning as something that it has been applied . Please specify how and in which phase of the algorithm development
-the perfomances reported in the abstract, are the result of a cross validation approach?
-More space should be devoted in the introduction also to introduce the importance of machine learning to the biomedical applications. This is only left at the final part, and more information and relevant references should be added. For example:
- Dimitri, Giovanna Maria, et al. "A multiplex network approach for the analysis of intracranial pressure and heart rate data in traumatic brain injured patients." Applied network science 2.1 (2017): 1-12
-Wang, Fei, Lawrence Peter Casalino, and Dhruv Khullar. "Deep learning in medicine—promise, progress, and challenges." JAMA internal medicine 179.3 (2019): 293-294
-Bianchini, Monica, et al. "Deep neural networks for structured data." Computational Intelligence for Pattern Recognition. Springer, Cham, 2018. 29-51
-Khodatars, Marjane, et al. "Deep learning for neuroimaging-based diagnosis and rehabilitation of autism spectrum disorder: a review." Computers in Biology and Medicine 139 (2021): 104949
-Shoeibi, Afshin, et al. "Epileptic seizures detection using deep learning techniques: a review." International Journal of Environmental Research and Public Health 18.11 (2021): 5780
-Cao, Chensi, et al. "Deep learning and its applications in biomedicine." Genomics, proteomics & bioinformatics 16.1 (2018): 17-32
-Figure 1 is unreadable. Please improve and add further details to the caption
-Please clarify and improve the methodology section. Moreover Figure 2 is unreadable and should be improved consistently
-I would try to collapse figures concerning the various tasks in one, as it is hard to follow the flow with so many figures.
-please add details to the validation procedure used for testing the CNN and also to the data augmentation procedure
-as it is, section 5 should be removed
-Table 1: Define the acronyms
-not clear if the system is overfitting? PLease discuss this
- I think the dataset should be released to enhance the novelty of the work
Author Response
Reviewer 1
- In the abstract you mention transfer learning as something that has been applied. Please specify how and in which phase of the algorithm development
Response: In this study, pretrained GoogleNet has been used and only the last layers are trained again. Thus the algorithm can extract features robustly even with limited amount of training data.
-The performances reported in the abstract, are the result of a cross validation approach?
Response: the reported accuracy in the abstract is the result of the test set. Cross validation has not been used.
-More space should be devoted in the introduction also to introduce the importance of machine learning to biomedical applications. This is only left at the final part, and more information and relevant references should be added. For example:
Response: All the suggested references have been duly added as advised and recommended to further strengthen the literature of the article.
-Figure 1 is unreadable. Please improve and add further details to the caption
Response: Figure 1 has been redrawn and rescaled to enable clear readability.
-Please clarify and improve the methodology section. Moreover Figure 2 is unreadable and should be improved consistently.
Response: The methodology has been improved and Figure 2 has also been redesigned for an improved readability
-I would try to collapse figures concerning the various tasks in one, as it is hard to follow the flow with so many figures.
Response: Figures 3 to 7 have been collapsed as advised to ensure they can be followed with ease while reading
-please add details to the validation procedure used for testing the CNN and also to the data augmentation procedure.
Response: Data is divided into two sets. Training and testing. Data augmentation is applied to the input data. For each epoch, input data is rotated, shifted and reflected randomly.
-as it is, section 5 should be removed
Response: Section 5 has been removed as advised in your review comments
-Table 1: Define the acronyms
Response: the acronyms used in the table have been written completely as advised for a definition
-not clear if the system is overfitting? Please discuss this
Response: Loss values in learning curves (loss vs iteration number) for training and testing sets are not diverging as the training carries on. Thus, over fitting is not expected.
- I think the dataset should be released to enhance the novelty of the work
Response: Although, we collected many samples, due to the pandemic we couldn’t collect as many samples as we desire. We are planning to release the dataset. However, doing so we are planning to collect more samples.
Reviewer 2 Report
Autism spectrum disorder has been a condition which affects people for a very long period of time. The authors in this paper, have proposed machine learning based, automated ASD diagnosis method using handwriting as input. In this approach, several different tasks are given to pupils such as drawing rectangles and these drawings are fed into CNN to diagnose ASD. Authors have claimed that they have used transfer learning to increase accuracy. Also, for each task a different network trained and classification results for the same pupil are combined by taking median of estimated classes (majority voting) by the networks. Consequently, a dataset with 104 pupils (split as 70% for training and 30% for testing) is formed and authors have shown that the proposed approach can correctly classify ASD with an accuracy of 93.55%, where sensitivity, and specificity are calculated as 93.33%, and 93.75% respectively.
The paper is interesting, however, needs minor revision and , I have given the following comments.
1) In the introduction section, you have talked mostly on Autism spectrum disorder and related stuff, however, I would suggest to add one paragraph to discuss your proposed method a bit in the introduction section as well.
2) At the end of the introduction you must mention which section describes what?
such as section 2 represents related work, section 3 represents such as such, section 4 talks about such.....
3) Add one extra objective by mentioning the methods and their obtained results?
4) Methodology representation is weak: you must draw a CNN (proposed) architechture..
5) Write algorithms for all proposed methods(must)
6) In the introduction you must also provide an overall working diagram of your whole process of work
7) Fig 3 to Fi 7 could be accommodated in a single figure.
8) Try to mention about the activation fucntion you have used and mention the equation of it.
9) Preprocessing(data) algorithm is missing, try to write an algorithm for preprocessing.
10) Confusion matrix, F1 SCORE, Auc Curve are finally accuracy table is missing
11) A solid comparative study table is requried where you sould also consider other similar methods like CNN (May be ANN or Dilated CNN). Meaning comparisons must be there to validate your work.
12) A discussion part must be added before the conclusion.
13) You must cite the following papers
a)Lefter, R., Ciobica, A., Timofte, D., Stanciu, C., & Trifan, A. (2019). A descriptive review on the prevalence of gastrointestinal disturbances and their multiple associations in autism spectrum disorder. Medicina, 56(1), 11.
b) Roy, S. S., Rodrigues, N., & Taguchi, Y. (2020). Incremental dilations using CNN for brain tumor classification. Applied Sciences, 10(14), 4915.
c) Xu, K., Feng, D., & Mi, H. (2017). Deep convolutional neural network-based early automated detection of diabetic retinopathy using fundus image. Molecules, 22(12), 2054.
d) Ahmed, I. A., Senan, E. M., Rassem, T. H., Ali, M. A., Shatnawi, H. S. A., Alwazer, S. M., & Alshahrani, M. (2022). Eye Tracking-Based Diagnosis and Early Detection of Autism Spectrum Disorder Using Machine Learning and Deep Learning Techniques. Electronics, 11(4), 530.
e) Balas, V. E., Roy, S. S., Sharma, D., & Samui, P. (Eds.). (2019). Handbook of deep learning applications (Vol. 136). New York: Springer.
Author Response
Reviewer 2
1) In the introduction section, you have talked mostly on autism spectrum disorder and related stuff, however, I would suggest adding one paragraph to discuss your proposed method a bit in the introduction section as well.
Response: The methodology of the research has been duly introduced in the introductory chapter as advised.
2) At the end of the introduction you must mention which section describes what?
Response: The sections of the article have been mentioned and described at the end of the introduction section
3) Add one extra objective by mentioning the methods and their obtained results?
Response: Introduction section is now enhanced,
4) Methodology representation is weak: you must draw a CNN (proposed) architecture.
Response: The CNN of the research is drawn and presented in figure 2 as the architecture of GoogleNet
5) Write algorithms for all proposed methods(must)
Response: Both pre-processing procedure (this was not available before) and training procedures are defined in the paper using block diagrams.
6) In the introduction you must also provide an overall working diagram of your whole process of work
Response: A flowchart has been added to show an overall working diagram of the research.
7) Fig 3 to Fi 7 could be accommodated in a single figure.
Response: Figures 3 to 7 have been collapsed into a single figure and labeled accordingly.
8) Try to mention about the activation function you have used and mention the equation of it.
9) Preprocessing(data) algorithm is missing, try to write an algorithm for preprocessing.
Response: An algorithm for the data preprocessing has been added and a flowchart has also been added labeled as figure 5.
10) Confusion matrix, F1 SCORE, Auc Curve are finally accuracy table is missing
Response: A confusion matrix and F1 score has been added to the article accordingly.
11) A solid comparative study table is required where you should also consider other similar methods like CNN (May be ANN or Dilated CNN). Meaning comparisons must be there to validate your work.
Response: A table for comparison with other papers using deep-learning for ASD is added (even though they are based on eye-tracking data, since using hand writing data for ASD classification is not available in the literature).
Response: A comparative table of studies has been added to the article and adequately discussed in the discussion section.
12) A discussion part must be added before the conclusion.
In this paper, a practical approach which is using hand writing data for detecting ASD is proposed. The main advantage of the approached comes form is simplicity of its application where no professional expertise is necessary. While, we achieved good accuracy (93.55%), as deep learning is a data driven approach, it is possible to improve accuracy even more. Thus, in future, although it is time consuming to collect data, we are planning to extend the dataset.
13) You must cite the following papers
Response: All references given were cited in the article accordingly.
Author Response File: 
Author Response.docx
