Multivariate Analysis of Vocal Fold Vibrations on Various Voice Disorders Using High-Speed Digital Imaging

Round 1

Reviewer 1 Report

The idea of investigating the discriminative power and/or capability for differential diagnosis of different HSV-based measures is very interesting and could guide future studies on which measure could be more appropriate. However, the paper lacks clearly defined rationale, motivation, and significance. Also, there are some major concerns regarding the methodology of the study.

 

Major comments:

Rationale and significance: The rationale and motivation of this study should be stated more clearly. For example, the authors have mentioned that “the choice of parameter is at the researchers’ discretion,” which is not a problem in itself. Depending on the research questions and hypotheses of a study the researcher should consult with the literature and select the most appropriate measures for that specific study. Obviously, if the research questions change, the researcher should select a different set of parameters. The study seems to suggest selecting a pre-determined set of parameters for a specific disorder independent from the research question and hypotheses of a study. Instead, a more beneficial approach would be a series of studies that would highlight when to use a specific parameter and for what research question. Alternatively, and probably more in line with the current study, would be finding a set of disorder-specific parameters that best reflect the severity of the disorder or capture the improvement following an intervention. 

Material: 

• The dataset is highly imbalanced with some disorders being significantly under-represented. Not only would this imbalance affect the generalization of the findings, but it could pose a serious problem for running statistical analysis.
• Important information from the participants is missing, including their sex/gender and ages.
• The diagnostic category of polyps is mentioned throughout the paper; however, in Table 1, no information about polyps is provided. Additionally, it is not clear why disorders with very small sample sizes (<10) were included due to the sample size being underpowered for statistical analysis.
• The results section is mentioning differences between the assessability of videostroboscopy and high-speed digital imaging. However, no information is provided about how videostroboscopy was collected. Also, the assessability was not defined. Additionally, it is not clear how this analysis relates to the aim of this study.
• The study seems to be a retrospective analysis of a data set; however the writing suggests otherwise (e.g. “… were enrolled”. Or “… were recruited”). Please clarify.

Analysis:

• There are many dependent variables. How were multiple comparisons accounted for during the statistical analysis?
• The lack of clear information in the paper implies that the t-test has been used for categorical variables, which is problematic.
• Some classes had very few samples. With such a small sample size, how did the authors have the statistical power to run so many statistical tests? Also, the violation of homogeneity of variance (which has not been tested/mentioned) when classes are highly imbalanced could have serious consequences on the outcome of the analyses.

Presentation:

• No figure on the distribution of the most prominent measures or their box plots is presented.
• With so many dependent variables, it is very hard for the audience to follow the paper. Also, the same measure is often computed using different approaches; but the measures are not grouped or compared with each other. First, a couple of tables should be added to the paper (and not in the appendix) where each measure is defined, and, more importantly, the phenomenon that they can capture should be stated. Second, if a measure is meaningful for a research question, the measure should remain relatively robust to its computation approach. Especially given that the sample size is very low and so many dependent variables are included, one could get a statistical significant result by chance by keeping the construct fixed and changing its computation approach.
• Discussion is treated very lightly and no overarching discussion is presented. The outcomes from different analyses should be synthesized to draw more general conclusions.
• The conclusion is extremely short and lacks specific takehome messages for the reader.

There are many issues with the manuscript in its current form that were not described above. Scientific rigor and manuscript writing are not of high quality.

Author Response

Thank you for your valuable opinions and comments.

Rationale and significance

Thank you for your valuable comment. We expanded the introduction to clarify the motivation, rationale and hypotheses of the current study.

Important information from the participants is missing, including their sex/gender and ages.

We added the information to Table1.

The diagnostic category of polyps is mentioned throughout the paper; however, in Table 1, no information about polyps is provided. Additionally, it is not clear why disorders with very small sample sizes (<10) were included due to the sample size being underpowered for statistical analysis.

The information of vocal fold polyp was missing.  We added the subject number of polyp to Table1.

The results section is mentioning differences between the assessability of videostroboscopy and high-speed digital imaging. However, no information is provided about how videostroboscopy was collected. Also, the assessability was not defined. Additionally, it is not clear how this analysis relates to the aim of this study.

Thank you for raising an important point.  We added the rationale to perform VS and HSDI in the introduction section.  Further, we added a paragraph named “Videostroboscopy and assessability” in the method to clarify the definition of assessability and added expanded the explanations in the discussion and conclusion.

The study seems to be a retrospective analysis of a data set; however the writing suggests otherwise (e.g. “… were enrolled”. Or “… were recruited”). Please clarify.

Thank you.  The study is a retrospective analysis of a data set.  We corrected the sentences.

There are many dependent variables. How were multiple comparisons accounted for during the statistical analysis?

The lack of clear information in the paper implies that the t-test has been used for categorical variables, which is problematic.

Also, the violation of homogeneity of variance (which has not been tested/mentioned) when classes are highly imbalanced could have serious consequences on the outcome of the analyses.

Thank you for your valuable comments.  We added the details of statistic analysis as follows.

“For the univariate analyses between normal subjects and patients with laryngeal diseases, Kolmogorov–Smirnov tests were applied to assess normal distribution of the parameters.  For continuous parameters with normal distribution, univariate analyses of variance followed by Student’s t-tests were performed.  For the other parameters (e.g. categorical, non-normally distributed continuous parameters), Kruskal–Wallis H tests followed by Mann-Whitney’s U tests were performed.  First, the normal subjects and all the patients were compared.  Then, laryngeal diseases with more than 10 subjects were separately compared to normal subjects”. 

Some classes had very few samples. With such a small sample size, how did the authors have the statistical power to run so many statistical tests?

Thank you.  As you say, we think it is also a limitation.  We added a sentence in the limitation paragraph.

No figure on the distribution of the most prominent measures or their box plots is presented.

Thank you.  However, for fear that adding several figures will expand too much the manuscript which is already very long (we added another figure and expanded the manuscript in response to the reviewers’ comments this time), we hesitated to do this task.  If you think that addition of figures is still necessary, we are ready to add them in the next round. 

With so many dependent variables, it is very hard for the audience to follow the paper. Also, the same measure is often computed using different approaches; but the measures are not grouped or compared with each other. First, a couple of tables should be added to the paper (and not in the appendix) where each measure is defined, and, more importantly, the phenomenon that they can capture should be stated.

Thank you for your comment.  We changed appendixes to tables, changed their layout, and inserted them to the method section.  Further, we added the general guidance of parameters in the method section to improve readability.  Additionally, we added a paragraph named “parameter” in the discussion regarding investigating the same measure calculated by different methods, and added a citation.

Second, if a measure is meaningful for a research question, the measure should remain relatively robust to its computation approach. Especially given that the sample size is very low and so many dependent variables are included, one could get a statistical significant result by chance by keeping the construct fixed and changing its computation approach.

Thank you for your comment.  We addressed this matter to the paragraph “limitation”.

Discussion is treated very lightly and no overarching discussion is presented. The outcomes from different analyses should be synthesized to draw more general conclusions.

Thank you for your comment.  We expanded the discussion by adding discussion of analysis program in the paragraph “Analysis Method”, by adding a paragraph “parameter”, and by adding discussion of the paragraph “Limitation”. 

The conclusion is extremely short and lacks specific takehome messages for the reader.

Thank you. We expanded the conclusion. 

Reviewer 2 Report

Improve style of citation  (e.g. rather ", see [1]" than just "[1]") 

Check abstract 

1)   "The four key parameters ... " - specify them or do not use "The"

2)  Is R2 0.52 really high compared to R2 0.43 ? 

Check formatting of the appendix - seems unclear. 

 

Author Response

Improve style of citation (e.g. rather ", see [1]" than just "[1]")

Thank you.  We corrected the style of citations of key articles as you mentioned.

Check abstract.

"The four key parameters ... " - specify them or do not use "The"

Is R2 0.52 really high compared to R2 0.43 ?

Thank you.  We corrected the points you raised.  Statistically yes since the R² is usually divided by 0.50.  But, not that different as you say.  We corrected the expression of the abstract to suit your point by integrating “high” and “moderate” into “moderate-to-high”.

Check formatting of the appendix - seems unclear.

Thank you.  We corrected the layout to be more readable.

Reviewer 3 Report

 

This manuscript catalogs laryngeal exam parameters from 350 subjects (normal and diseased).

The authors have published several previous works on this topic.  The exams were all completed before 2013, so I suspect many of the subjects have been included in prior published papers. It would be helpful if the authors could explain what new information or approach is taken in this manuscript that would add to the current body of literature.

 

 

“PD” should be defined. (Page 3 line 107)

 

Page 4 line 175 contains a typo in the definition of R2.

 

More details of how the parameters were calculated would be helpful. Specifically, what user inputs were required? (Versus automated Matlab calculations). 

 

Table 3 contains some minor inconsistencies in Normal values -these should all be the same when re-listed for different diseases.

 

Odds Ratio for NVA in VF scar is not feasible to calculate. Similarly I don’t think the 95% CI is helpful as applied to the OR. 

Author Response

The authors have published several previous works on this topic.  The exams were all completed before 2013, so I suspect many of the subjects have been included in prior published papers. It would be helpful if the authors could explain what new information or approach is taken in this manuscript that would add to the current body of literature.

Thank you.  We added more details of what have been done and what have not in our studies to the introduction.

“PD” should be defined. (Page 3 line 107)

Thank you.  We spelled it out; “phase difference” (PD).

Page 4 line 175 contains a typo in the definition of R2.

Thank you.  We corrected the typo: from “0.5>R²≧0.5” to “0.5>R²≧0.2”.

More details of how the parameters were calculated would be helpful. Specifically, what user inputs were required? (Versus automated Matlab calculations).

Thank you.  We added the classification of parameters to the tables, and added the information as to necessary manual input to the method section.

Table 3 contains some minor inconsistencies in Normal values -these should all be the same when re-listed for different diseases.

Thank you for your comments.  We corrected the inconsistencies.

Odds Ratio for NVA in VF scar is not feasible to calculate.

Similarly I don’t think the 95% CI is helpful as applied to the OR.

Thank you for your valuable comments.  As you pointed out, we omitted the row of 95% CI since the p value can substitute most of the role of 95% CI.  The OR of NVA should be infinite.  We corrected the data.

Round 2

Reviewer 3 Report

Manuscript is improved from original submission, now with greater detail added. Previous datasets were apparently analyzed via univariate analysis. This manuscript now combines all the previous subjects and parameters and employs multivariate analysis to identify key parameters. The authors propose a multi-faceted laryngeal evaluation approach -this would indeed be complete but probably not necessary in many circumstances, even in research. Nonetheless this manuscript does propose certain evaluation parameters to be specifically meaningful in specific disorders.   Grammar needs attention throughout.

Author Response

Grammar needs attention throughout.

Thank you for your valuable comment. We submitted the manuscript to a language-editing service (Editage, www.editage.com), and had it edited.