Reliability of Wu Huanqun’s Table Tennis Game Analysis Method in Authors’ Own Modification

Round 1

Reviewer 1 Report

The present study present a modified observational method to analyze table tennis performance. The main goal of the authors is to evaluate its reliability (and validity?) and they present the results of an analysis done in the 2019 World Championship.

The statistical design and procedures are adequate and well performed. In addition, the additional data provided ease the comprehension of the method. However, there are some concerns that worry me out. Above, I questioned the validity of the method assessed. The references apported of Wu Huanqun's method are not a validity study. Thus, I don't know to what extent assessing the reliability of a modified method that seems it has not been previously validated is adequate. Maybe, this study is also a validation work. In this line, altought I am aware that counting on the world champions player to get data is a privilege, performing the study with just one player analyzed and with only 7 expert observers is enought to conclude about reliability (and validity) of a method. 

The authors have done a good work. But I suggest to increase the analysis with more players, maybe same players in different matches were the circumstances are also different. Also more observers, the reliability must be checked intra and inter obervers. 

In the other hand, the observing method must be better described. Where the games recorded and viewed afterwards (would the method be valid in a live view?), the observers watched the full games or just the target strokes? in that case, how were them selected and by whom?.... 

Author Response

Dear Editors and Reviewers of Applied Sciences

We appreciate all the constructive comments and valuable observations very much. We also thank you for the effort and time put into the review of our manuscript.

In the following response, each comment has been carefully considered point by point and replied to. Responses to the Reviewers are in a dialogue and marked with tab and italics; changes in the revised manuscript are tracked.

Response to Reviewer 1 comments:

…Above, I questioned the validity of the method assessed. The references apported of Wu Huanqun's method are not a validity study. Thus, I don't know to what extent assessing the reliability of a modified method that seems it has not been previously validated is adequate.

Thank you very much for this comment. In fact, we do not have information whether the method, although widely used in China (and not only), was validated. Therefore, this work can also be treated as an introduction to the evaluation of validity. In our opinion, by determining the compliance and homogeneity of experts' assessment in the applied observation method (and thus its high reliability, in this case: "inter-rater reliability"), we have demonstrated an important basis for directing future research on the validation of this method. Making more observations as well as further evaluation of the method validation can and should be an issue for further research. We added above statements to the chapter Discussion.

Maybe, this study is also a validation work. In this line, altought I am aware that counting on the world champions player to get data is a privilege, performing the study with just one player analyzed and with only 7 expert observers is enought to conclude about reliability (and validity). The authors have done a good work. But I suggest to increase the analysis with more players, maybe same players in different matches were the circumstances are also different. Also more observers, the reliability must be checked intra and inter obervers.

Thank you very much for this comment. In the work, we originally presented the assessment made on one competitor by 7 experts. Nevertheless, the competitor was assessed on many levels - as many as 51 elements were assessed. In order not to complicate the problem, we initially decided to present only one player's ratings. The same experts also assessed the second competitor (M. Falck). Recognizing the Reviewer's opinion that it would be good to include more observations in the work, we decided to present these additional assessments, and thus present and test the compliance of 37 additional assessed elements (Tab. 3). Therefore, the text in the chapters: Methods, Results and Discussion has also been supplemented. We think this will make the results of our work more convincing

In the other hand, the observing method must be better described. Where the games recorded and viewed afterwards (would the method be valid in a live view?), the observers watched the full games or just the target strokes? in that case, how were them selected and by whom?..

Thank you very much for this comment. All needed informations, pointed out above, were added. This method requires a lot of skill from the evaluator. Its use and application, however, refer to the observation of the recorded material. A skilled judge, however, can make an "live" assessment at the first level of observation. In our work, however, we investigated the use of recorded and sorted material.

In the section Mehods it stays now: “The match was recorded and the material was cut into pieces – rallies won by individual player (51 rallies won by Ma Long, 37 rallies won by Falk). Despite the fact that the method can be used live partially recording and possibility of replaying the actions is better method. Each of the experts viewed the same material, prepared and pre-sorted by the authors of the paper. Every expert watched the full match (all rallies) and the number of replays was unlimited”.

There are also additional information in Appendix A (the steps of an expert).

Response to Reviewer 2 comments:

Add citation „

 Many authors dealing with this subject in table tennis have used a variety of tools for  technical, tactical, or biomechanical observation and analysis. „  

We thank the Reviewer for this comment. However, this sentence is an introduction to the following ones. The following sentences also contain references to literature ([3, 7, 8, 10, 11] etc.

Change to passive voice. „We received the results of the analyzes from  of them. “

We thank the Reviewer for this comment, we have changed it. It stays now: “Seven of experts responded and delivered their observations and calculations”.

Table 1. The observational sheet needs a clearer and more self-readable version

Thanks for this consideration. We added information to Table 1, completing headings and endnotes

The name of the statistical analysis sections is needed.

            Thanks, we added such a subheading

Ethical approval is missing

We thank the Reviewer for this comment. However, in our case, we checked the "not applicable" option when it comes to presenting the ethical consent form, as this is an observational study, without any experiments on a living human.

It is not necessary in the result sections „After applying the Shapiro-Wilk test, it was found that the results do not have a normal distribution, “

            We removed this sentence.

Conclusions must be changed so that they clearly present the result of the work and not another theoretical analysis.

We thank you for this comment. We modified this chapter. However, we left the last sentence which, in our opinion, points out the meaning and value of our study.

Author Response File: Author Response.docx

Reviewer 2 Report

Table tennis is a popular sport that requires a lot of attention. Performance analysis is one of the important analysis techniques. The validation of different assessment methods is an important part of science. The Introduction is written in an appropriate manner and contains all the necessary information. I recommend the manuscript for publication, but some changes are necessary

Add citation „

 Many authors dealing with this subject in table tennis have used a variety of tools for  technical, tactical, or biomechanical observation and analysis. „  

Change to passive voice. „We received the results of the analyzes from  of them. “

Table 1. The observational sheet needs a clearer and more self-readable version

The name of the statistical analysis sections is needed.

Ethical approval is missing

It is not necessary in the result sections „After applying the Shapiro-Wilk test, it was found that the results do not have a normal distribution, “

Conclusions must be changed so that they clearly present the result of the work and not another theoretical analysis.

Author Response

Dear Editors and Reviewers of Applied Sciences

We appreciate all the constructive comments and valuable observations very much. We also thank you for the effort and time put into the review of our manuscript.

In the following response, each comment has been carefully considered point by point and replied to. Responses to the Reviewers are in a dialogue and marked with tab and italics; changes in the revised manuscript are tracked.

Response to Reviewer 1 comments:

…Above, I questioned the validity of the method assessed. The references apported of Wu Huanqun's method are not a validity study. Thus, I don't know to what extent assessing the reliability of a modified method that seems it has not been previously validated is adequate.

Thank you very much for this comment. In fact, we do not have information whether the method, although widely used in China (and not only), was validated. Therefore, this work can also be treated as an introduction to the evaluation of validity. In our opinion, by determining the compliance and homogeneity of experts' assessment in the applied observation method (and thus its high reliability, in this case: "inter-rater reliability"), we have demonstrated an important basis for directing future research on the validation of this method. Making more observations as well as further evaluation of the method validation can and should be an issue for further research. We added above statements to the chapter Discussion.

Maybe, this study is also a validation work. In this line, altought I am aware that counting on the world champions player to get data is a privilege, performing the study with just one player analyzed and with only 7 expert observers is enought to conclude about reliability (and validity). The authors have done a good work. But I suggest to increase the analysis with more players, maybe same players in different matches were the circumstances are also different. Also more observers, the reliability must be checked intra and inter obervers.

Thank you very much for this comment. In the work, we originally presented the assessment made on one competitor by 7 experts. Nevertheless, the competitor was assessed on many levels - as many as 51 elements were assessed. In order not to complicate the problem, we initially decided to present only one player's ratings. The same experts also assessed the second competitor (M. Falck). Recognizing the Reviewer's opinion that it would be good to include more observations in the work, we decided to present these additional assessments, and thus present and test the compliance of 37 additional assessed elements (Tab. 3). Therefore, the text in the chapters: Methods, Results and Discussion has also been supplemented. We think this will make the results of our work more convincing

In the other hand, the observing method must be better described. Where the games recorded and viewed afterwards (would the method be valid in a live view?), the observers watched the full games or just the target strokes? in that case, how were them selected and by whom?..

Thank you very much for this comment. All needed informations, pointed out above, were added. This method requires a lot of skill from the evaluator. Its use and application, however, refer to the observation of the recorded material. A skilled judge, however, can make an "live" assessment at the first level of observation. In our work, however, we investigated the use of recorded and sorted material.

In the section Mehods it stays now: “The match was recorded and the material was cut into pieces – rallies won by individual player (51 rallies won by Ma Long, 37 rallies won by Falk). Despite the fact that the method can be used live partially recording and possibility of replaying the actions is better method. Each of the experts viewed the same material, prepared and pre-sorted by the authors of the paper. Every expert watched the full match (all rallies) and the number of replays was unlimited”.

There are also additional information in Appendix A (the steps of an expert).

Response to Reviewer 2 comments:

Add citation „

 Many authors dealing with this subject in table tennis have used a variety of tools for  technical, tactical, or biomechanical observation and analysis. „  

We thank the Reviewer for this comment. However, this sentence is an introduction to the following ones. The following sentences also contain references to literature ([3, 7, 8, 10, 11] etc.

Change to passive voice. „We received the results of the analyzes from  of them. “

We thank the Reviewer for this comment, we have changed it. It stays now: “Seven of experts responded and delivered their observations and calculations”.

Table 1. The observational sheet needs a clearer and more self-readable version

Thanks for this consideration. We added information to Table 1, completing headings and endnotes

The name of the statistical analysis sections is needed.

            Thanks, we added such a subheading

Ethical approval is missing

We thank the Reviewer for this comment. However, in our case, we checked the "not applicable" option when it comes to presenting the ethical consent form, as this is an observational study, without any experiments on a living human.

It is not necessary in the result sections „After applying the Shapiro-Wilk test, it was found that the results do not have a normal distribution, “

            We removed this sentence.

Conclusions must be changed so that they clearly present the result of the work and not another theoretical analysis.

We thank you for this comment. We modified this chapter. However, we left the last sentence which, in our opinion, points out the meaning and value of our study.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

I want to thank the authors for their effort improving their manuscript. I think the research offers now better results and information. However, I still have some concerns: as long as the used method seems to not be validated, this paper is an opportunity to do such thing. I appreciate all the work behind this experiment, with the inclusion of the second player, more relevant data is involved in the analysis. However, in my opinion, to validate such complex analysis, more players and matches should be included. Different contexts may evoque different performance from the same players. I do not know if an "introduction to the evaluation of validity" of the method is good enough to justify an analysis of the "inter-rater reliability". What I am saying is that the work is good, but I am not sure if relevant enough. In my opinion the method should be validated including all the factors that could influence the variability and reproducibility of the method, and here the observers are higher represented (7) over players (2) and games analyzed (1). In favor of relevancy is the fact that the method is widely used in China, which is an important country in table tennis. Also, the complexity of the experiment may justify this "partial validation".