The Agreement Between Bushnell and Stalker Radar Guns for Measuring Ball Speed in Throwing and Kicking

Round 1

Reviewer 1 Report (New Reviewer)

Comments and Suggestions for Authors

Dear Authors,

Thank you for submitting your manuscript entitled "The agreement between Bushnell and Stalker radar guns for measuring ball speed in throwing and kicking" to Applied Sciences. I appreciate the opportunity to review your work and commend you for addressing an important and original topic within Biomedical Engineering.

In my opinion, the manuscript fits the Special Issue on Physiological Monitoring and Performance Evaluation in Exercise and Sport as it deals with a practical method of monitoring performance in sports activities, namely ball speed in throwing and kicking. The study's outcomes provide insights into how technology can be employed in training and competition, meeting the Special Issue's aim of improving performance monitoring across sports contexts.

The manuscript addresses an important and practical issue in sports science: the validation of affordable radar technologies for measuring ball speed in throwing and kicking. The comparison between the Bushnell radar gun and the well-established Stalker system is a valuable contribution to performance monitoring. By validating more cost-effective equipment, this research has significant implications for accessibility and wider adoption in sports training and research environments.

Despite the notable strengths of the manuscript – such as a robust experimental design that adheres to well-established guidelines for reliability and concordance studies (GRRAS), and a sufficient sample size determined by power analysis that ensures the statistical rigor of the results – there are several areas in which improvements are needed to enhance the scientific rigor and overall impact of the manuscript. For your consideration and that of your colleagues, I have outlined the requested changes below and provided additional comments:
1. LITERATURE REVIEW
1.1. While the literature review is adequate, it could benefit from further expansion, particularly in discussing recent advances in sports monitoring technology. The introduction does a good job of explaining the importance of radar guns but misses the opportunity to place this study in the broader context of wearable sensors, video analysis systems, and other emerging technologies that are revolutionizing performance monitoring.
1.2. In addition, the manuscript would benefit from the inclusion of more recent references that discuss the use of low-cost performance monitoring tools in a broader range of sports, including practical applications in real-world coaching or rehabilitation scenarios.
2. DISCUSSION OF PRACTICAL IMPLICATIONS
2.1. Although the results are clearly presented, the discussion could be enriched by a deeper analysis of the practical implications of the results. The authors show that the Bushnell radar consistently underestimates the ball speed by about 1.4 km/h compared to the Stalker radar. However, the discussion should elaborate on how practitioners can account for this bias in real-world settings. For example, the potential implications for coaching, player development, and rehabilitation practices could be further discussed.
2.2. In addition, extending the analysis to a wider range of ball speeds or different environmental conditions would enhance the generalizability of the findings.
3. LANGUAGE AND STYLE
3.1. Although the manuscript is written in clear and technically correct English, there are some minor grammatical issues and awkward phrasing that could benefit from further proofreading. Improving the fluency and flow of the text, particularly in the introduction and discussion sections, would improve readability and make the manuscript more appealing to the reader.
3.2. Some sentences, especially in the Methods and Results sections, could be simplified to avoid overly complex structures that might hinder comprehension.
4. DISCUSSION OF LIMITATIONS
4.1. While the authors correctly identify the narrow range of velocities tested as a limitation, they could expand on other potential limitations. For example, the study was conducted in a controlled laboratory environment, and real-world conditions (such as outdoor weather variations) could influence the performance of radar guns. Additionally, further tests could be conducted at different angles, heights, and distances to better simulate various sporting environments.
4.2. The limitation regarding manual handling of the Bushnell radar could be further explored. Given that one radar gun was handheld, it would be important to discuss potential sources of variability introduced by this method.
5. RECOMMENDATIONS FOR FUTURE RESEARCH
5.1. The manuscript could be strengthened by suggesting specific avenues for future research. For example, future studies could examine the performance of the Bushnell radar across a wider range of sports and conditions, including extreme ball velocities and real-world environmental variables. Further studies could also compare radar guns with other technologies, such as high-speed cameras or motion capture systems.

The scientific quality of your manuscript is commendable, with several aspects that merit recognition. Additionally, it represents a valuable contribution to the field, and with the suggested revisions, it has the potential to become an important addition to the literature.

Comments on the Quality of English Language

The manuscript is well written and meets the standards of academic English. With a few minor adjustments to grammar, word choice, and sentence structure, it will be more polished.

While the manuscript is generally well organized, some sections could benefit from smoother transitions between ideas. This would improve the reader's ability to follow complex ideas, especially in the introduction and discussion sections.

Some sentences are quite long and could be broken up to avoid confusion. For example, in the discussion section, sentences that combine several ideas could be broken into shorter, clearer sentences to improve readability.

There are occasional minor grammatical errors, such as missing articles (e.g., "the") or unnecessary repetition of words. These can be corrected with a thorough proofreading.

In a few cases, word choice could be refined for more formal academic writing. For example, "exert maximum velocity" might be better expressed as "generate maximum velocity" or "achieve maximum velocity.

In my opinion, a final proofreading pass should address the remaining minor issues

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

please see attached

Comments for author File: Comments.docx

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report (New Reviewer)

Comments and Suggestions for Authors

Dear Authors,

I would like to congratulate you on the excellent work done in reviewing the manuscript "Assessing the agreement between Bushnell and Stalker radar guns for measuring ball speed in throwing and kicking."

All the requests have been carefully addressed, and the improvements introduced have significantly enriched the manuscript. I have no new requests or recommendations.

The quality of the work reflects your team’s scientific rigor and dedication.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

applsci-3176071.

 Assessing the agreement between Bushnell and Stalker radar guns for measuring ball speed in throwing and kicking

The objective of this study was to evaluate the consistency of the Bushnell radar device in measuring ball velocity during throwing and kicking activities, compared to the reference provided by a Stalker radar gun. The study involved 64 people who, as it turned out, performed five hand throws and five-foot strikes. The manuscript has some areas for improvement regarding the type of evidence and the absence of other analyses necessary to meet the stated objectives.

 

General and specific comments

Apart from the absence of any fundamental evidence in the analysis of inter-device agreement, the authors' main argument for concluding that the device studied shows excellent agreement with the gold standard is based on the high ICC value, which needs to be revised to reach this conclusion. In fact, the abstract only includes the results of the ICC, which is the calculation that raises the most doubts. In addition, some of the figures presented are not explained, which limits their contribution to the data analysis. On the other hand, the discussion is oriented in the comments on other devices that have also offered good agreement, which is not an argument to justify the agreement of the device studied in the present manuscript.

Line 175 The expression “to exert maximum power” should be replaced by “maximum velocity”. If this instruction was given, it is doubtful that the participants understood it.

Line 184 It is not explained why the device is placed at a row height of 1.2 meters. It seems obvious that when striking the soccer ball the angle of exit of the ball is very different from that originating from a height of 1.2 meters, and certainly very different from that when throwing by hand. If the authors considered the angle indifferent, they should have done the tests and conformed to this.

Line 203 The coefficient of variation of the sample has no relevance to the agreement between the devices. This coefficient could be relevant for the calculation of the ICC, which will be discussed below. The standard error of the estimate can be an indicator of agreement as long as the maximum error of estimate is calculated, which would indicate the extreme differences that could occur between the measurements of the two devices.

Line 207 The ICC is not sufficient to conclude that there is an agreement between devices. This calculation requires several reflections on its meaning. The variability of the sample influences the ICC. Consequently, the value of the coefficient can be very high, but the reliability of the measurements may be low or very low. For this reason, it is necessary to add the coefficient of variation associated with this analysis. The variability of the sample does not alter this coefficient. The ICC has very little relevance without the corresponding coefficient of variation of the two or more measurements analyzed. On the other hand, the manuscript does not indicate which measurements were used to calculate the ICC. From the study data, it can be deduced that each participant made five throws of each type (64 x 5 = 320), although this is not specified in the manuscript. Calculating the ICC on two measurements is different from calculating it on five. However, what is most relevant is that the reliability of the measurements of a group of subjects does not provide information on the agreement of one device with another. It would be helpful if the ICC had been calculated by comparing the measurements from the two devices. Still, an important question arises: which of the five measurements from each participant was used to analyse the results, if any? We do not see any information about all these issues in the manuscript, apart from the lack of the main indicator of reliability, which is the coefficient of variation when two or more measurements are made.

Line 212 It is indicated that a t-test was applied to check for possible differences between the velocities of the two devices. This calculation does not make sense, as the ANOVA indicating the differences is already included in the calculation of the ICC. Therefore, if a t-test is applied, it can be assumed that the ICC was not calculated with the data from the two devices. Of course, if the differences do not refer to the devices, the calculation is meaningless. However, it always remains to be seen which of the five measures the comparison is made with. This situation, therefore, adds to the doubts about the study's results. Moreover, if the differences were significant (p < 0.001), the agreement between devices decreases, whatever the type of calculation.

Line 225 The values of the limits of agreement in the Bland-Altman chart (4.8 + 2.02 = 6.82 km/h) are more than 10% of the mean. Some consideration should be given to the practical significance of these possible maximum differences.

Two figures representing regression lines appear, but there is no information about them. Neither the regression equation nor the value of correlation (r) is indicated. It is observed that the regression line does not pass through the origin of the axes. That is, the equation's independent term (a) must be zero or very close to zero. This condition is decisive for assessing the agreement of the measurements of two devices. The regression line must coincide or be very close to the bisector of the angle of the coordinate axes, which requires a coefficient of X (b) equal to 1 or very close to 1. This condition must be ratified with the calculation of Lin's concordance correlation coefficient, which is not applied in the manuscript. The fact that there is a high relationship between the measures of the two devices does not mean that there is agreement between these measures. There can be a near-perfect correlation and a high systematic error.

Line 254 The ICC is neither the only nor the best indicator of inter-device agreement, especially in this case, where it is not specified by which measures the calculation is made..

Line 276 The SEE is only applicable with implementation. It would serve to calculate the maximum error in the prediction, but none of this is indicated in the results. Furthermore, the maximum estimation error does not agree with the maximum error observed in the Bland-Altman analysis.

The discussion compares the study results with the agreement found when comparing the Stalker with other devices. This comparison does not add anything to the agreement that the device being evaluated in the study may have. It is irrelevant for this study whether other devices have an agreement with the Stalker.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript presents an investigation into the agreement between two radar guns, the Bushnell and Stalker, for measuring ball speed in throwing and kicking activities. The authors have conducted a thorough comparative analysis, providing valuable insights into reliability of these widely used measurement tools in sports science. The study addresses a significant gap in the literature by evaluating the performance of a more affordable radar gun (Bushnell) against a gold-standard device (Stalker), thereby offering practical implications for both researchers and practitioners. The manuscript is well-structured and the findings are of potential interest to the readership of Applied Sciences. I suggest some minor to moderate revisions before recommending publication.

 

1.      Abstract: Given the relatively high Cohen’s d, the difference between the devices is not that small compared to the variance between the subjects. I would consider slightly rewording your conclusion. I would say that the difference between the devices is practically relevant; however, because of the high correlation between the devices, practitioners could still use either device with the consideration of adjusting normative values accordingly. Please also adjust discussion and conclusion accordingly.

2.      Introduction is very well written. The section in lines 99 – 119 could be shortened by approximately 50 %. It is not entirely clear how and why the Stalker is the gold standard – has it been validated against other, even more precise radars or other devices. More references and explanation on this matter would be beneficial. For the study in lines 83-85 (ref. 13), some more information (including ICC or other statistics) could be added.

3.      Methods: Very well written. The accuracy values (line 182 and 183) are the manufacturer provided values or has this been measured in specific studies?

4.      Discussion: Lines 340-345 – it could be interesting to add how these velocities compare to the top velocities reached in the actual gameplay (e.g., of tennis).

 

Minor edits:

-        Line 83: Suggest replacing “delved into comparing” with “compared”

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

No comments

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for addressing my comments. I still think the wording in conclusions is not optimal.

 

For instance, in abstract: 

"In conclusion, this study suggests that the agreement between both radar guns is excellent. However, the Bushnell consistently underestimates the ball speeds of throws and kicks, with differences that are statistically significant and practically  relevant. Despite this, the high correlation between the devices indicates that practitioners can use  either device, provided they adjust normative values accordingly."

 

"Agreement" implies that devices provide same values which is not the case.

I suggest you rewrite as "...this study suggests that the correlation between both radar guns is excellent." and avoid the term agreem