Comparison of Two Methods to Estimate the Maximal Velocity of a Ball during an Overhand Throw ^†

Abstract

Maximal ball velocity is a significant performance indicator in many sports, such as baseball. Doppler radar guns are widely assumed to underestimate velocity. Accuracy increases as the cosine angle between the radar gun and the object decreases. The purpose of this study was to investigate the impact of player handedness and the location of the radar gun on the accuracy of ball velocity. Throws were analyzed in four conditions: the radar gun on the right side, throwing with the right arm, then with the left arm; and the radar gun on the left side, throwing with the right arm, then with the left arm. The Cronbach’s alpha for all four conditions showed $\alpha$-values above 0.97; however, a paired t-test indicated significant differences between the 3D motion analysis and the radar gun. Bland–Altman plots show a high degree of scatter in all conditions. Results suggest that the radar gun measurements can be highly inconsistent when compared to 3D motion analysis.

1. Introduction

An accurate measurement of ball velocity is a key performance variable in almost all ball-based sports. The quicker a ball can reach its destination (e.g., a tennis serve and soccer kick) the less time the receiving or defending end has to react and reach the ball [1,2]. In baseball and softball, throwing, pitching, and batting speeds are significant performance skills and, therefore, are of great interest to coaches, scouts, athletes, sponsors, and fans.

A number of methods have been used to measure maximal ball velocity in sport. The gold standard measurement tool for any physical movement is three-dimensional (3D) motion capture (MC) [3,4,5]. However, these systems are expensive, require interpretive software, data cleaning and analysis are time consuming, and they cannot be used in the field.

Radar guns are portable, user friendly, and provide immediate results. Speed can be measured successfully in many weather conditions, both during game time and practice. They are commonly installed in stadiums and baseball clubs [6]. Doppler radar guns measure the fastest moving object in their radius [7], therefore, their placement on the field is somewhat flexible [8]. The radial velocity of the ball is calculated using the shift in the frequency of the reflected signal called the Doppler frequency [9]. When tracking a baseball pitch, the peak speed is the same as the release speed, since a ball only slows down after it is thrown. Doppler radar guns are widely assumed to underestimate velocity, as they measure the radial velocity; accuracy increases as the cosine angle between the radar gun and the object decreases [10] and, thus, proper calibration is imperative. It is hypothesized that ball trajectory and release point can influence the accuracy of a radar gun, as ball position is directly related to cosine angle.

Many studies have used a Doppler radar gun to measure ball velocity [8,11,12,13,14,15,16]; however, very few have validated the accuracy of the radar gun used [11,12]. While those studies indicated high accuracy, either the protocols were not provided, or the comparison standard was another radar gun. Therefore, a gap in the literature focusing on the validity of a Doppler radar gun using a gold standard persists. The purpose of this study was to determine the impact of radar gun placement and thrower handedness on accuracy compared to three-dimensional motion analysis.

2. Methods and Materials

2.1. Protocol

Participants were recruited through posters at the University of Calgary and through invitations to acquaintances. Upon arrival, participants were asked to self-report their age and gender. Researchers explained the protocol and answered any questions pertaining to the study. Each participant was taken through a 10-min dynamic warmup specifically designed for an overhand throw and allowed five additional minutes of practice throws.

Each participant completed a total of 24 successful overhand throws (12 with each the dominant and non-dominant arm) in four different conditions to examine the impact of release position on accuracy:

Condition RR: Stalker Radar Gun (RG) placed to the right and participant throws six times with the right arm;

Condition RL: RG placed to the right and participant throws six times with the left arm;

Condition LR: RG placed to the left and participant throws six times with the right arm;

Condition LL: RG is placed to the left and participant throws six times with the left arm.

Figure 1 shows the schematic of the experimental set up. Using an indoor baseball (Jugs Sport Lite-Flite) covered in retro-reflective tape (to be recorded by the MC system), participants were told to aim for a 1x1m² square (target) but were not specifically instructed to throw for power or accuracy. If the participant threw outside the target, a retry was conducted. Only successful throws were included in the dataset.

2.2. Materials

2.2.1. Stalker Radar Gun

The RG target type was set to a ball and configured to a baseball setting, which measures the peak speed and the roll-down speed of a baseball pitch. According to the RG manual, it analyzes not only the ball, but also the pitcher’s motion to report the most accurate release speed [10]. The lowest recording speed was set to 20 km/h and the highest recording speed was set to 200 km/h. For this research, only peak speed was recorded. The radar gun rested on a tripod measuring 1.41 m in height. The cosine angle can be set up in the radar gun settings; however, the study protocol was to correct the error for the angle during data processing and record the raw speed value. All range and peak settings followed manual guidelines.

2.2.2. 3D Motion Capture

Seven MC cameras were used and captured data at 500 Hz for 5-s intervals. The system was calibrated before each trial to ensure accurate data collection. MC data was processed using Cortex (v. 13.3) software. A 2D ball velocity was calculated using the x- and z-coordinates. The x-axis was defined as the direction in which the ball traveled. Data were smoothed using a low-pass fourth order Butterworth filter with a cut-off frequency of 6 Hz.

2.3. Statistical Analysis

Data were analyzed using SPSS version 23.0 (SPSS Inc., Chicago, IL, USA). The validity of the RG by placement and throwing arm compared to MC was assessed using Cronbach’s alpha. A paired t-test and Bland–Altman plots were used to compare the maximal velocity recorded by RG and maximal 2D velocity recorded by MC by placement of radar gun and throwing arm.

3. Results

Data were collected as part of a larger study that aimed to assess changes in participants’ overhand throwing performance bilaterally. All participants signed an informed consent form. Ethical approval for this study was obtained through the University of Calgary. Data from 20 participants (age: 33.15 ± 12.36 years, nine females, 14 right-handed, one ambidextrous) were included in this study.

A total of 704 throws were captured. Of these, 206 could not be used due inaccurate throwing and 19 had no radar gun reading or an obvious error in measurement. Therefore, 480 throws were included in this dataset.

Cronbach’s alpha, mean absolute percent error (MAPE), and paired t-test results are summarized in

Table 1. Validity statistics by condition, Cronbach’s alpha, mean absolute percent error (SD), and paired t-test.

			Paired t−Test
Condition	Cronbach’s Alpha	Mean Absolute Percent Error (SD)	95% CI of the Difference	T (df = 119)	Sig. (2−tailed)
RR	0.973	3.0 ($\pm$5.7)	(−1.85, −0.65)	−4.12	***
RL	0.983	7.0 ($\pm$8.7)	(−3.91, −2.49)	−8.90	***
LR	0.997	3.4 ($\pm 2.3$)	(−1.86, −1.29)	−11.15	***
LL	0.996	3.6 ($\pm 4.6$)	(−1.77, −0.91)	−6.13	***

*** p < 0.001.

. Cronbach’s alpha correlation between the RG and MC were 0.973, 0.983, 0.997 and 0.996 for conditions RR, RL, LR, LL respectively. The MAPE ranged from 3.0 ($\pm$5.7)–7.0 ($\pm$8.7) with condition RL having the greatest MAPE. The paired t-test showed significant differences in all four conditions (−4.12 p < 0.001; −8.90 p < 0.001; −11.15 p < 0.001; −6.13 p < 0.001) and the 95% confidence interval of the differences ranged from −3.91, −0.65.

Table 2. Throws by Range of Absolute Percent Error.

Condition	0%	0.1–0.9%	1.0–5.9%	6.0–9.9%	10%+
RR	4	30	78	2	6
RL	0	3	87	15	15
LR	0	8	104	5	3
LL	7	24	67	10	12

summarizes the number of throws by the range of absolute percent error. Conditions RR and LL were the only two conditions with cases that had a 0% error. The majority of cases (56–86%) in all conditions fell between 1.0–5.9% error, with LR having 104 cases in this category.

Figure 2 Presents Bland–Altman plots that show the differences between the RG and MC.

4. Discussion

This paper looks at the validity of the Doppler radar gun for an overhand throw by comparing data to a 3D motion capture system. To the best of the authors’ knowledge, this is the first study that examines the validity and reliability of a sports radar gun.

Results indicate that the radar gun is highly correlated with the MC, with Cronbach’s alpha values of 0.973, 0.983, 0.997 and 0.996 for conditions RR, RL, LR, and LL, respectively. This suggests that the radar gun is a valid measure of ball velocity under the conditions assessed based on a correlation measurement. However, the MAPE results of 3.0 (±5.7), 7.0 (±8.7), 3.4 (±2.8) and, 3.6 (±4.6) for conditions RR, RL, LR, and LL, respectively, indicate there is a significant difference in the accuracy of the speed measured depending on the placement or handedness of the participant. Bland–Altman plots show high degrees of inconsistency in the dispersion of data points in each condition. This is further evidence that undermines the validity of the radar gun. One possible explanation for the high Cronbach’s alpha is the heterogeneous nature of the participants.

For practical use, it is important to understand the difference between the means and how the conditions of the throw impacts results. In this study, the radar gun was placed behind on the right-hand side of the participant and then behind on the left-hand side, while the participant threw with both left and right arms. The paired t-test shows that the validity of the radar gun is lowest when placed on the left and the participant is throwing with their right arm. Following the same trend, the accuracy is lower when the radar gun is on the right and the participant is throwing with their left arm. In all four conditions, there is a highly significant difference between RG and MC.

The reliability of the radar gun is in agreement with previous findings [11]. However, it is difficult to compare these results with other papers, because the specific placement of the radar gun is not mentioned in some papers [5,11,15]. Placement could significantly affect findings due to the angle error [8]. Results concerning the accuracy of the radar gun are linked to the placement of the radar gun and the handedness of participants.

It is important to note that interference can occur during data collection. Noise from the environment or from the unit itself not only impedes the radar’s ability to determine the presence of a target but also injects errors in the tracking measurements of a system [17]. Interference can affect the radar gun reading, as noted in this study in the 19 trials when the radar gun did not record any speeds. Of these 19 times, most occurred when the radar gun was on the left-hand side of the participant. It is difficult to identify the specific cause of the interference during trials.

5. Conclusions

Based on the paired t-test, the findings of this study indicate that the Stalker Doppler radar gun is valid for measuring ball speed; however, it is less accurate when the radar gun is placed on the opposite side of the throwing arm. This could lead to serious disadvantages for players when using their arm on the opposite side of where the radar gun is placed. Therefore, when using this radar gun during decisional measurements, it is recommended to consider the handedness of the player and position of the radar gun. However, the analysis of the MAPE and Bland–Altman plots are not consistent with the paired t-test results. Users should be cautious when they interpret radar gun data because of the lack of validity based on position and thrower handedness.

6. Future Directions

Further research should investigate the impact of confounding factors that could interfere with a Doppler signal in a sports setting. For example, when using a fixed height for the radar gun throughout the trials, it would be interesting to explore whether the maximum ball height affects the radar gun’s measurement accuracy, in addition to participant throwing characteristics, including skill level and speed ranges.