Computer Vision-Based Tennis Ball Tracking Using You Only Look Once for Training Analytics ^†

Abstract

Tennis is an exceptionally fast-paced sport where the ability to return the ball precisely to an opponent’s weak zones often determines match outcomes. Although wall practice serves as a fundamental and effective training method, accurately capturing and analyzing the spatial distribution of ball impact points during high-speed rallies remains highly challenging. Leveraging computer vision, we propose a two-stage detection pipeline that integrates You Only Look Once Version 12 and MobileNetV2 to generate candidate bounding boxes, stabilized by a Kalman filter with a predict–update mechanism. This approach ensures robust and reliable object tracking, providing valuable insights into tennis training performance, placement accuracy, and actionable insights for sports analytics.

1. Introduction

A tennis ball training system is designed to enhance training accuracy by reducing the dependence on subjective human visual judgment during practice in this study. Previous studies have indicated that accurately detecting ball impact points in racket sports is challenging, especially for small, fast-moving balls, which are prone to missed or erroneous detections [1]. In addition, traditional human observation is susceptible to factors such as attention lapses, fatigue, and subjective bias, often resulting in inconsistent training outcomes [2]. The developed system leverages deep learning techniques to enable real-time detection of tennis ball impacts on the wall, effectively overcoming challenges such as lighting variations, background noise, motion blur, and missed detections.

The automatic and reliable tennis ball tracking system operates under diverse environmental conditions, providing precise feedback on impact locations during wall-based drills. The system detects small, fast-moving tennis balls using You Only Look Once Version 12 (YOLOv12) [3] with a self-attention mechanism, enhancing recognition of small targets and improving temporal motion consistency across frames. Candidate bounding boxes are classified with the lightweight MobileNetV2, while an accuracy-based matching algorithm filters out background noise and false positives, improving robustness and real-time performance, especially in cluttered or outdoor environments. For dynamic tracking, a Kalman filter (KF) addresses jitter and occlusion by combining historical trajectories with current observations [4]. This enables smooth trajectory estimation and predictive positioning, resulting in a stable and responsive tracking system suitable for real-time tennis training.

2. Related Work

2.1. YOLOv12

YOLOv12 is a YOLO-based object detection framework using attention mechanisms [5]. It includes the Area Attention Module (A2) for faster computation and the Residual Efficient Layer Aggregation Network for block-level residual connections, improving stability and convergence. Flash Attention optimizes memory, and unnecessary components are removed [6]. Similar YOLO frameworks are applied in sports, e.g., cricket, for shot detection and ball tracking [7].

2.2. MobileNetV2

MobileNetV2 is a lightweight CNN designed for efficient inference on resource-limited platforms. It has proven effective for object detection on low-power edge devices [8] and feature extraction for unmanned aerial vehicle localization [9]. In this study, MobileNetV2 is used to verify whether candidate bounding boxes correspond to actual tennis balls, ensuring robust recognition in real time.

2.3. KF

The KF is a recursive state estimation algorithm widely used in object tracking. Its core principle is to predict the next state based on the previous state and correct the prediction using the Kalman gain [10,11]. As shown in Equation (1), x′ₜ denotes the prior state estimate at time t, while zₜ − Hx′ₜ represents the innovation, i.e., the difference between the actual measurement zₜ and the predicted measurement x′ₜ. The Kalman gain Kₜ determines the update magnitude, and multiplying it with the innovation yields the updated posterior state estimate $x_t^{^}$

$${\mathrm{x}}_{\mathrm{t}}^{^} = {{\mathrm{x}}^{\prime }}_{\mathrm{t}} + {\mathrm{K}}_{\mathrm{t}} ({\mathrm{z}}_{\mathrm{t}} - \mathrm{H}{{\mathrm{x}}^{\prime }}_{\mathrm{t}})$$

(1)

3. Methodology

3.1. Dataset Preparation

We prepared two datasets for training YOLOv12 and MobileNetV2, respectively. The overall process was divided into two stages: in the first stage, we constructed the training dataset for YOLOv12; in the second stage, under more complex environments, the bounding boxes detected by YOLOv12 were manually filtered to confirm whether they contained tennis balls. These filtered samples were then used to build both the training and testing datasets, which were subsequently fed into the MobileNetV2 model for further training.

3.1.1. Datasets for YOLOv12

The tennis dataset, from local practice and YouTube wall videos, consisted of 1902 annotated frames. Preprocessing included auto-orientation, resizing to 640 × 640, and contrast enhancement [12]. Data augmentation—horizontal flipping and ±15% brightness/exposure changes—tripled the dataset to 4492 images. The dataset was split into 70% training, 15% testing, and 15% validation. Figure 1 shows examples before and after preprocessing and augmentation.

3.1.2. Datasets for MobileNetV2

To perform binary classification (tennis ball vs. non-tennis ball), 500 images were collected from YOLOv12 candidate bounding boxes and categorized into Tennis Ball and Not Tennis Ball. The dataset was split 80/20 for training (400 images) and validation (100 images), with a separate test set reserved. Images were resized to 224 × 224, converted to tensors, and normalized for MobileNetV2 input to improve generalization [13].

3.2. Ball Trajectory Tracking Algorithm

We developed a two-stage tennis detection and tracking method: YOLOv12 first detects candidate bounding boxes, then MobileNetV2 classifies them while the optimal tracking confidence selection algorithm filters true ball positions. These positions are input to a KF to estimate 3D position, velocity, and depth, enabling precise ball trajectory tracking [10,11], as shown in Algorithm 1.

Algorithm 1: Two-Stage Tennis Ball Detection and Tracking
	Input: Video frames from a tennis wall practice
	Output: Estimated ball positions(${ball}_x,{ball}_y,{ball}_z$), velocity(${ball}_v$)
1	Initialize KF with state vector: $\left({ball}_x,{ball}_y,{ball}_z,{ ball}_v\right)$
2	for each frame (time step $\Delta t$):
3		Predict ball state using KF;
4		Stage 1—YOLOv12 detection: Detect candidate bounding boxes $B{B}_i$ = $\left(x_c,y_c,w,h\right)$;
5		Stage 2—MobileNetV2 classification: validate candidates with probability ≥ $T_{conf}$
6		Select $B{B}_i$ with highest prob_ball and tracking confidence;
7		Update KF with selected $B{B}_i$ position;
8		if valid $B{B}_i$ exist:
9			$\mathsf{\Delta }pos\leftarrow \left(bal{l}_x^{\left(t\right)}-bal{l}_x^{\left(t-1\right)},bal{l}_y^{\left(t\right)}-bal{l}_y^{\left(t-1\right)}, bal{l}_z^{\left(t\right)}-bal{l}_z^{\left(t-1\right)}\right)$;
10			$bal{l}_v={\displaystyle \frac{\mathsf{\Delta }pos}{\mathsf{\Delta }t}}$;
11			Return Estimated ball positions(${ball}_x,{ball}_y,{ball}_z$), velocity(${ball}_v$);
12		else:
13			Return previous state(${ball}_x,{ball}_y,{ball}_z$) and velocity(${ball}_v$);
14		end if
15	end for

3.3. Ball-to-Wall Collision Detection

In wall practice scenarios, the system detects and tracks the tennis ball and estimates its motion states. At each frame, the position $\left(x,y\right)$ and velocity components $\left(v_x,v_y\right)$ are derived from the bounding box center $\left(Cx,C_y\right)$ detected by YOLO and refined by MobileNetV2. These coordinates are fed into a KF with the state vector defined as follows.

$$State=\left(x,y,v_x,v_y,z,v_z\right)$$

(2)

A collision with the wall is determined when the vertical velocity $v_y$ changes sign from negative to positive, indicating the ball’s reversal after impact. To avoid multiple detections of the same bounce, a distance threshold $d$ = 15 pixels is applied: a new rebound point $\left(x_r,y_r\right)$ is only recorded if its distance from the last detected rebound point exceeds d. This ensures robust detection and accurate statistics of wall impact events. The mathematical condition can be summarized as follows.

$$\left(x_r,y_r\right)=\left\{\begin{array}{c}{v}_y\left(t-1\right)<0 and v_y\left(t\right)>0\\ {(x}_r,y_r)\in WallZone\\ dist\left(\left(x_r,y_r\right),\left(x_{last},y_{last}\right)\right)>d\end{array}\right.$$

(3)

4. Results and Discussions

4.1. Evaluation of Wall Impact Point Detection

To evaluate the system’s accuracy in detecting tennis ball impact points on the wall, five one-minute wall practice videos were used as the test dataset. The dataset includes manually annotated ground-truth coordinates and system-detected coordinates, normalized using Euclidean distance [14]. Figure 2 shows the per-frame mean absolute error (MAE) between system-detected and ground-truth points, indicated by the red line. Most errors were below 35 pixels, with an overall MAE of 20.39 pixels, calculated as shown in Equation (4).

$$\mathrm{MAE}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\sqrt{{\left(X_{\mathrm{detected},i}-X_{\mathrm{gt},i}\right)}^2+{\left(Y_{\mathrm{detected},i}-Y_{\mathrm{gt},i}\right)}^2}$$

(4)

4.2. Model Evaluation

Using a test set of 3885 images, we evaluated both YOLOv12 and the two-stage MobileNetV2 approach. YOLOv12 achieves superior recall, successfully capturing the majority of tennis balls and maintaining stable detection performance even under challenging conditions such as boundary proximity or varying illumination (

Table 1. Comparison of detection performance between YOLOv12 and two-stage models.

Model	True Positive	False Positive	False Negative	Precision	Recall	F1-Score
YOLOv12	3351	98	276	97.2%	92.4%	94.7%
YOLOv12 + MobileNetV2 (Two-Stage)	2774	18	647	99.4%	81.1%	89.3%

). However, YOLOv12 exhibited a higher susceptibility to background noise, occasionally misclassifying non-ball objects such as fallen leaves on the court or boundary lines. Specifically, its false positives were 5.44 times higher compared to the two-stage filtering method (YOLO + MobileNetV2). In contrast, while the two-stage classifier reduced recall to 0.811, it achieved a markedly higher precision of 0.994, demonstrating its effectiveness in filtering out false positives and thereby improving the overall reliability of predictions. This trade-off highlights the design choice between maximizing detection coverage versus ensuring prediction reliability, depending on the target application scenario.

5. Conclusions and Future Works

We developed an AI-based model for tennis wall practice, capable of automatically detecting ball impact points and quantifying training performance. The two-stage filtering pipeline is adopted to reduce both false positives and false negatives, while integrating advanced detection models such as DEtection TRansformer and Shifted Window Transformer to improve the precision of candidate bounding boxes. Additionally, enhancements in illumination adaptation and frame interpolation will be explored to ensure robust system performance under dynamic and variable environmental conditions. Beyond tennis, the proposed framework shows promise for extension to other fast-paced ball sports and performance analytics scenarios, highlighting its potential for broader applicability across domains.