A System for Analysing the Basketball Free Throw Trajectory Based on Particle Swarm Optimization

: This paper describes a system for the automatic detection and tracking of a ball trajectory during a free throw. The tracking method is based on a particle swarm optimization (PSO) algorithm. The proposed method allows for the measurement of selected parameters of a basketball free throw trajectory. Ten parameters (four distances, three velocities, and three angle parameters) were taken into account. The research material included 200 sequences captured by a 100 Hz monocular camera. The study was based on a group of 30 basketball players who played in the Polish Second Division during the 2015/2016 season and the Youth Polish National Team in 2017. The experimental results showed the differences between the parameters in both missed and hit throws. The proposed system may be used in the training process as a tool to improve the technique of the free throw in basketball.


Introduction
An important issue in sport is to analyse the factors that influence the achievement of the best sports results. In every sports discipline, there are elements whose performance has a direct impact on the final outcome. One such element in basketball includes the free throw which is very crucial in this kind of sport; therefore, it is essential to go through a scientific analysis to improve the free throw skills and hit rate of basketball players [1]. A free throw is a special component of the technical preparation of every player, which is based on automatic movement. It is always performed in the same way (with a suitable rhythm and speed).
A substantial body of research into the different aspects of basketball has been carried out to date [2][3][4][5][6]. These studies include research into the ways to improve a free throw. One example is the research conducted by Englert et al. [7] in which the scientists rated the levels of concentration of the player throwing free throws. In other studies, Hamilton and Reinschmidt [8] analysed the speed of the ball, the throw angle and the impact of those components on the effectiveness of a free throw. Button et al. [9], however, have evaluated the posture of a player during a free throw. Another paper [6] describes the analysis of selected kinematic parameters of free throws performed during the learning process, and identifies which one has the greater impact on the effectiveness of the throws. They used a 200 Hz camera and SIMI-Motion software (Simi Reality Motion Systems GmbH, Unterschleissheim, Germany). Gablonsky and Lang [3] presented a mathematical model of the free throw, which involves the initial angle, velocity and trajectory of the ball. This research has been explored further by Murphy [10]. The author focused on finding the best values for the parameters of a free throw. The player's body height, and the angle and speed of the ball were all taken into consideration. A similar problem was considered by Tran and Silverberg [11], who analysed the initial angle, speed, rotation and height of the ball's flight. An analysis presented by them showed that the angle of the ball during the throw should be about 52 • , which allows for an effectiveness level of 70%.
The quality of the technical elements is based on the precision of the movements. It is really difficult to rate because it is only a visual observation. For this reason, every year, we have many new studies concerning the automatic and semi-automatic analyses of movement in sport [4,[12][13][14][15][16][17]. For example, Rahma et al. [13] have developed a system to automatically analyse a basketball player's free throw using recorded video sequences. The system detects the player and then the main shooting positions. Barros et al. [18] have proposed an algorithm for measuring the movements of handball players. The method they proposed was able to correctly determine the movements of 84% of players when applied to the second period of the same game used for training, and 74% when applied to a different game. The research by Perše et al. [5], which provided a system to detect the basic technical elements during a basketball game, is also noteworthy. This system showed the trajectory of the players' moves in both defence and offence. Another study proposed the motion tracking of a tennis racket using a monocular camera and the markerless technique [17], whereas the work by Sheets et al. [19] makes use of a markerless motion capture system to test for kinematic differences in the lower back, shoulder, elbow, wrist, and racket between the flat, kick, and slice serves. In the study, seven male NCAA Division 1 players were tested on an outdoor court in daylight conditions. Video analysis was also used by Chen et al. [20], who presented a method based on observing the typical moves of individual players. The system automatically detects if the team are playing rather defensively or offensively.
One of the basic problems with computer vision is the detection and tracking of objects [21]. The methods used for their tracking are applied in various fields, including the army, industry, medicine, and robotics [22]. It is very difficult to track moving objects in real time, e.g., a ball in sports games [21,[23][24][25][26]. However, at present, this problem is an important component of sports analysis because the information about the position of the ball and its velocity may be obtained [26]. Difficulties in tracking the ball appear in team games because the ball is often obscured by the players. This issue was dealt with in a study by Wang et al. [27]. The authors suggested an innovative approach to the subject by defining tracking in terms of deciding which player is in possession of the ball at any given time. Kurano et al. [25] described a method of automatic ball trajectory extraction in American football. They focused on ball holder prediction and constructed a play search and 3D virtual display system using information concerning the ball trajectory and the Unity development environment. Another solution was suggested by Wang et al. [28]. To track a tennis ball, they developed a multi-view ball tracking method by particle filter. Cheng et al. also dealt with the tracking of the position of the ball in sport [29]. The model proposed by them for tracking the ball in volleyball was based on the ball size adaptive (BSA) tracking window, a ball feature likelihood model and an anti-occlusion likelihood measurement (AOLM) based on a Particle Filter for improving accuracy.
The devices that may be used to correct various technical elements should also be mentioned. One example is the ShotLoc system [30] (Hoops Innovation Ltd. LCC, Canada), whose task is to optimize the position of the player's hand on the ball during a free throw. Noah Shooting System [31] (NOAH Basketball, Athens, Alabama, USA) is a real time system that gives players immediate information on the angle of the ball flight and the distance of the ball to the basket ring. It is used to improve effectiveness and build muscle memory.
The main objective of this study is to develop a system for the measurement and analysis of selected parameters of a basketball free throw trajectory. The system is based on a particle swarm optimization (PSO) algorithm and utilised video data captured by a monocular camera. The main contribution of this work is to develop methods of ball detection, tracking and recovery after tracking failure and also to analyse the obtained parameters. The PSO algorithm has been used to track the ball. For automatic ball detection and redetection, the circularity factor and the size of segmented objects are taken into account. The estimated trajectory of the ball is used for parameter calculation. In this paper, the statistical analysis of both missed and hit throws is conducted. The analysis performed takes into account the differences in the heights of the basketball players. Graphs of dependence between the most important parameters were also determined.
The paper is an extension of a conference paper [15]. In relation to the aforementioned paper, the developed system has been significantly improved (automatic redetection of the lost ball has been added). In addition, the tracking algorithm has been thoroughly tested, and the results obtained have been compared with the results of the particle filter method. The part of the paper regarding parameter analysis has also been significantly developed.

Particle Swarm Optimization
In the ball tracking process, a particle swarm optimization algorithm (PSO) [32,33], was used. Its usefulness in solving problems related to object tracking has been repeatedly confirmed [22,[34][35][36][37][38]. PSO is a stochastic optimization technique which operates on a vector of real-value data. In that algorithm, the particle swarm is used in order to find the best solution. Each of the particles in the swarm represents a hypothetical (a candidate) solution to the problem. During an iterative process, particles explore the search space effectively by exchanging a limited amount of information. Each i-th particle contains the current position x i , velocity v i , and its best position pbest i . Moreover, the particles have access to the best global position gbest, which has been found by the swarm. The d-th components of the velocity and position of each particle are updated using the following equations: where c 1 , c 2 are positive constants equal to 2.05, r 1,d , r 2,d are taken from a uniform distribution in [0, 1] and χ is a constriction factor, which is given by the following equation [33]: where ϕ = c 1 + c 2 and ϕ > 4 The selection of the best position for the i-th particle (pbest i ) and the best global position (gbest) are based on the fitness function value f (·), which will be discussed later. The PSO algorithm in our paper solves the minimization problems, which may be expressed as follows [32]:

Camera Calibration
In order to obtain distance parameters from the image, it is necessary to determine the transition from the two-dimensional image space to the three-dimensional space, which allows for the determination of the distance between the objects in the image. In order to maximally simplify the system initialization, a simplified calibration procedure was used. This procedure is based on the distance between the characteristic objects (landmarks) on the scene (the basketball court), such as lines, the intersection of lines, basket, etc. It was assumed that the ball moves perpendicular to the camera, the distance of the ball from the camera during the flight to the basket does not change and the distortions of the camera are negligible. The calibration procedure is limited to the selection of two landmarks on the stage, which is at the same distance from the camera as the ball and giving the distance between them in the image in pixels and the real distance in meters. Given the distance between landmarks in the image, the corresponding distance in the real world and the ball position on the image, the distance parameters of the ball may be estimated. In the presented study, the distance between the free throw line (L 1 on Figure 1) and the line defining the end of basketball court (L 2 on Figure 1) was used.

Basketball Detection
An important part of the system developed to obtain the trajectory of a basketball free throw is the detection of the ball. This ball detection method enables automatic tracking initialization and redetection in case of a tracking failure. To extract the moving objects, the background subtraction algorithm developed by Zivkovic and van der Heijden [39] as well as morphological methods [40] were used. After extraction for each object, two conditions are checked; if both are true, the object is classified as a ball. The first condition concerns the size of the object and is determined by comparing the radius of the enclosing circle of the object with the radius of the ball. The first condition has the form: where r o is the radius of the enclosing circle of the considered object, r b is the radius of the ball and m is the margin factor, whose value was set at 22% of r b . The parameter r b is set during calibration; we assume that the change in the size of the ball during flight is negligible. The second condition uses the circularity factor f c = 4πA P 2 [41], where A is the area of the object and P is the perimeter of the object. The value of f c for a perfectly round object is equal to 1. The second condition is described by the formula: where T c is the circle threshold equal to 0.78. The values of m and T c have been determined experimentally. If both conditions are true, it means that the object under consideration is a ball and the tracking process can be started. Additionally, in order to minimize the risk of false detections, all of the objects in the image in front of the free-throw line and simultaneously below the chest of the reference player (player with a height corresponding to the average height of all competitors) are removed.

Ball Tracking
The proposed method of tracking is a 2D method; its result is the position of the ball in the image in the form of pixel coordinates. In our application, the position of the i-th particle represents the hypothetical position of a ball and the fitness function determines whether a considered part of the image contains the tracked object or not. This function is defined as the ratio of the number of pixels in the extracted objects that are located within the circle defined by the ball position and radius r b to the number of pixels that are outside the circle.
The block diagram of the ball detection and tracking algorithm is shown in Figure 2. At the beginning, the value of the boolean variable is_detected, which determines if the ball is detected; it is set to FALSE (see block 1 in Figure 2). Next, a new image frame is loaded (block 2) and it is determined whether or not the ball position from the previous frame is known (block 3), if not, the ball detection procedure is started (block 4, see Section 2.3). If the position of the ball in the previous frame is known, then the PSO algorithm is initialized from it (block 6). In the next steps (blocks 7,8,9,10), the algorithm tries to find the current position of the ball. In block 11, the algorithm checks whether the current position of the ball has been determined. If so, the value of the best particle gbest is assigned to the variable ball_pos. Otherwise, the variable is_detected is set to FALSE. The algorithm ends when there are no more frames available for processing (block 14).

Parameter Calculations
The parameters of the free throw (see Figure 3) may be calculated using the estimated trajectory. The analysis included ten parameters at three points of the throw. The measured parameters were: velocities in three points (v 1 , v 2 , v 3 ), angles of the moving ball (α 1 , α 2 , α 3 ), height parameters (h 1 , h 2 ) and a distance parameters (l 1 , l 2 ). It should be noted that the value of parameter h 1 is also closely related to the player's height, in addition to the player's technique. A description of the specified parameters is shown in Figure 3. The procedure for calculating the parameters from a 2D ball trajectory is listed below: • Calculate the 2D coordinates of the ball trajectory using the PSO algorithm.

•
Smooth the trajectory using the moving average filter (window = 5), • Determination of auxiliary points (P 1 -point when the ball leaves player's hands, P 2 -the highest point of the ball trajectory and P 3 -point when the ball makes contact for the first time with the rim line or with the basket board) (Figure 3). Points P 1 and P 3 determine the range of the analysed trajectory.

•
Calculation of altitude and distance parameters on the basis of determined points.

•
Determination of auxiliary vectors anchored in the analysed points to determine angular parameters.

•
The angular parameters are calculated from the formula: where: α is the calculated angle (degrees), v-vertical vector, h-horizontal vector, v is the Euclidean norm of vector v and h is the Euclidean norm of vector h.

•
Conduct a statistical analysis including basic statistical measures, a test of the significance of differences between hit and missed throws, and an analysis of the relationship between the α 1 and v 1 parameters.

Data Collection
The study was conducted on a group of basketball players who played in the Polish Second Division during the 2015/2016 season and the Youth Polish National Team in 2017. The analysed group consisted of 30 males aged 18.8 ± 2.3. The players were described by the following parameters: body height 188.0 ± 8.7 cm, body mass 80.4 ± 9.2 kg, and BMI 22.7 ± 1.5. Throughout the research, the sequence of a free throw under regulation conditions was captured. In the analysis, 200 throws were used. The sequences were captured by a monocular 100 Hz Basler Ace acA645-100gc camera (Basler AG, Ahrensburg Germany). The camera was placed 4.6 m from the predicted trajectory of the ball and perpendicular to it. It should be noted that the ideal perpendicular positioning is very difficult to implement under experimental conditions.

Algorithm Evaluation
The system developed for ball tracking was evaluated in 20 video sequences of the basketball free throw. The quality of tracking was determined by analyses carried out through qualitative visual evaluations as well as quantitatively through the use of manually obtained ground truth data. In Figures 4-6 the ball tracking results for selected players are presented. In Figures 4 and 5, the ball went into the basket. However, Figure 6 shows a missed shot. It may be observed that the proposed method for tracking the ball has very high levels of accuracy. However, in some situations, tracking errors may occur due to imperfections in the computer vision methods for extracting objects in the scene. For example, in sequence s1 (see Figure 4), the ball was detected later than in the case of other sequences, which resulted in the tracking starting with a delay (see Figure 4, frame #130). Another example may be seen in frame #148 in Figure 4 where the tracking inaccuracies caused by changes in the lighting level on stage may be observed. This change in lighting was caused by the light source being obscured by the ball. Table 1 presents the Root Mean Square Error (RMSE) for results obtained by PSO and a particle filter (PF) algorithm, which is commonly used for object tracking [21,23,28,29]. The RMSE error for one sequences has the form: where: N-is the total number of sequences, F-is the total number of frames, x i , y i -are the estimated coordinates of the ball, x GT i , y GT i -the manual settings (ground truth) coordinates of the ball. The ground truth data were obtained by manually pointing out the positions of the ball on the image sequences. For each algorithm, errors were determined for the 5 configurations selected so that for both algorithms the number of calls of the objective function was the same. For most configurations, the PSO algorithm produced much better results, with the exception of the configuration with the smallest number of particles. In addition, Table 2 contains a number of incidents in which the tracking failed, and redetection was necessary. The best result (RMSE = 5.82 pixels) was obtained for the configuration of 50 particles and 10 iterations of the PSO algorithm.  Figure 7 presents the determined trajectory of the ball in selected sequences. The trajectories were filtered using the moving average filter (window = 5). In the initial phase of tracking, the errors caused by the change in lighting conditions, mentioned earlier, may be observed (see Figure 7, sequence s1) However, it should be noted that these errors can be eliminated by carrying out additional analyses of the ball trajectory.    Table 2 shows the basic statistics of the analysed parameters (x-mean, sd-standard deviation, d-difference between the mean values of the missed and hit throws). The table presents data for all athletes grouped by body height. The parameters were calculated from 200 sequences including 155 hit and 45 missed throws. Additionally, to specify the statistical significance of the difference between hit and missed throws the t-Student's test was used (p).

Using in Practice
In the case of the parameter h 1 , measured from the lowest point of the altitude of the ball relative to the basket height, it may be observed that players who throw the ball into the basket reached the lower value of this parameter of 542 mm. The differences between hit and missed throws for this parameter had a statistical significance (p < 0.001). This parameter reached the highest value among the shortest competitors (body height 171-180 cm; 695 cm for hit and 946 for missed throws). An analysis of the data showed that the maximum altitude of the ball for all participants (h 2 ) in hit and missed throws were at the same level (approximately 3900 mm). In addition, in the body height groups, there were no significant differences in h 2 between hit and missed throws. The distance analysis of the first part of the ball trajectory l 1 (measured from P 1 to P 2 -see Figure 3), showed that this parameter was higher for hit throws (2198 mm), but the difference was relatively small (13.3 mm). Considering the length l 2 (measured from P 2 to P 3 ), it may be seen that the hit throws had a longer distance (1648 mm) than the missed throws (1549 m). From analysing the values of l 2 for individual height groups, it is noteworthy that, in each case, the value is higher for hit throws. Differences in the l 2 parameters are statistically significant (p < 0.001). The values of the initial angle α 1 for most players who throw the ball into the basket reaches 50.0 • . Since, according to earlier studies [42,43], the initial angle of the ball should be about 52 • , it may be assumed that in such a case we are dealing with a correct throw. In missed throws, this parameter was at the level of 48.9 • . The analysis conducted also showed that the lowest α 1 in hit throws occurred for athletes with a body height in the range of 191-200 cm, while the largest angle (55.8 • ) during the throw was obtained by athletes belonging to a group of people with a body height of 181-190 cm. In the case of missed throws, this parameter was placed at the lowest level (46.2 • ) among the tallest athletes. Another analysed parameter is the angle α 2 (the angle of ball trajectory at the highest point of the trajectory). The angle α 2 measured during hit throws assumes similar values for each research group with a slight advantage among athletes whose body height is between 191-200 cm. Analysing α 2 for a missed throw shows that the situation is different. In the group of the shortest athletes, this angle assumes the lowest value at the level of 8.8 • , and, among the tallest athletes, it reaches the highest value (12.1 • ). For the whole group parameter, α 2 was slightly higher for hit throws (α 2 = 11.2 • , p < 0.05). In the case of the angle α 3 , the values of this parameter are larger in the hit throws than in the missed throws (p < 0.001). However, we should pay attention to players with a body height of 191-200 cm. This group produced the lowest α 3 values during hits (44.5 • ) and missed (38.3 • ) throws. The last considered parameters are the throw velocities. According to Hamilton and Reinschmidt [8], the optimum speed v 1 , which is achieved when the ball is thrown, is approximately 7.3 m/s. By analysing v 1 , without dividing the height of bodies, for hit and missed throws, it is shown that the velocity assumes similar values (5.5 for hit and 5.6 m/s for missed throws). The lowest velocity v 1 in hit throws are produced by the tallest athletes (5.3 m/s). In the case of missed throws, the highest value (5.7 m/s) of the ball occurs during the throws by almost the tallest athletes (191-200 cm). The conducted analysis also indicated that, regardless of the height of the athlete's body, for parameters v 2 and v 3 , there were no significant differences in the values of these variables. According to Ref. [44], we also present the relationship between α 1 and v 1 for all competitors and for individual body height groups as shown in Figure 8. The blue dotted lines present the typical values for the analysed groups. For example, players with a body height of 181 cm to 190 cm should make a throw at 5.6 m/s and at an angle of about 56 • . The size of the ellipse surrounding the points represents the dispersion of the results (blue for hit throws, red for missed throws). On the basis of Figure 8, it may be observed that, for all participants, the dispersion of missed throws is significantly larger than the dispersion of hit throws. By analysing the studied dependencies in individual groups, it may be concluded that, among smaller players (171-180 cm and 181-190 cm), missed throws are characterized by a greater dispersion in terms of the parameters v 1 and α 1 . Among the tallest players, hit throws are characterized by a higher dispersion value.

Conclusions
The paper presents a system for ball detection, tracking and trajectory analysis during a basketball free throw. The experimental results were based on 200 sequences of 30 basketball players. The values of 10 parameters were measured. The results showed the differences between the parameters in missed and correct throws. The analysis showed that significant differences between missed and hit free throws were observed for parameters h 1 , l 2 , α 2 and α 3 . Additionally, an analysis of the relationship between body height and the parameters of the trajectory was calculated.
The proposed method of analysing ball trajectory in free throws may be used in practice by coaches in the training process as a tool to improve the execution of free throws. Coaches using this system will be able to monitor the trajectory of the ball and the parameters of the free throw, which will help to improve the correct motor habit. He can also eliminate some of the technical mistakes of players performing a free throw. On the basis of α 1 and v 1 , among other factors, the coach could adjust the player's body posture (i.e., upper limb positions). They could also pay attention to the initial velocity of the ball-the player may make a throw with more or less strength, which will change the speed of the ball. A basketball coach was consulted concerning the whole idea of the system. Future work will focus on improving the proposed system and developing an expert system that would allow for the automatic evaluation of performed free throws.