# GPS-Aided Video Tracking

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Data Acquisition and Processing Methods

#### 2.1. The Approach

_{i}, camera), which are information tuples containing at least the object position and the timestamp. The latter is necessary in cases where there is no temporal synchronization between camera and GPS devices.

#### 2.2. Input Data from Sensors

_{i}= (position, timestamp, objected)

_{i}.

_{1}, d

_{2}, …, d

_{m}} with d

_{i}= (position, timestamp, histogram)

_{i}. This presupposes that detections represent individual objects, which, however, can be violated, when people are close to each other.

**Figure 2.**(

**a**) The background image of the observed scene. (

**b**) The detected changes. (

**c**) The resulting object detections framed by red bounding boxes.

#### 2.3. Preprocessing the Data

#### 2.4. Data Fusion

**Figure 3.**The data assignment task: The initial setting contains two continuous GPS trajectories (connected red and yellow dots) and unassigned camera detections (isolated blue boxes), both in the common local coordinate system. The time progression is indicated below.

#### 2.4.1. Hidden Markov Models

_{ij}}, a

_{ij}= P(S

_{j}(t + 1)|S

_{i}(t))

_{jk}}, b

_{jk}= P(V

_{k}(t)|S

_{j}(t))

_{j}and

_{max}

_{max}is chosen depending on the scenario.

**Figure 4.**The HMM for multiple trajectories. On top, the dynamic Bayes network formed by the sequence of states (in white). The colored nodes below are the possible state values (the assignment tuples) for the state sequence S1 to S6. The colors encode the number of dummy assignments (green: only real; yellow: at least one dummy; red: only dummy assignments). The gray edges symbolize the transition possibilities between the corresponding states (shown for the transitions between t

_{0}and t

_{1}).

_{1}and S

_{2}are given with K being the number of objects and H

_{1,i}and H

_{2,i}being the ith histogram of the first and second tuple, respectively,

_{i}are the pairwise Euclidean distances between the positions in the state S and the positions from the assigned (GPS) observations V. The GPS inaccuracy is modeled using the standard deviation σ

_{GPS}.

#### 2.4.2. The Viterbi Algorithm

_{t}is the probability of the most probable state sequence.

#### 2.5. Output Trajectories

**Figure 6.**The trajectories (red, orange) are generated from the assignment tuples contained in the nodes of the Viterbi path. Some detections have been discarded (isolated gray boxes).

#### 2.6. Performance of the Algorithm

^{2}t), where t is the number of time steps. Therefore, it is important to use an efficient object detection algorithm, which minimizes the number of false detections.

## 3. Experimental Section

#### 3.1. Experiments

#### 3.2. Experimental Setup

#### 3.2.1. Experiment 1—Accuracy

**Figure 7.**Overview of the setup for the first and second experiment: The four edge points waypoints for the first experiment are marked in red. The gray traces are the locations of the detections of the camera.

#### 3.2.2. Experiment 2—Completeness and Correctness

_{GPS}= 12 m, v

_{max}= 8m/s.

#### 3.2.3. Experiment 3—Multiple Object Tracking

_{GPS}= 12 m, v

_{max}= 8m/s.

## 4. Results

#### 4.1. Experiment 1—Accuracy

**Figure 8.**The resulting trajectories of the accuracy experiment: GPS (black) and result of our approach (red). The dots are the object current position determined by GPS (black) and our approach (red). The ground truth polygon is marked by a dashed blue line.

#### 4.2. Experiment 2—Correctness of Assignments

With GPS | Without GPS | |
---|---|---|

Total (objects/unassignable) | 2238 (2030/208) | 2238 (2030/208) |

Recall (%) | 2109 (94.2%) | 2013 (89.9%) |

Misses (%) | 129 (5.8%) | 225 (10.1%) |

**Figure 9.**The results of the second experiment: (

**a**) the scene containing both tracked objects and their trajectories; (

**b**) top view of the complete dataset including the trajectories generated by GPS (both in black) and by our approach (red, green). Furthermore, three regions A, B and C (blue) are marked which contain situations being referred to in the text. (

**c**) The result when GPS information is not used.

**Figure 10.**Situation in which a person leaves the field of view for about 8 s. The algorithm manages to keep the correct assignment (sequence from top left to bottom right).

**Figure 11.**Another situation where the person marked green is occluded for about 1.5 s (8 frames). The assignment is continued correctly after the situation has cleared up.

#### 4.3. Experiment 3—Multiple Object Tracking

**Figure 12.**An example for the “retroactive” assignment correction: (

**a**) the last trajectory parts (yellow box) are assigned wrongly; (

**b**) the assignment has been corrected a few time steps later.

**Figure 14.**The generated colored trajectories of the tracked players are close to their reference traces (black) (

**left**). The colored dots represent the current player positions determined by our approach. The gray ones are the reference positions. The trajectories obtained from the video-only version of our approach (

**right**). Significantly, the red and the green players are often mixed up with other players.

## 5. Conclusions and Outlook

^{2}) for each frame where m is the number of possible states, which is mainly determined by the number of detections. Although we have shown that the approach works well, there are several open issues for future work. First, the presented approach can be refined and extended. For instance, we could integrate a kinematic movement model for the objects, as well as a more rigorous modeling of the observations, basically leading to a full-fledged Kalman filter step for the continuous state variables. This would ease the integration of other types of observations, at different measurement frequencies, e.g., GPS and radio tracking, camera and radio tracking or multiple camera tracking. Regarding our image processing approach, there are several possible improvements. The image detection could be made more robust if prior information about the expected position were available, such as the position predicted by a kinematic movement model. The computed feature similarities, so far the similarities of the hue histograms, could be extended by image correlation or tracking approaches, which are expected to work better if the persons wear similarly colored clothing. Also, the features derived from the image observations should be part of the state so that they are updated instead of replaced in every time step.

## Acknowledgments

## Author Contributions

- Udo Feuerhake: Literature review, modeling, programming, data acquisition, analysis, writing.
- Claus Brenner: Modeling, revisions.
- Monika Sester: Basic concept, revisions.

## Conflicts of Interest

## References

- Home: Hawk-Eye. Available online: http://www.hawkeyeinnovations.co.uk/?page_id=1011 (accessed on 9 January 2015).
- Coutts, A.J.; Duffield, R. Validity and reliability of GPS devices for measuring movement demands of team sports. J. Sci. Med. Sport
**2010**, 13, 133–135. [Google Scholar] [CrossRef] [PubMed] - Gray, A.J.; Jenkins, D.; Andrews, M.H.; Taaffe, D.R.; Glover, M.L. Validity and reliability of GPS for measuring distance travelled in field-based team sports. J. Sports Sci.
**2010**, 28, 1319–1325. [Google Scholar] [CrossRef] [PubMed] - Johnston, R.J.; Watsford, M.L.; Kelly, S.J.; Pine, M.J.; Spurrs, R.W. The Validity and reliability of 10 Hz and 15 Hz GPS units for assessing athlete movement demands. J. Strength Cond. Res. 2013. [CrossRef] [PubMed]
- Randers, M.B.; Mujika, I.; Hewitt, A.; Santisteban, J.; Bischoff, R.; Solano, R.; Zubillaga, A.; Peltola, E.; Krustrup, P.; Mohr, M. Application of four different football match analysis systems: A comparative study. J. Sports Sci.
**2010**, 28, 171–182. [Google Scholar] [CrossRef] [PubMed] - Varley, M.C.; Fairweather, I.H.; Aughey, R.J. Validity and reliability of GPS for measuring instantaneous velocity during acceleration, deceleration, and constant motion. J. Sports Sci.
**2012**, 30, 121–127. [Google Scholar] [CrossRef] [PubMed] - Barris, S.; Button, C. A review of vision-based motion analysis in sport. Sports Med.
**2008**, 38, 1025–1043. [Google Scholar] [CrossRef] [PubMed] - Xing, J.; Ai, H.; Liu, L.; Lao, S. Multiple player tracking in sports video: A dual-mode two-way bayesian inference approach with progressive observation modeling. IEEE Trans. Image Process.
**2011**, 20, 1652–1667. [Google Scholar] [CrossRef] [PubMed] - Iwase, S.; Saito, H. Parallel tracking of all soccer players by integrating detected positions in multiple view images. In Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, 26 August 2004.
- Barros, R.M.L.; Misuta, M.S.; Menezes, R.P.; Figueroa, P.J.; Moura, F.A.; Cunha, S.A.; Anido, R.; Leite, N.J. Analysis of the distances covered by first division brazilian soccer players obtained with an automatic tracking method. J. Sports Sci. Med.
**2007**, 6, 233–242. [Google Scholar] [PubMed] - Yang, T.; Pan, Q.; Li, J.; Li, S.Z. Real-time multiple objects tracking with occlusion handling in dynamic scenes. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20–26 June 2005.
- Sugimura, D.; Kitani, K.M.; Okabe, T.; Sato, Y.; Sugimoto, A. Using individuality to track individuals: Clustering individual trajectories in crowds using local appearance and frequency trait. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 27 September–4 October 2009.
- Yang, C.; Duraiswami, R.; Davis, L. Fast multiple object tracking via a hierarchical particle filter. In Proceedings of the Tenth IEEE International Conference on Computer Vision, Beijing, China, 17–21 October 2005.
- Rabiner, L.; Juang, B.H. An introduction to hidden Markov models. IEEE ASSP Mag.
**1986**, 3, 4–16. [Google Scholar] [CrossRef] - Forney, J.G.D. The viterbi algorithm. IEEE Proc.
**1973**, 61, 268–278. [Google Scholar] [CrossRef] - Martinerie, F. Data fusion and tracking using HMMs in a distributed sensor network. IEEE Trans. Aerosp. Electron. Syst.
**1997**, 33, 11–28. [Google Scholar] [CrossRef] - Zen, H.; Tokuda, K.; Kitamura, T. A Viterbi algorithm for a trajectory model derived from HMM with explicit relationship between static and dynamic features. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Montreal, QC, Canada, 17–21 May 2004.
- Duckham, M. Decentralized Spatial Computing: Foundations of Geosensor Networks; Springer: Berlin, Germany, 2012. [Google Scholar]
- Yilmaz, A.; Javed, O.; Shah, M. Object tracking: A survey. ACM Comput. Surv.
**2006**, 38, 1–45. [Google Scholar] [CrossRef] - Zivkovic, Z. Improved adaptive Gaussian mixture model for background subtraction. In Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, 23–26 August 2004.
- Rabiner, L. A tutorial on hidden Markov models and selected applications in speech recognition. IEEE Proc.
**1989**, 77, 257–286. [Google Scholar] [CrossRef] - Dugad, R.; Desai, U.B. A Tutorial on Hidden Markov Models; Signal Processing and Artificial Neural Networks Laboratory Department of Electrical Engineering Indian Institute of Technology: Bombay, India, 1996. [Google Scholar]
- Xie, X.; Evans, R. Multiple target tracking using hidden Markov models. In Proceedings of the Record of the IEEE 1990 International Radar Conference, Arlington, VA, USA, 7–10 May 1990.
- DEBS 2013. Available online: http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails (accessed on 4 July 2015).

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Feuerhake, U.; Brenner, C.; Sester, M.
GPS-Aided Video Tracking. *ISPRS Int. J. Geo-Inf.* **2015**, *4*, 1317-1335.
https://doi.org/10.3390/ijgi4031317

**AMA Style**

Feuerhake U, Brenner C, Sester M.
GPS-Aided Video Tracking. *ISPRS International Journal of Geo-Information*. 2015; 4(3):1317-1335.
https://doi.org/10.3390/ijgi4031317

**Chicago/Turabian Style**

Feuerhake, Udo, Claus Brenner, and Monika Sester.
2015. "GPS-Aided Video Tracking" *ISPRS International Journal of Geo-Information* 4, no. 3: 1317-1335.
https://doi.org/10.3390/ijgi4031317