# Camera-Based System for Drafting Detection While Cycling

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- We trained, applied, and analyzed the performance of real-time CNN-based object detector, specifically for detecting race, time trial, and triathlon bicycles.
- We describe two methods for estimating the distance from the camera to the cyclists behind. We performed sensitivity analysis and investigated the systematic errors that can occur which are caused by making simplifying assumptions. The accuracy of the distance estimators is also verified in a realistic scenario using a Light Detection And Ranging (LiDAR) scanner.
- We developed an efficient method which determines the probability of violating the drafting rule, based on successive distance estimations and a model of the measurement error. The behavior of this method is rigorously tested in a realistic scenario and through the use of simulations.

## 2. Drafting Detection

#### 2.1. Bicycle Detection

#### 2.2. Bicycle Tracking

#### 2.3. Distance Estimation

#### 2.3.1. Wheel Position-Based Method (WPm)

#### 2.3.2. Handlebar Height-Based Method (HHm)

**that of camera rotation. An in-depth sensitivity analysis is performed in the next section**.

#### 2.3.3. Sensitivity Analysis

#### 2.3.4. Systematic Errors

#### 2.4. Drafting Probability

#### 2.4.1. Theoretical Probability Determination

- Systematic errors: when one or more parameters in the distance estimation formulae Equations (4)–(6) are erroneously set, this leads to a systematic over- or underestimation of the distance.
- The distance estimation noise: less noise means more certainty about the estimated distance; thus, the individual probabilities in Equation (7) are closer to either 0 or 1. In Section 3.2, we will investigate the noise level for our test set-up and through simulations.
- The sampling rate: for independent measurements and a given noise level and distance, there is always a higher probability that at least one of the measurements is far off from the real distance, which can significantly influence the calculated probability in Equation (9). Conversely, when the sampling rate is low, there is a higher chance of sampling bias. Hence, a trade-off exists, which will be investigated in Section 3.2.

#### 2.4.2. Efficient Probability Calculation

## 3. Evaluation and Results

#### 3.1. Distance Estimation

#### 3.1.1. Static Test

#### 3.1.2. Dynamic Test

#### 3.2. Drafting Probability

#### 3.2.1. Estimation of Measurement Noise Level

#### 3.2.2. Drafting Violation Probability

#### 3.2.3. Current Limitations and Future Work

- In its current form, the proposed system is only able to track one cyclist behind at a time. A more complex track management system should be added such that multiple cyclists can potentially be followed throughout the sequence.
- The values for ${\sigma}_{{x}_{w}}$ and ${\sigma}_{{x}_{h}}$ were now only estimated with the use of simulations. To more accurately estimate the parameters, real training data is required, with different cameras, bicycles, camera heights, tilt angles, as well as in different environments, with accurate ground truth, like we obtained in our dynamic test.
- We have assumed all distance measurements are independent. However, in practice, the measured distance between two cyclists rarely varies significantly between successive frames. This can be observed by comparing the real data extracted from the dynamic test (Figure 15) with the simulations (Figure 16). Thus, low pass filtering or the incorporation of a temporal model (e.g., a Kalman filter) with transitional probabilities could further increase the accuracy of the measurement error model. An additional potential benefit of a Kalman filter is that it automatically estimates the measurement error, which can further optimize the estimation of the drafting violation probability.
- As mentioned before and verified in this section, the estimated probability depends on the sampling rate. In addition, the exact moments the first sample is selected (just at the beginning of drafting or not) might influence the results, notably at low sampling rates and when the actual drafting time is close to the limit. The rule enforcers should be well aware of these characteristics and standardize the sampling settings and/or allow for enough buffer for questionable cases. This can, e.g., be realized by allowing slightly shorter distances and longer drafting times, or by imposing a drafting probability threshold that differs from 50%.

## 4. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A Distance for h_{1} ≠ h_{2}

- When either ${h}_{1}={h}_{2}$ or $\theta =0$, there is always one zero solution.
- When ${h}_{1}>{h}_{2}$, there is exactly one negative solution.
- When ${h}_{2}>{h}_{1}$, both solutions have the same sign, but one solution will have a much smaller value than the other, e.g., let ${h}_{2}=0.8$ m, ${h}_{1}=0.85$ m, and $\theta =10\xb0$. From Equation (A5), an object at (real distance) 10 m can theoretically be confused with an object at 132 $\mathsf{\mu}$m, which in practice cannot even be fully captured on the image plane.

## Appendix B Sensitivity Analysis of Distance Formulae

#### Appendix B.1 Height of the Camera h_{1}

#### Appendix B.2 Focal Length of the Camera f

#### Appendix B.3 Tilt Angle θ

#### Appendix B.4 Bottom Position of Detected Bounding y_{w}

#### Appendix B.5 Height of Detected Bounding Box y_{h}

## References

- Blocken, B.; van Druenen, T.; Toparlar, Y.; Malizia, F.; Mannion, P.; Andrianne, T.; Marchal, T.; Maas, G.J.; Diepens, J. Aerodynamic drag in cycling pelotons: New insights by CFD simulation and wind tunnel testing. J. Wind. Eng. Ind. Aerodyn.
**2018**, 179, 319–337. [Google Scholar] [CrossRef] - International Triathlon Union. ITU Competition Rules. Available online: https://www.triathlon.org/uploads/docs/itusport_competition-rules_2019.pdf (accessed on 15 July 2019).
- Fister, I.; Fister, D.; Fong, S.; Fister, I. Widespread mobile devices in applications for real-time drafting detection in triathlons. J. Emerg. Technol. Web Intell.
**2013**, 5, 310–321. [Google Scholar] [CrossRef][Green Version] - Liu, L.; Fang, C.; Chen, S. A Novel Distance Estimation Method Leading a Forward Collision Avoidance Assist System for Vehicles on Highways. IEEE trans. Intell. Transp. Syst.
**2017**, 18, 937–949. [Google Scholar] [CrossRef] - Huang, L.; Zhe, T.; Wu, J.; Wu, Q.; Pei, C.; Chen, D. Robust Inter-Vehicle Distance Estimation Method Based on Monocular Vision. IEEE Access
**2019**, 7, 46059–46070. [Google Scholar] [CrossRef] - Van den Bossche, S.; Philips, W.; Veelaert, P. Computervisiegebaseerde Detectie Van Drafting Tijdens Het Fietsen. Master’s Thesis, Ghent University, Gent, Belgium, 2019. [Google Scholar]
- Zhang, Z. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Anal. Mach. Intell.
**2000**, 22, 1330–1334. [Google Scholar] [CrossRef][Green Version] - Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. CoRR
**2018**, arXiv:abs/1804.02767. [Google Scholar] - Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell.
**2017**, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed][Green Version] - He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.E.; Fu, C.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the European Conference of Computer Vision (ECCV), Amsterdam, The Netherlands, 8–16 October 2016; pp. 21–37. [Google Scholar]
- LukeźIăź, A.; Vojíř, T.; Čehovin Zajc, L.; Matas, J.; Kristan, M. Discriminative Correlation Filter Tracker with Channel and Spatial Reliability. Int. J. Comput. Vis.
**2018**, 126, 671–688. [Google Scholar] [CrossRef][Green Version] - João, F. Henriques.; Caseiro Rui.; Martins Pedro.; Batista Horge. High-Speed Tracking with Kernelized Correlation Filters. IEEE Trans. Pattern Anal. Mach. Intell.
**2015**, 37, 583–596. [Google Scholar] [CrossRef] - Grabner, H.; Grabner, M.; Bischof, H. Real-Time Tracking via On-line Boosting. In Proceedings of the British Machine Vision Conference (BMVC), Edinburgh, Scotland, UK, 4–7 September 2006; pp. 6.1–6.10. [Google Scholar] [CrossRef][Green Version]
- Babenko, B.; Belongie, S.; Yang, M.H. Visual tracking with online multiple instance learning. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, USA, 20–25 June 2009; pp. 983–990. [Google Scholar] [CrossRef]
- Kalal, Z.; Mikolajczyk, K.; Matas, J. Forward-Backward Error: Automatic Detection of Tracking Failures. In Proceedings of the 20th International Conference on Pattern Recognition, Istanbul, Turkey, 23–26 August 2010; pp. 2756–2759. [Google Scholar] [CrossRef][Green Version]
- Kalal, Z.; Mikolajczyk, K.; Matas, J. Tracking-Learning-Detection. IEEE Trans. Pattern Anal. Mach. Intell.
**2012**, 34, 1409–1422. [Google Scholar] [CrossRef] [PubMed][Green Version] - Bolme, D.S.; Beveridge, J.R.; Draper, B.A.; Lui, Y.M. Visual object tracking using adaptive correlation filters. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 2544–2550. [Google Scholar] [CrossRef]
- Danelljan, M.; Häger, G.; Shahbaz Khan, F.; Felsberg, M. Accurate Scale Estimation for Robust Visual Tracking. In Proceedings of the British Machine Vision Conference. Nottingham, England, UK, 1–5 September 2014. [Google Scholar] [CrossRef][Green Version]
- Martinez, L. Bike by Left Martinez from the Noun Project, CC BY 2.0. Available online: https://thenounproject.com/term/bike/1671741/ (accessed on 17 July 2019).
- Ananth. Video by Ananth from the Noun Project, CC BY 2.0. Available online: https://thenounproject.com/term/video/938942/ (accessed on 17 July 2019).
- Rezaei, M.; Terauchi, M.; Klette, R. Robust Vehicle Detection and Distance Estimation Under Challenging Lighting Conditions. IEEE trans. Intell. Transp. Syst.
**2015**, 16, 2723–2743. [Google Scholar] [CrossRef] - Feller, W. Application to the Theory of Succes Runs. In An introduction to probability theory and its applications, 3rd ed.; John Wiley & Sons Inc.: New York, NY, USA, 1968; pp. 322–326. [Google Scholar]
- Clifford, A. Multivariate Error Analysis: A Handbook of Error Propagation and Calculation in Many-Parameter Systems; Wiley: Michigan, MI, USA, 1973. [Google Scholar]

**Figure 1.**Typical drafting rule check in a triathlon race. A motorbike rides next to the athletes, and the referee performs a visual estimate of the distance between them. If the estimated distance is too small (e.g., smaller than 10 m) for an extended period (e.g., longer than 20 s), the drafting athlete receives a penalty.

**Figure 2.**Example of a frame with manually annotated bounding boxes from our training set, captured at a non-drafting triathlon race in Kapelle-op-den-Bos, Belgium.

**Figure 3.**Tracking benchmark for CSRT (Channel and Spatial Reliability Tracker) [12], KCF (Kernelized Correlation Filters) [13], Boosting [14], MIL (Multiple Instance Learning) [15], TLD (Tracking, Learning, and Detection) [16], Medianflow [17], MOSSE (Minimum Output Sum of Squared Error) [18], and DSST (Discriminative Scale Space Tracking) [19]. (

**a**) Success plot: success rate versus Intersection Over Union (IoU) threshold. (

**b**) Area Under Curve (AUC) of (

**a**) versus execution speed.

**Figure 5.**Demonstration of how the distance can be computed from the position of the bounding box. The distance d between the two athletes is measured between the leading edges of their bicycles’ front wheels. In the proposed methods, we estimate the distance $x={x}_{w}={x}_{h}$, which differs from d by a fixed constant distance $({b}_{1}-{b}_{2})$. This figure was modified from References [20,21]. (

**a**) Wheel Position-Based method (WPm), estimated from ${y}_{w}$; (

**b**) Handlebar Height-Based method (HHm), estimated from ${y}_{h}$.

**Figure 6.**Examples of situations where the road surface is not flat: (

**a**) cobbled road; (

**b**) bridge deck.

**Figure 7.**Example of an unwanted, significant change in tilt angle for one sequence, likely after hitting a rough road patch; (

**a**) back wheel still visible; (

**b**) back wheel no longer visible.

**Figure 9.**Example of static distance calculation test. The camera bicycle is placed at the 0 m mark, and the cyclist behind takes place at one of the other marks, each spaced 1 m apart.

**Figure 10.**Error plots for the static test for both distance estimation methods. (

**a**) Signed error; (

**b**) absolute error.

**Figure 11.**Example of dynamic distance calculation test. An electric cargo bicycle with mounted Light Detection And Ranging (LiDAR) scanner drives next to the cyclists. The LiDAR data provides ground truth for the distance determination. (

**a**) Input image from the camera mounted on Bicycle 1; (

**b**) top view of the LiDAR scanlines.

**Figure 12.**Measured distances on our dynamic test sequence. The draft zone is colored pink. The LiDAR data is considered ground truth.

**Figure 13.**Error plots for the dynamic test for both distance estimation methods. (

**a**) Signed error; (

**b**) absolute error.

**Figure 15.**Drafting violation probability in two parts of our dynamic test. The distance to the cyclist was estimated with the HHm (red solid line). The ground truth distances are shown with red dots as markers. The draft zone is colored pink. The evolution of the drafting violation probability is superimposed on the same graphs (blue solid line). A blue dashed vertical line indicates when $P\left({v}_{1:n}\right|{\mathbf{x}}_{1:n})>0.5$ for the first time. The sampling period for both the LiDAR and the sampled distance measurements was $1s$. (

**a**) Cyclist behind driving to edge of draft zone, then further away, before approaching again and eventually entering the draft zone after approximately 27 s; (

**b**) cyclist behind driving to edge of draft zone, stays close to edge, before eventually entering the draft zone after approximately 27 s.

**Figure 16.**Example simulations of drafting violation probability over a given time frame. The cyclist enters the drafting zone in the beginning, and then settles at a new distance after 10 s (red dashed line). Gaussian noise is independently added to all parameters to simulate an estimated distance (red solid line). The draft zone is colored pink. The evolution of the drafting violation probability is superimposed on the same graphs (blue solid line). A blue dashed vertical line indicates when $P\left({v}_{1:n}\right|{\mathbf{x}}_{1:n})>0.5$ for the first time. (

**a**) Cyclist riding at 9 m, sampling period 2 s, small systematic errors; (

**b**) cyclist riding at 8 m, sampling period 1 s, systematic overestimation of the true distance because ${h}_{1}>{h}_{2}$.

**Figure 17.**Histogram of time before $P\left({v}_{1:n}\right|{\mathbf{x}}_{1:n})>0.5$, with different sampling periods ${T}_{s}$ and for different distances ${d}_{f}$. One thousand iterations were performed for every combination of distance and sampling period. Ideally, the algorithm should award a penalty after ${T}_{L}=20$ s of drafting, and never when the cyclist stays out of the draft zone.

**Table 1.**Approximate relative errors with respect to the different parameters in the distance estimation formulae (4)–(5) derived from sensitivity analysis. We assume that the tilt angle $\theta $ is small and $\partial u\phantom{\rule{-1.111pt}{0ex}}/\phantom{\rule{-0.55542pt}{0ex}}\partial v\approx \Delta u\phantom{\rule{-1.111pt}{0ex}}/\phantom{\rule{-0.55542pt}{0ex}}\Delta v$ for all parameters u and v.

Method | WPM | HHm |
---|---|---|

Relative errors | $\frac{\Delta {x}_{w}}{{x}_{w}}$ | $\frac{\Delta {x}_{h}}{{x}_{h}}$ |

Camera height ${h}_{1}$ | $\frac{\Delta {h}_{1}}{{h}_{1}}$ | $\frac{\Delta {h}_{1}}{{h}_{1}}$ |

Focal length f | $\frac{\Delta f}{f}$ | $\frac{\Delta f}{f}$ |

Tilt angle $\theta $ | $-\Delta \theta {\displaystyle \frac{2}{sin\left(2\alpha \right)}}$ | $-\Delta \theta tan\alpha $ |

Bottom position of bounding box ${y}_{w}$ | $\frac{\Delta {y}_{w}}{{y}_{w}}$ | not relevant |

Height of bounding box ${y}_{h}$ | not relevant | $-{\displaystyle \frac{\Delta {y}_{h}}{{y}_{h}}}$ |

**Table 2.**Estimated mean and standard deviations of the normally distributed parameters in our simulations. These estimations were obtained through observation of realistic measurements. Note that the standard deviation for ${h}_{2}$ is significantly bigger than the standard deviation for ${h}_{1}$ because the latter can be measured much easier in a real application. Furthermore, we assume that the standard deviation for the error on $\theta $ is much smaller when it is actually measured (e.g., in Equation (4)) than when we assume it is always 0 (e.g., in Equation (6)). Finally, we note that the means of ${y}_{w}$ and ${y}_{h}$ are both distance dependent, so we did not include them in this table.

Parameter | Mean | Error Standard Deviation $\mathit{\sigma}$ (estimated) |
---|---|---|

Camera height ${h}_{1}\left(m\right)$ | $0.900$ | $0.005$ |

Bicycle height ${h}_{2}\left(m\right)$ | $0.900$ | $0.025$ |

Focal length $f\left(\mathrm{pixels}\right)$ | 1000 | 25 |

Tilt angle ${\theta}_{approx}(\xb0)$, with approximation Equation (6) | 0 | 5 |

Tilt angle $\theta (\xb0)$, without approximation | 0 | 1 |

Bottom position of bounding box ${y}_{w}\left(\mathrm{pixels}\right)$ | not relevant | 5 |

Bounding box height ${y}_{h}\left(\mathrm{pixels}\right)$ | not relevant | $5\sqrt{2}$ |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Allebosch, G.; Van den Bossche, S.; Veelaert, P.; Philips, W. Camera-Based System for Drafting Detection While Cycling. *Sensors* **2020**, *20*, 1241.
https://doi.org/10.3390/s20051241

**AMA Style**

Allebosch G, Van den Bossche S, Veelaert P, Philips W. Camera-Based System for Drafting Detection While Cycling. *Sensors*. 2020; 20(5):1241.
https://doi.org/10.3390/s20051241

**Chicago/Turabian Style**

Allebosch, Gianni, Simon Van den Bossche, Peter Veelaert, and Wilfried Philips. 2020. "Camera-Based System for Drafting Detection While Cycling" *Sensors* 20, no. 5: 1241.
https://doi.org/10.3390/s20051241