Investigating Intense Rainfall Influence on Distance Measurement with a Time-of-Flight Camera Sensor Using Optical Ray-Tracing Simulation Technique ^†

Abstract

Robust, fast and reliable examination of the surroundings is essential for further advancements in autonomous driving and robotics. Time-of-Flight (ToF) camera sensors are a key technology to measure surrounding objects and their distances on a pixel basis in real-time. Environmental effects, like rain in front of the sensor, can influence the distance accuracy of the sensor. Here we use an optical ray-tracing based procedure to examine the rain effect on the ToF image. Simulation results are presented for experimental rain droplet distributions, characteristic of intense rainfall at rates of 25 mm/h and 100 mm/h. The ray-tracing based simulation data and results serve as an input for developing and testing rain signal suppression strategies.

1. Introduction

Assisted and autonomous driving vehicles are currently under heavy development. Robust, fast and reliable examination of the surroundings is essential for safe operation of such vehicles even in complex situations. Time-of-Flight (ToF) camera sensors (Figure 1) are among the technologies that can deliver this information. Small atmospheric particles, like rain, fog or haze, can significantly decrease the measurement accuracy for many distance measurement principles. In real-world situations, camera-based ToF measurement errors arise from many different sources and the contribution of a single effect on the overall inaccuracy might be difficult to determine. Optical simulations allow quantitative examination of single effects and can overcome the aforementioned limitations. In this publication, we investigate how intense rainfall influences distance measurements and their accuracy on Time-of-Flight (ToF) camera sensors using ray-tracing methods.

2. Simulation Method and Considered Effects

Optical ray-tracing with Monte-Carlo methods based ray generation is used to obtain the ToF sensor’s intensity and depth signals. Ray-tracing is performed with the commercial software Zemax OpticStudio©. The model comprises the ToF sensor, including the source, the detector and the receiving optics, the scenery, composed by several cubes with scattering surfaces, and the rain droplets, modeled as water spheres (Figure 2). Upon raytracing, the intensity and travelled distance associated with each ray reaching the detector is stored for further analysis. The optical path length of each ray can be used to reconstruct the measured depth for different sensor implementations or readout schemes (e.g., Direct-ToF and Correlation-ToF). This simulation approach—out of the box—supports multiple and extended light sources and allows accounting for all effects within the geometrical optics model. Especially global illumination (e.g., multi-object reflection/scattering ray-paths, translucent objects) and aberration effects (e.g., distortion caused by the ToF sensor’s lens) are supported. Further details about the simulation procedure can be found in [2]. In the following, the Direct-ToF implementation is used to reconstruct the depth signals.

3. Results

Figure 3a–c show the intensity and depth signals of the scenery presented in Figure 2 in absence of rain. Figure 3d–i show the influence of droplet distributions at 25 mm/h and 100 mm/h rates (intense rainfall model) on the ToF sensor accuracy. Different from our previous work, where rain was modeled as a collection of water spheres homogeneous in size [3], here the droplet distributions (number and size distribution per unit volume) are modeled according to a modified Gamma distribution [4]. The rainfall rate dependent input parameters were taken from the intense rainfall model given in [5]. A rainfall rate of 24 mm/h applies to 0.5% of the rainiest month in the Northern Hemisphere tropics according to [5]. The depth accuracy of the ToF sensor is affected mostly by back-reflection of the rays from the rain droplets (Figure 3d–g). Therefore, the depth estimation results mainly in too short object distances. The 100 mm/h rate shows a larger distance error, since it roughly contains twice the number of droplets of the 25 mm/h rate. Additionally, the influence of a raindrop increases with smaller distance towards the ToF sensor.

The two histograms in Figure 3h,i give a more detailed analysis of the distance error distribution. The non-cumulative plot (Figure 3h) shows the distribution of the relative distance errors, which shows a pronounced asymmetry towards shorter distances. The positive error component amounts to 0.13% at a rainfall rate of 25 mm/h and 0.14% at a rate of 100 mm/h. Figure 3i shows the cumulative error distribution, obtained by summing up all errors from a depth difference of −30% to 0. The analysis shows that 22% of all pixels are affected by a distance error larger than 1% at the 25 mm/h rain rate, whereas this value increases to 28% of all pixels at the 100 mm/h rain rate.

4. Discussion and Conclusions

The ranging errors on ToF camera signals caused by rain are due to different ray paths which end at the same sensor pixel. There are two main interaction ray paths for rain droplets influencing the distance measurement. The first includes direct reflection or scattering from the rain droplets without further interaction with other objects. As the rain droplets are typically in front of the surrounding objects, this ray path will generate too short distance information. The second contribution arises from rays that interact with the rain after a first interaction with the scene, and are either reflected or scattered back towards the scene. This results in a too long distance estimation of the scenery. The simulation results clearly show that the first effect outweighs the second.

The comparison of the two rainfall rates shows that intense rainfall significantly lowers the accuracy of the ToF camera’s scene depth estimation. The higher examined rainfall rate shows a stronger impact on the distance reconstruction. This dependence is expected, as the higher rainfall rate comprises a larger number of droplets with larger diameters per volume unit, resulting in a larger cross-section for light-droplet interaction.

Each cube in the test scene has surfaces forming different angles with the optical axis of the ToF sensor. The Gaussian scattering model implemented on all these surfaces enables the investigation of directional influences. Surfaces with a large angle between optical axis and surface normal vector appear less bright. Hence, the relative contribution from rays interacting with rain droplets is higher for these surfaces as compared to surfaces with a high scattering coefficient towards the ToF sensor, and the rain perturbation is larger. This can be notably seen on cube edges in Figure 3d,f.

The rainfall analysis presented here serves as useful input for developing rain-perturbation suppression strategies.