Quantifying Automotive Lidar System Uncertainty in Adverse Weather: Mathematical Models and Validation

Abstract

Lidar technology is a key sensor for autonomous driving due to its precise environmental perception. However, adverse weather and atmospheric conditions involving fog, rain, snow, dust, and smog can impair lidar performance, leading to potential safety risks. This paper introduces a comprehensive methodology to simulate lidar systems under such conditions and validate the results against real-world experiments. Existing empirical models for the extinction and backscattering of laser beams are analyzed, and new models are proposed for dust storms and smog, derived using Mie theory. These models are implemented in the CARLA simulator and evaluated using Robot Operating System 2 (ROS 2). The simulation methodology introduced allowed the authors to set up test experiments replicating real-world conditions, to validate the models against real-world data available in the literature, and to predict the performance of the lidar system in all weather conditions. This approach enables the development of virtual test scenarios for corner cases representing rare weather conditions to improve robustness and safety in autonomous systems.

1. Introduction

Developed in the 1960s, lidar (light detection and ranging) technology was originally used in spacecraft and planes for surface mapping. In 1971, Apollo 15 used this technology to map the surface of the moon as part of its scientific mission. The penetration of this technology in other fields, in particular automotive, took several decades. The German automotive OEM Audi was the first to release in 2017 the Audi A8, equipped with state-of-the-art technology, capable to supporting SAE level 3 autonomous driving, underpinned by the first automotive-grade lidar application. While a large variety of sensors, such as front cameras and radar, were used in the automotive industry for years, lidar was essential to make the attainment of this level of autonomy possible. Lidar’s perception of the environment is much more precise than radar, and it is more reliable than front cameras, as the lidar has a higher resolution and illuminates its field of view.

The lidar emits short laser pulses with a very high sample rate and scans its field of view with a detection system. Following the impact with an object, the laser beam is reflected back to the lidar. Based on the angular information and time of flight of the detected photons, the lidar will construct a point cloud of the surroundings, as illustrated in Figure 1. However, environmental phenomena, such as rain and fog, impact the lidar’s performance by diverting photons and thus reducing the efficiency of the system and impacting the robustness of the system due to the noise associated with the perception of the environment.

Many Advanced Driver Assistance Systems (ADAS) rely on sensing technologies and components such as lidar to underpin autonomous driving features. Higher levels of automated driving commonly employ a sensor fusion approach [1,2] to centrally process the information from multiple sensors and combinations of sensors to enhance the robustness of the driving environment monitoring. Given the safety consequences associated with the robust and dependable operation of ADAS systems, which must be assured throughout the lifetime of the vehicle, suppliers of such technologies are legally obliged to prove the robustness of their products. The German industrial standard, exemplified by the OEM specific norm VW 80000 [3], specifies an expected lifetime of a vehicle of 15 years, an operating time of 8000 h and a mileage of 300,000 km. During the development phase, the product must be validated by performing various software, system, or hardware-related tests, such as rigorous accelerated life tests, which simulate the aging process during its lifetime, considering the diverse mission profiles experienced in real-world usage. As an example, to comply with the IEC 60068-1 [4], all OEMs have adapted their test specifications corresponding to the mission profiles and the climate and road conditions in their target markets. This requires extensive validation programs that are very costly in terms of resources and time, with accelerated test durations often exceeding 100 days. Validation also requires proving the functional safety of the component, as defined by ISO 26262 [5], which means that circumstances where the component might not be safe to use have to be robustly identified. The concept of “corner case” is often used in the literature relating to autonomous systems to refer to points that mark the limits of the safe zone.

In the literature and in practice, there are different definitions of what a corner case is. For example, ref. [6] describes corner cases as non-predictable relevant objects located in a relevant position, whereas [7] includes overexposed sensors and partially concealed objects. A different approach is suggested by [8], where three types of corner cases are suggested. The first type covers hardware-related cases, such as a broken lens, whereas the second type is concerned with the attributes of the environment, where, for example, a person in a costume could not be detected as a relevant object. A third type is defined as the temporal layer related to cases where the sequence of individual scenes forms an overall result, such as the observation of a traffic accident. In order to identify or construct such corner cases, simulations or real ADAS components can be operated in large environmental simulation chambers to record datasets containing corner cases.

Significant effort was made to evaluate the impact of the weather conditions on the lidar performance. Rasshofer et al. The work in [9] laid the foundation in this topic by providing an overview of the physics of the lidar detection during adverse weather and estimating the theoretical disturbance of the detection. An expansion of this work was presented in [10], where the physical model of the backscattered noise was improved and a computer algorithm to simulate adverse weather situations was introduced. In the meantime, experiments on real lidar systems under defined weather conditions were reported in [11,12], with an analysis of the detection performance. However, the prior work only focused on weather conditions, which are common in Europe, namely rain, fog, and snowfall. As lidar systems will be utilized in all parts of the world, dust and polluted air may have a significant impact on the automotive lidar detection performance. Furthermore, empirical models for calculating the extinction and backscattering coefficient are not available for all weather conditions, although they are essential for a realistic simulation-based validation of the lidar performance under real-world conditions.

The aim of this paper is to establish a comprehensive methodology for modeling and simulating lidar systems under the influence of adverse weather conditions. The contributions of the work presented in this paper are twofold: (i) revised empirical models for the extinction and backscattering coefficients are introduced for all of the five major environmental factors, namely rain, fog, snowfall, dust and particulate pollution with potential impact upon the efficiency and safety of automotive lidar applications; and (ii) the development and validation of a physics-based virtual test environment for the simulation and evaluation of automotive lidar systems under real-world adverse weather and traffic conditions.

The organization of this paper is as follows: Section 2 provides a literature review of the mathematical and physical models. In Section 3, the methodology of this work is explained by articulating two research questions and the setup of the simulation environment. Section 4 discusses the proposed empirical laws for the weather-related coefficients, and Section 5 contains a comparison of simulations, which utilize the proposed empirical models, and real-world experiments. Finally, in Section 6, the results are discussed, and in Section 7, this work comes to a conclusion.

2. Literature Review

2.1. Mathematical Models for Lidar Applications

As described above, lidar systems emit laser beams with which they scan their surroundings. Here, the laser beam travels through the air, hits an object, is reflected back, passes through the lidar’s optical detection system, and is registered by photo diodes. The detected laser power $P_{\mathit{\lambda }}\left(r\right)$ depends on the sensor efficiency $P_{\mathrm{Sensor}}\left(r\right)$, the reflection characteristics of the object $\beta \left(r\right)$, and the atmospheric losses $P_{\mathrm{loss}}\left(r\right)$, which depend on the distance r of the object to the lidar. Accordingly, the detected laser power $P_{\mathit{\lambda }}\left(r\right)$ can be described in Equation (1):

$$\begin{array}{c}\hfill P_{\mathit{\lambda }}\left(r\right)=P_{\mathrm{Sensor}}\left(r\right)\beta \left(r\right)P_{\mathrm{loss}}\left(r\right).\end{array}$$

(1)

The influence of the three factors impacting the detected laser power $P_{\mathit{\lambda }}\left(r\right)$ are discussed below. The sensor efficiency $P_{\mathrm{Sensor}}\left(r\right)$ equation is shown in Equation (2).

$$\begin{array}{c}\hfill P_{\mathrm{Sensor}}\left(r\right)=P_0{\displaystyle \frac{c\tau }{2}}A\eta {\displaystyle \frac{\xi \left(r\right)}{r^2}}\end{array}$$

(2)

The first group of factors in Equation (2) denotes the intrinsic characteristics of the lidar system. Here, $P_0$ is the initially emitted laser power, c is the speed of light, $\tau$ stands for the spatial pulse width, the active sensor surface is described by A, and the internal sensor losses are indicated by $\eta$. $\xi \left(r\right)$ stands for a sensor-specific function, which describes the overlap of two cones that originate from the emitter and detector in a bistatic beam setup (see Figure 2). These factors can be viewed as constant for this consideration and are not of deeper interest.

The backscattering coefficient $\beta \left(r\right)$ is an external factor, which is responsible for two things: firstly, if the laser beam hits an real target, some photons will be scattered back to the lidar, depending on the objects reflective characteristics; secondly, it will add noise to the point cloud by reflecting photons back to the lidar before the beam hits a real target. Small airborne particles cause scattering effects and scatter some photons back to the lidar sensor and will be detected by the detection system.

The atmospheric loss factor, $P_{\mathrm{loss}}\left(r\right)$, shown in Equation (3), stands for the distance-related signal loss based on the extinction coefficient $\alpha$. Photons will get absorbed or scattered by air molecules or particles. Here, the physical characteristics, namely the refractive index n and the particle diameter, have a major impact on the coefficient. The equation considers the photon beam’s journey to the object and back to the sensor, hence the “2” factor in the exponent term.

$$\begin{array}{c}\hfill P_{\mathrm{loss}}\left(r\right)=exp\left(-2{\int }_0^r{\alpha }_{\mathrm{ext},˘}\left(r^{\prime }\right)d{r}^{\prime }\right)\end{array}$$

(3)

Plugging the Equations (2) and (3) in Equation (1) yields the laser equation as stated by [13]:

$$\begin{array}{c}\hfill P_{\mathit{\lambda }}\left(r\right)=P_0{\displaystyle \frac{c\tau }{2}}A\eta {\displaystyle \frac{\xi \left(r\right)}{r^2}}\beta \left(r\right)exp\left(-2{\int }_o^r{\alpha }_{\mathrm{ext},˘}\left(r^{\prime }\right)d{r}^{\prime }\right),\end{array}$$

(4)

Although Equation (4) incorporates the effect of relevant environmental factors, the equation is not sufficient in a technical context, due to the fact that the laser diode array and the detector are in practice spatially separated as illustrated in Figure 2. This configuration is known as a bistatic beam setup. Depending on the distance r to the lidar, a potential target is only partially illuminated ($R_1\le r\le R_2$) or even not at all ($r\le R_1$). Additionally, lidar uses pulsed lasers to scan the surroundings, which is also not considered in Equation (4). To address this issue, [9] have proposed lidar Equation (5), which considers the constraints of a physical device.

$$\begin{array}{c}\hfill \begin{array}{c}\hfill P_r\left(r\right)={\displaystyle \frac{c\eta A}{2}}{\int }_0^{rc/2}{P}_0{sin}^2\left({\displaystyle \frac{\pi }{{\tau }_{\mathrm{H}}}}t\right){\displaystyle \frac{exp(-2{\int }_0^r{\alpha }_{\mathrm{ext},˘}\left(r^{\prime }\right)d{r}^{\prime })}{r^2}}\xi \left(r\right){\beta }_0\delta (r-{\displaystyle \frac{c{t}^{\prime }}{2}}-R_0)d{t}^{\prime }\end{array}\end{array}$$

(5)

While structurally similar to (4), Equation (5) includes some additional factors: the squared sine function describes the pulsed laser wave, $\xi \left(r\right)$ is the illumination factor of the bistatic beam configuration, ${\beta }_0$ is the reflection coefficient of the object surface, which depends on the color and surface structure of the object, e.g., cars’ refelctivity varies between a high of 80% (for white cars) and a low of 4% for black cars [14,15,16]. The delta function $\delta (r-{\displaystyle \frac{c{t}^{\prime }}{2}}-R_0)$ at the end stands for the spatial response of reflected laser power of a so-called hard target. A hard target in this context is a solid object that cannot be penetrated by the laser beam. By contrast, a soft target, such as rain, fog, or even a glass window, is considered to be penetrable by the laser beam. For soft targets, Equation (5) has to be adapted, e.g., [10] have proposed a model for soft targets (Equation (6)), derived from the model proposed by [9].

$$\begin{array}{c}\hfill \begin{array}{c}\hfill P_r^{\mathrm{soft}}(\mathrm{r})={\displaystyle \frac{c\eta A}{2}}{\int }_0^{rc/2}{P}_0{sin}^2\left({\displaystyle \frac{\pi }{{\tau }_{\mathrm{H}}}}t\right)\\ \hfill ·{\displaystyle \frac{exp(-2{\int }_0^r{\alpha }_{\mathrm{ext},˘}(r^{\prime }-{\displaystyle \frac{c{t}^{\prime }}{2}})d{r}^{\prime })}{{(r-{\displaystyle \frac{c{t}^{\prime }}{2}})}^2}}\xi (r-{\displaystyle \frac{c{t}^{\prime }}{2}})\beta \\ \hfill ·U(R_0-r+{\displaystyle \frac{c{t}^{\prime }}{2}}{R}_0)d{t}^{\prime }\end{array}\end{array}$$

(6)

As soft targets are placed in between the lidar and a hard target of interest (e.g., rain droplets in the path of the laser beam), Equation (6) calculates the potential reflected power based on the backscattering coefficient $\beta$ of the scattering particles. Equation (6) also considers the fact that as soon as the beam reaches the hard target at position $R_0$, backscattering of the particle ceases to exist, so a Heaviside step function U replaces the delta function of a hard target.

Equation (6) shows that in adverse weather conditions, the extinction and backscattering coefficients can have a major impact on the lidar’s perception. There are two different ways to calculate these coefficients: (i) calculating them directly using Mie Scattering and specific particle density functions, or (ii) by using empirical models. Both approaches are discussed briefly in the next sections.

2.2. Mie Scattering

The theoretical models behind Mie Scattering can be directly derived from the Maxwell equations, given that the extinction and backscattering efficiencies $Q_{\mathrm{ext}}$ and $Q_{\mathrm{back}}$ depend on the particle geometry and refractive index of the particle material. To compute the extinction coefficient $\alpha$ or backscattering coefficient $\beta$ requires integration over the particle diameter D [17]:

$$\begin{array}{c}\hfill \alpha ={\displaystyle \frac{\pi }{8}}{\int }_0^{\infty }{D}^2{Q}_{\mathrm{ext}}\left(D\right)N\left(D\right)dD\end{array}$$

(7)

$$\begin{array}{c}\hfill \beta ={\displaystyle \frac{\pi }{8}}{\int }_0^{\infty }{D}^2{Q}_{\mathrm{back}}\left(D\right)N\left(D\right)dD\end{array}$$

(8)

where $D^2$ denotes the particle surface factor, Q the efficiency, and $N\left(D\right)$ is a weather-specific particle density function. The factor $\pi /8$ before the integral is related to the cross section of a sphere ($\pi r^2$) and the conversion from radius r to diameter D for these equations. Exemplary density functions can be found in the literature, e.g., [9,18]. Figure 3 shows the scattering anisotropy for water and SiO₂, the main mineral of sand. The scattering anisotropy is dimensionless and is denoted as g and takes values between −1 and 1, where 1 stands for completely forward scattering, 0 for an uniform scattering into all directions, and −1 for total backscattering. As SiO₂ has a higher refractive index, the particle will scatter less light into the forward direction compared with water.

2.3. Review of Empirical Models for Extinction and Backscattering

Empirical models have been developed in the literature relating to different environmental conditions, as discussed in the following subsections.

2.3.1. Empirical Models for Fog

Fog can occur for a variety of reasons, depending on the geographical conditions. Under continental conditions, fog can occur mostly in spring and autumn, when during clear and windless nights, warm humid air radiates its energy into the upper atmosphere, thereby cooling down and reaching the condensation point of water, causing small water droplets to form in the air. This type of fog is called, quite intuitively, radiation fog. Another mechanism of fog is referred to as advection fog, occurring mostly in coastal areas, where warm, humid air from the sea gets pushed over cold land masses by wind. As the air cools down, the carried water begins to condensate [18].

Several empirical models have been proposed for fog. Naboulsi et al. [19] proposed expressions for the extinction coefficient $\alpha$ models based on the visibility V [m], for both advection and radiation fog mechanisms, as shown in

Table 1. Comparison of different coefficient models for fog.

Coefficient	Author	Fog Type	Equation
$\alpha$	[19]	Advection	${\alpha }_{\mathrm{Adv}}(\mathit{\lambda })={\displaystyle \frac{0.11478\mathit{\lambda }+3.8367}{V}}$
	[19]	Radiation	${\alpha }_{\mathrm{Rad}}(\mathit{\lambda })={\displaystyle \frac{0.18126{\mathit{\lambda }}^2+0.13709\lambda +3.7502}{V}}$
	[20]	Generic	$\alpha (\mathit{\lambda })={\displaystyle \frac{3.91}{V}}{{\displaystyle \frac{\mathit{\lambda }}{550\mathrm{nm}}}}^{-q}$
$\beta$	[9]	Generic	$\beta ={\displaystyle \frac{0.046}{V}}$

. However, they also stated that the mechanisms of fog generation are very dynamic and affected by many other factors, so in reality, the actual particle size distributions may vary considerably. A more generic model was proposed by Kim et al. [20], based on improving the earlier model from Kruse et al. [21]; we refer to this as the Kim–Kruse model for a visibility range below 500 m, which is especially relevant for automotive lidar applications.

Comparative analysis of the three models, summarized in Figure 4, shows that they are very similar. However, the model for radiation yields higher extinction coefficients for a set visibility, and can therefore be considered as a worst-case scenario.

In relation to the backscattering coefficient $\beta$, only one source could be identified in the literature, defined by [9], as shown in Table 1.

2.3.2. Empirical Models for Rain

The empirical models for the extinction coefficient of rain are usually based on the precipitation rate R [mm/h]. For example, in Germany, precipitation rates over 10 mm/h occur only around 30 times a year, and usually last only a couple of minutes, down to even several seconds for very high precipitation rates of 50 mm/h to 100 mm/h.

Table 2. Comparison of different coefficient models for rain.

Coefficient	Author	Rain Type	Equation
$\alpha$	[22]	Continental	$\alpha =1.076R^{0.67}[\mathrm{db}/\mathrm{km}]$
	[23]	Thunderstorm	$\alpha =0.16R^{0.74}$
	[24]	Tropical	$\alpha =0.365R^{0.63}[\mathrm{db}/\mathrm{km}]$
$\beta$ a	n/a	n/a	n/a

^a Values marked with “n/a” could not be found in the literature.

shows three different empirical models of common rain types, and Figure 5 depicts the calculated extinction coefficients in relation to the precipitation rate [22,23,24]. However, no empirical model for the backscattering coefficient ${\beta }_{\mathrm{rain}}$ of rain could be found in the literature.

2.3.3. Empirical Models for Snow

Depending on the environment, two types of snow can be distinguished. If the snowflake is constantly frozen during its fall to the ground, the snow is usually not sticky and is called dry snow. But if the flake is falling through warmer layers of air, it will partially melt and will stick to other snowflakes, hence the name wet snow [25]. The unit of snow is the same as rain, which is measured in mm/h. However, as the density of snow is much lower than that of rain, the values can not be directly compared. Figure 6 displays the empirical models based on the values in

Table 3. Comparison of different coefficient models for snow.

Coefficient	Author	Snow Type	Equation
$\alpha$	[26]	Dry	$(5.42·{10}^{-5}{\mathit{\lambda }}_{\mathrm{nm}}+5.5)R^{1.38}[\mathrm{db}/\mathrm{km}]$
	[26]	Wet	$(1.02·{10}^{-4}{\mathit{\lambda }}_{\mathrm{nm}}+3.79)R^{0.72}[\mathrm{db}/\mathrm{km}]$
	[27]	Dry	$\alpha =17.30R[\mathrm{db}/\mathrm{km}]$
	[27]	Wet	$\alpha =1.39R[\mathrm{db}/\mathrm{km}]$
$\beta$ a	n/a	n/a	n/a

^a Values marked with “n/a” could not be found in the literature.

, which shows empirical extinction coefficient ${\alpha }_{\mathrm{snow}}$ models for both types of snow [26,27].

Here, again, no empirical model for the backscattering coefficient ${\beta }_{\mathrm{snow}}$ of snow could be found in the literature.

2.3.4. Empirical Models for Dust

The literature search yielded no automotive-specific lidar study regarding the impact of dust on the laser beam intensity. However, dust storms can have a substantial influence on optical communication systems that are based on laser systems. For this technical application, numerous studies during sand storms were performed [28,29]. The common measure for dust is the total suspended particle mass [µg/m³] (TSP). The work in [30] presented several models, which put the TSP in relation to the visibility during a sand storm. In Figure 7, the models are represented with data from [31,32,33], showing a good correlation between unrelated reference sand storm models. Depending on the source of the sand and the mineral composition, the particle distributions can vary a lot, which was also discussed by [34], who showed that the sand consists mostly of Quartz and that the particle distribution can have several local maxima. This is in contrast to the other weather phenomena, where the particle distribution follows a log-normal distribution function with only a single maximum.

2.3.5. Particulate Matter PM2.5

In highly populated regions, particulate matter originating from burning fossil fuels can concentrate in the lower atmosphere. Although a recent study [35] showed that these airborne carbonaceous remains are a problem worldwide, accumulating to a total suspended particle mass of 150 µg/m³ during very polluted weather situations, no relevant work regarding their impact on automotive lidars could be found. The name particulate matter PM2.5 is rooted in the fact that the vast majority of particles have a diameter below 2.5 µm. However, depending on the source of the particles, the particle size distribution can vary, and various studies over several days reported on measurements of PM2.5 particle diameters. In the study of [36] carried out in Hungary the mean particle size was found to be around 0.65 µm, whereas the study of [37] carried out in the Beijing region measured the mean particle size around 0.1 µm. A similar result can be found in the norm [38], which defines soot. Therefore, it can be assumed that particulate matter with a higher mean particle size originates from nature and can therefore be categorized as dust. Thus, the small average particle sizes can be described as man-made and are considered in the following sections as particulate matter PM2.5.

3. Methodology

3.1. Research Questions

The review of the relevant literature showed that the existing mathematical models are insufficient to enable a realistic simulation study of automotive lidar performance under real-world environmental conditions.

Therefore, two research questions were formulated to guide this study:

Research Question 1: How can a comprehensive set of empirical models for the extinction and backscattering coefficient be developed to cover all environmental weather conditions relevant to automotive lidar applications?

The literature review revealed that for some weather phenomena, no empirical law is available, whereas in other cases, multiple laws are provided, with no clear prescriptive guidance on usage in an engineering context. In a first step, the empirical models available are to be evaluated and their use in the simulation discussed. This research proposes to address missing models by using the Mie theory, as well as the plausibility of the proposed models to be analyzed.

Research Question 2: Do the proposed models, utilized in conjunction with virtual driving simulation environments, produce valid results?

The use of the proposed empirical models is to be verified by including the effect of the weather conditions on the lidar perception in the simulation via the empirical mathematical model. The validation exercise revolves around the simulation of a specific lidar system, i.e., the Valeo Scala 2 system. The results from the simulation will be compared with results on actual vehicles, reported in the literature. For weather situations for which there is no comparable real experiment, a prediction of the expected behavior is evaluated under the corresponding weather effect.

3.2. Setup of the Simulation

For this work, the process illustrated in Figure 8 was used to simulate detection experiments under adverse weather situations. The foundation of this process is the virtual environment Carla [39], which simulates road traffic scenes. In this environment, any of the common ADAS sensors and their outputs can be simulated by adding them to a primary vehicle, hence the technical expression ego vehicle. Additionally, other road users such as cars, bicycles, and pedestrians and the environmental conditions can be controlled and dynamically simulated. The ego vehicle can either move within the simulation automatically or can be static. For this work, a Valeo lidar system was simulated, based on publicly available information. The system scans the environment via raycasting, which yields a semantic point cloud, where additional information is also generated for each point. Besides the XYZ coordinate, also the incidence angle on the surface, type of object, and, if available, the object ID is annotated. This semantic point cloud gets periodically published as a ROS2 topic [40]. Within ROS2, the reflected intensity of each point will be calculated in real time based on the weather-related extinction coefficient $\alpha$, the incidence angle, and the reflection characteristic of the object. In the next step, noise is added based on the backscattering coefficient $\beta$. To generate a realistic point cloud, the actual extinction and backscattering coefficient of the current weather must be known. The calculation of these coefficients by empirical models will be discussed in the following section. This newly constructed point cloud also contains the semantic information and is published as a ROS2 topic. Finally, the sensor efficiency will be evaluated. Every point in the cloud with a calculated intensity below a sensor-specific threshold will be discarded from the point cloud. This filtered point cloud can then be used to perform detection experiments. For the detection experiment, a target car will be placed in front of the ego car at a defined distance. In Carla, different weather scenarios will be generated, and the points that are related to the target car will be counted in the semantic-filtered point cloud. By increasing the extinction coefficient, the amount of points should start to decrease significantly above a certain extinction coefficient threshold.

4. Proposed Empirical Models for Weather Factors

In this section, we introduce empirical models for the extinction and backscattering coefficients for weather factors including fog, rain, snow, dust, and particulate matter. Both coefficients are highly dependent on the wavelength of the laser beam. For the purpose of this work, a wavelength of 905 nm was selected to calculate the models, which is commonly used in automotive lidars.

4.1. Fog Model

Utilizing Equations (7) and (8) and particle density functions from [18] for given visibilities to calculate the extinction and backscattering coefficient showed that the empirical models for the extinction coefficient yield reasonable values and even match up with the value to the fourth digit. However, this is not the case for the backscattering coefficient. The direct calculation result in values with the same order of magnitude as the extinction coefficient, whereas the only empirical model for the backscattering coefficient suggests a value 2 orders of magnitude smaller [9]. As a further approach, one can also exploit the observation that Equations (7) and (8) only differ by the efficiency terms $Q_{\mathrm{ext}}$ and $Q_{\mathrm{back}}$ and share the same particle density. Thus, the ratio of $\alpha /\beta$ can be assumed to be invariant and can be expressed by the ratio of integrals. Using the particle density function for radiation fog from [18] returns the result below:

$$\begin{array}{c}\hfill {\displaystyle \frac{\alpha }{\beta }}={\displaystyle \frac{{\displaystyle \frac{\pi }{8}}{\int }_0^{\infty }{D}^2{Q}_{\mathrm{ext}}\left(D\right)N\left(D\right)dD}{{\displaystyle \frac{\pi }{8}}{\int }_0^{\infty }{D}^2{Q}_{\mathrm{back}}\left(D\right)N\left(D\right)dD}}\approx 1.44\end{array}$$

(9)

Therefore, an empirical dependency or the backscattering coefficient dependency is proposed as follows:

$$\begin{array}{c}\hfill {\beta }_{\mathrm{fog}}\approx {\displaystyle \frac{{\alpha }_{\mathrm{fog}}}{1.44}}\end{array}$$

(10)

Important to note that this result is only valid for this kind of particle density function; if a different particle density function is used, the ratio of ${\displaystyle \frac{\alpha }{\beta }}$ in Equation (9) will be slightly different, reflecting the fact that the result is highly influenced by the amount of particles with approximately the same diameter size as the photon’s wavelength.

However, a comparative calculation shows that the proposed approach provides a model which is comparable to the computation of the integrals in Equations (7) and (8).

4.2. Rain Model

Comparing the three models in Table 2 yields a result which is not very intuitive at first sight: for the same precipitation rate, the model for continental rain results in the highest extinction coefficient and tropical rain the lowest (see Figure 5). The root cause for this result is the very different particle density distributions, which assume that tropical rain has many more large droplets. As the volume of a sphere rises with the third power of the radius, a larger quantity of water is bound in a few large droplets instead of many small ones, as is the case for continental rain. Thus, if a light wave is traveling through tropical rain, it is subject to fewer scattering events, compared with traveling through continental rain, which results in a lower extinction coefficient [41]. As continental rain delivers the largest coefficient, it can be considered as the worst-case scenario for lidar experiments.

Since no empirical model for the backscattering coefficient could be found, the same approach as for the backscattering coefficient of fog was used, i.e., to assume that the ratio $\alpha /\beta$ is an invariant. Here, the particle density function of continental rain from [9] was used to calculate the ratio:

$$\begin{array}{c}\hfill {\beta }_{\mathrm{rain}}\approx {\displaystyle \frac{{\alpha }_{\mathrm{rain}}}{0.60}}\end{array}$$

(11)

It is noticeable in the equation above that the backscattering coefficient of rain is larger than the associated extinction coefficient. Figure 9 shows the cumulative efficiencies $Q_{\mathrm{ext}}$ and $Q_{\mathrm{back}}$ for the particle density function used. This shows clearly that the backscattering efficiency dominates the scattering behavior.

4.3. Snow Model

The highest extinction coefficient ${\alpha }_{\mathrm{snow}}$ is observed for using dry snow, as shown in Figure 6. The explanation for this is similar to the reasoning provided for rain. For dry snow, density can be assumed to be four times smaller than for wet snow, as discussed by [27]. To attain a similar precipitation rate, the amount of dry snowflakes must be 4 times higher than for wet snow; hence, dry snow yields more scattering events than wet snow. To classify the amount of snowfall in relation to visibility, one can refer to [42], where a visibility of 100 m is considered to occur during a dry snow precipitation rate of 6 mm/h, which is considered heavy snowfall according to [43]. To calculate the missing backscattering coefficient ${\beta }_{\mathrm{snow}}$, the same approach as for fog was used, i.e., consider $\alpha /\beta$ as invariant. The refractive index was retrieved from [44], and the particle size distribution from [45]. Following this approach, the following empirical equation is proposed:

$$\begin{array}{c}\hfill {\beta }_{\mathrm{snow}}={\displaystyle \frac{{\alpha }_{\mathrm{snow}}}{1.26}}\end{array}$$

(12)

4.4. Dust Model

Due to the fact that no suitable model for the coefficients could be found in the literature, a simplified model is proposed based on the literature available in different research areas.

Using a visibility-related model for the number of particles, ref. [46] have calculated the coefficients by assuming a fixed effective particle diameter. Here, it is not taken into account that particularly small particles with a diameter in the order of the laser wavelength have a large influence on both coefficients, which is also depicted in Figure 3. The particle size distribution of sand storms can be very different [34], as the it is highly influenced by the soil of the storm’s origin. The ISO 12103-1 [47] standard describes the parameters of different types of dust. Here, the dust type ”A4 - coarse test dust” can be considered similar to the dust referred to in the literature [34]. Based on this definition of dust, the weight m of a fixed number of particles N can be calculated using the dust’s bulk density $\rho$ of 1.2 g/cm³ [47] and the particle density distribution $PDF$:

$$\begin{array}{c}\hfill m={\int }_0^{\infty }{\rho }_{bulk}·{\displaystyle \frac{4}{3}}\pi r^3·N·PDF\left(r\right)dr\end{array}$$

(13)

This result can then be used to estimate the number of particles in relation to the visibility V and the total suspended particle mass M. To calculate the mass, the visibility model of [33] was used, as this model showed a reasonable empirical determination of $R^2=0.79$ [30]:

$$\begin{array}{c}\hfill N\left(V\right)={\displaystyle \frac{M}{m}}={\displaystyle \frac{1}{m}}·{\left({\displaystyle \frac{3553.405}{V}}\right)}^{{\displaystyle \frac{127}{125}}}\end{array}$$

(14)

Plugging Equation (14) into Equation (7) makes it possible to calculate the extinction coefficient $\alpha$ depending on visibility V. The backscatter coefficient can be obtained in a similar way, by replacing Equation (14) in Equation (7):

$$\begin{array}{c}\hfill \alpha \left(V\right)={\displaystyle \frac{\pi }{8}}·{\int }_0^{\infty }{D}^2·Q_{ext}\left(D\right)·N\left(V\right)·PDF\left(D\right)dD\end{array}$$

(15)

$$\begin{array}{c}\hfill \beta \left(V\right)={\displaystyle \frac{\pi }{8}}·{\int }_0^{\infty }{D}^2·Q_{back}\left(D\right)·N\left(V\right)·PDF\left(D\right)dD\end{array}$$

(16)

Figure 10 shows the coefficients in relation to the visibility during dust storms by calculating 1000 data points with different visibility for Equations (15) and (16). To enable an easy calculation of both coefficients, a curve fit of an inverse power law was performed on the data point sets, which shows a coefficient of determination of $R^2$ = 1 for both fits:

$$\begin{array}{c}\hfill \alpha =5.26V^{1.016}\end{array}$$

(17)

$$\begin{array}{c}\hfill \beta =5.38V^{1.016}\end{array}$$

(18)

This model yields a reasonable value for the extinction coefficient, as the overall scattering anisotropy of SiO₂ particles is smaller than for water (see Figure 3), which results in a higher extinction coefficient. Below is a comparison of the extinction coefficient $\alpha$ of dust storms and fog at an exemplary visibility of 100 m. As expected, the extinction coefficient of dust storms is higher than water:

$$\begin{array}{c}\hfill {\alpha }_{Dust}{|}_{V=100m}=0.0488\end{array}$$

(19)

$$\begin{array}{c}\hfill {\alpha }_{Fog}{|}_{V=100m}=0.0391\end{array}$$

(20)

4.5. Particulate Matter

To calculate the coefficients for particulate matter, a similar approach as that for dust was chosen. However, it is very uncommon to describe the impact of particulate matter with visibility; therefore, the calculations are based on the more common parameter, the total suspended particle mass $TSP$. The particle density function was retrieved from [38], and the refractive index n from [48]. The calculation below was analogous to the backscattering coefficient ${\beta }_{\mathrm{PM}2.5}$:

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{PM}2.5}\left(TSP\right)={\displaystyle \frac{\pi }{8}}{\int }_0^{\infty }{D}^2{Q}_{ext}\left(D\right)N\left(TSP\right)PDF\left(D\right)dD\end{array}$$

(21)

$$\begin{array}{c}\hfill {\beta }_{\mathrm{PM}2.5}\left(TSP\right)={\displaystyle \frac{\pi }{8}}{\int }_0^{\infty }{D}^2{Q}_{back}\left(D\right)N\left(TSP\right)PDF\left(D\right)dD\end{array}$$

(22)

Figure 11 shows both coefficients in relation to the total suspended particle mass. As the refractive index n of soot has a large imaginary part, and hence a very large absorption coefficient, the backscattering coefficient ${\beta }_{\mathrm{PM}2.5}$ is one order of magnitude lower than the extinction coefficient ${\alpha }_{\mathrm{PM}2.5}$. By applying a curve fit with an inverse power law to 1000 calculated data points of Equations (21) and (22) in Figure 11, the following empirical models could be retrieved, and both show a coefficient of determination of $R^2$ = 1,

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{PM}2.5}\left(TSP\right)=9.50{10}^{-4}TSP\end{array}$$

(23)

$$\begin{array}{c}\hfill {\beta }_{\mathrm{PM}2.5}\left(TSP\right)=3.89{10}^{-5}TSP\end{array}$$

(24)

5. Results

The following sections explore the impact of adverse weather conditions on lidar performance through simulations in a virtual environment. Details of the simulation setup and validation against real-world experiments are presented, with analysis of results across various weather phenomena, including fog, rain, snow, dust, and particulate matter (PM2.5).

5.1. Simulation in Virtual Environment

For this work, the virtual environment CARLA [39] and the Robot Operating System 2 (ROS2) [40] were used to implement the proposed empiric models into a real-time simulation. Carla provides the simulated environment of a car in an urban traffic scenario and the associated sensor data, such as a semantic point cloud, where all points are annotated with the objects they represent. The point cloud is then handed over to ROS2, which acts as a middleware framework, providing a standardized API to enable communication between sensors and software. Within ROS2, all real-time calculations and evaluations are performed.

Within Carla, the lidar was configured to resemble the Scala 2 system. The detection properties of the lidar were determined on the basis of publicly available data sheets [49,50]. The semantic point cloud, containing both the spatial information and the beam incidence angle, was transferred from Cartesian coordinates into spherical coordinates. Within the spherical coordinates, the distance of each object to the lidar is represented by the parameter r, which makes it easy to calculate the atmospheric loss for each laser beam.

The weather effects on the reflected beam power were estimated using the empirical models for the extinction coefficient, the incident angle, and the object’s reflection characteristics. Then, the backscattering noise was added by inserting additional points with the calculated intensity and annotating them as noise. The lidar built-in algorithm usually orders the echoes of the same beam direction by intensity to highlight echoes of interest. Finally, all values below a certain threshold will be filtered out, considering internal sensor losses, and this altered point cloud will be published in the ROS2 network. At this stage, the detection algorithm was not considered/implemented, so the detection performance for different adverse weather situations was calculated directly out of the two semantic point clouds with and without weather effects. These steps for the external and internal factors are presented in the flow chart in Figure 8.

5.2. Validation by Comparison with Experiments

In order to evaluate the quality of the simulation, it was compared against performance data of real lidars under different weather conditions. Several publications ([11,12,51,52]) reported on physical experiments in a large environmental testing facility, called Cerema [53], which is capable of generating different rain and fog scenarios for Advanced driver-assistance systems (ADAS) testing. Out of these publications, [11,12] were chosen for comparison, as the first publication compares the performance of five different lidar systems (lidars benchmarked: Livox Horizon, Velodyne VLP-32, Ouster 128, Cepton 860, and AEye 4SightM) under adverse weather situations. The second publication also benchmarks the Valeo lidar considered in this work.

To recreate the real experiment in [12], which used a white target with a surface of 1 m² and reflectivity of 80% at a distance of 23 m, a similar scenario was set up in the simulation. Here, a car with 80% reflectivity was placed at a distance of 23 m in front of the simulated lidar (see Figure 12). As the detected number of points, which refer to the object, is normalized, a similar result is expected. The experiment of [11] was performed with a white car at a distance of around 18 m, so the results can be compared, but it is expected to perform slightly differently. However, as the reflectivity of the white car is unknown, the experiment could not be recreated. To obtain a real-world reference with lidars of similar performance, the comparators for the Scala 2 simulation experiments were the Livox and Aeye data [12] and the Scala 2 experimental data [11].

5.2.1. Results for Experiments Involving Fog

Figure 13 shows the comparison of the real experiment and the simulation. This shows that the simulation behaves very similar to the results from the physical experiments reported by [12]. The results for the Livox and Aeye lidar types have almost identical performance in the simulation, and therefore, the traces are indistinguishable. The results for the Valeo lidar also show good correlation between simulation and physical experiments, with the exception of the experiment with low visibility of (25 ± 5) m. Here, the experiment already detects some points, which make no practical difference, as the amount of returned points is very low.

Across the experiments, the results show that if visibility is higher than 40 m, then the number of detected points does not increase, and the graph is more or less linear. This finding is also very similar to the analysis of [11]. Overall, the simulation of foggy weather with the empirical models proposed in this paper shows results that are consistent and comparable to the real experiments, providing good validation.

5.2.2. Results for Experiments Involving Rain

As Figure 14 shows, the results of the simulation are comparable with the reference lidar results. For Livox and Aeye, the number of detected points is almost constant at 100% throughout the experiment, with the exception of an unexplainable outlier at 20 mm/h in the Livox experiment. The work in [11] just tested a precipitation rate of 55 mm/h for an unknown reason, and in this experiment, the mean detection rate of the points is around 100%. For precipitation rates above 100 mm/h, the results for the Aeye and Livox experiments are very different; the detection of the Livox lidar collapses to around 40%, as well as the outlier, whereas the detection of the Aeye is still around 100%. For these differences, the following root causes can be hypothesised:

• The particle distribution for the rain in the Cerema test facility is not known. As shown in Figure 5, the extinction coefficient is very dependent on the actual particle distribution. The rain generator might not be constant enough to produce the exact needed particle distribution throughout its operating range and test volume.
• Raindrops could possibly flow down the lidar window, thus deflecting the path of the laser beam, causing the rate of detected points to drop. This could explain the Livox outlier at 20 mm/h and above 100 mm/h.

5.3. Analysis of Simulation Results

5.3.1. Results from Simulation Experiments Involving Snow

The setup of this and the following simulations is exactly the same as for rain and fog. Figure 15 depicts the detection rate under snowfall, showing that the detected number of points decays to 0 around precipitation of 28 mm/h. This is unlike the results seen in Figure 14 for rain, which show a more stable signal returned. Given that water expands when it freezes, the density decreases, and the effective cross-section becomes larger; therefore, stronger scattering events occur.

5.3.2. Results from Simulation Experiments Involving Dust

Figure 16 illustrates the results from the simulation experiment involving dust. To have a more intuitive impression of this weather effect, visibility was used as a measure. These results are similar to fog, shown in Figure 13, which is reasonable, although the number of points decreases already at higher visibilities during dust storms compared with fog. As dust storms contain mainly SiO₂, the scattering anisotropy of dust particles is lower than that of water droplets (see Figure 3), and therefore the atmospheric loss is higher, leading to the observed behavior.

5.3.3. Particulate Matter PM2.5

Figure 17 shows the number of points in relation to the total suspended particle mass. The number of points returned drops to 0 around 120 µg/m³, which implies that the performance of the lidar fades completely during heavy air pollution.

6. Discussion

The lidar technology is a key technology for autonomous driving. Although this type of sensor compensates for the weaknesses of previous technologies, it is still subject to weather phenomena. As the lidar relies on the detection of self-emitted laser pulses, airborne particles, such as water droplets in fog and rain, can scatter or absorb the photons of the laser beam. This effect has the following two consequences: firstly, a loss of detectable signal intensity, which can be calculated using the extinction coefficient $\alpha$ and the distance to the object. Secondly, backscattered photons add noise, depending on the backscattering coefficient $\beta$, to the point cloud, which is constructed inside the lidar. Both coefficients can be calculated using either Mie scattering theory or empirical models. A main contribution of this work is to propose a full set of empirical models for the coefficients, adding to the limited set previously available in the literature, as shown in

Table 4. Empirical models used for this work.

	Fog	Rain	Snow	Dust	PM2.5
$\alpha$	[19]	[22]	[27]	$5.26V^{1.016}$	$9.50{10}^{-4}TSP$
$\beta$	${\displaystyle \frac{{\alpha }_{\mathrm{fog}}}{1.44}}$	${\displaystyle \frac{{\alpha }_{\mathrm{rain}}}{0.60}}$	${\displaystyle \frac{{\alpha }_{\mathrm{snow}}}{1.26}}$	$5.38V^{1.016}$	$3.89{10}^{-5}TSP$

.

The comparison of the theoretical and empirical approaches shows that the models for the extinction coefficient for fog are very similar, as depicted in Figure 4; however, the model for the backscattering coefficient appears to be too small. Using the ratio of the calculation of $\alpha /\beta$, a constant factor was calculated to approximate the backscattering coefficient.

The empirical models of rain for the extinction coefficient yield very different results depending on the mean drop size. Here, many small raindrops cause more scattering events, resulting in a large extinction coefficient, whereas the same precipitation rate with a few large drops results in a smaller coefficient. For the backscattering coefficient ${\beta }_{\mathrm{rain}}$, no model could be found and was therefore calculated in the same way as for fog.

Depending on the temperature of the atmosphere, either dry or wet snow occurs. Here, dry snowfall results in a higher extinction coefficient ${\alpha }_{\mathrm{snow}}$ due to its lower density, and hence the higher amount of flakes to achieve the same precipitation. For the backscattering coefficient ${\beta }_{\mathrm{snow}}$, no model could be found in the literature, so it was calculated similarly to fog.

For dust storms no applicable empirical models could be found in the literature. Using related work of different areas of a simple approach was developed, which considers industry standard type of dust, to calculate both extinction coefficients.

Although particulate matter is in some regions a common man-made environmental phenomenon, no related literature could be found. A similar approach to that for dust was used to derive empirical models for the coefficients.

In this work, we have implemented a rigorous simulation approach to validate the usefulness of the proposed models for lidar detection. We have presented a comprehensive simulation framework using the virtual environment Carla and ROS2. To evaluate the performance of the simulation, it was compared with real experiments by recreating the real scenarios in the simulation, which showed that the simulation of fog is very similar to the real experiments. Although the results of the simulation of rain do not contradict the experimental results available, it appears to be a bit more complicated, as several unknown factors might affect the real experiment, which were not known and hence not considered in the simulation experiment.

For dust, snow, and particulate matter, no real-world experiment could be found, and therefore, the simulation results can be considered as a prediction of the lidar detection capabilities.

As the derivation of the models has shown, the calculated coefficients depend heavily on the particle distributions of the respective weather phenomena. As these can vary considerably depending on the general weather situation and region, these models do not claim to be universally valid but rather represent a simplified approach to representing such weather phenomena within a simulation.

Future work will involve the implementation of mixed weather effects and eventually the evaluation of various object detection algorithms under tailored adverse weather situations.

7. Conclusions

This paper has introduced a comprehensive methodology for the evaluation of automotive lidar systems through simulation, underpinned by a complete set of empirical models for the effect of weather and atmospheric conditions on the performance of lidar sensors. This closes a gap in both the research literature and practice, in particular in relation to dust and particulate matter, for which no empirical model is available.

In this work, it was shown that utilizing these empirical models in simulations of rainy or foggy weather yields a similar detection performance as comparable real-world experiments, hence validating the proposed models. Also, the detection performance during snowfall, dust storms, and polluted air was predicted, following the same approach as for rain and fog.

Based on these models, embedded in a virtual simulation environment as described in this paper, the development and validation of lidar systems for autonomous driving can be significantly enhanced and accelerated by creating test experiments that simulate corner cases of real-world weather phenomena.