Driving Environment Perception Based on the Fusion of Vehicular Wireless Communications and Automotive Remote Sensors

Driving environment perception for automated vehicles is typically achieved by the use of automotive remote sensors such as radars and cameras. A vehicular wireless communication system can be viewed as a new type of remote sensor that plays a central role in connected and automated vehicles (CAVs), which are capable of sharing information with each other and also with the surrounding infrastructure. In this paper, we present the design and implementation of driving environment perception based on the fusion of vehicular wireless communications and automotive remote sensors. A track-to-track fusion of high-level sensor data and vehicular wireless communication data was performed to accurately and reliably locate the remote target in the vehicle surroundings and predict the future trajectory. The proposed approach was implemented and evaluated in vehicle tests conducted at a proving ground. The experimental results demonstrate that using vehicular wireless communications in conjunction with the on-board sensors enables improved perception of the surrounding vehicle located at varying longitudinal and lateral distances. The results also indicate that vehicle future trajectory and potential crash involvement can be reliably predicted with the proposed system in different cut-in driving scenarios.


Introduction
Automated driving has generated increasing interest in recent decades due to its potential to address some of the most challenging issues faced by people and communities across the world. Road traffic crashes account for 1.35 million deaths a year worldwide, and they are the leading cause of death among children and young adults aged 5 to 29 years [1]. In addition, many people in different parts of the world still lack access to mobility, particularly in rural areas, and the number of older people who are no longer physically capable of driving has been increasing globally at the fastest rate recorded [2]. Another important issue in large metropolitan areas throughout the world is excessive traffic congestion, which is directly related with the growing number of vehicles on the road [2,3]. These challenges can be addressed by successful implementation of automated driving on public roads. With automated vehicles it is possible to achieve greater road safety, universal access to mobility, and higher transportation efficiency [4]. SAE International proposed different levels for driving automation, which include six levels ranging from no driving automation (level 0) to full driving automation (level 5) [5]. In level 0-2 systems, the driver is expected to respond to any evident vehicle system failure. A level 0 system does not provide any driving automation functions, while a level 1 system supports the driving task by performing either the longitudinal or the lateral driving environments, and the results demonstrated that reliable environment perception and collision prediction can be achieved by introducing V2X communications, even in scenarios where it is difficult to avoid a collision with existing safety systems based only on on-board sensors.
In this paper, we present the design and implementation of cooperative environment perception based on the fusion of V2X communications and automotive remote sensors. In continuation of our previous work [23], the Kalman-filter-based approach is employed for the high-level fusion of radar, camera, and V2X communication data, and the proposed cooperative approach is tested and evaluated in test-track environments. The experiments are carried out with two test vehicles, where each vehicle is equipped with a 5.9 GHz dedicated short-range communications (DSRC) transceiver along with a global navigation satellite system (GNSS) receiver for the exchange of vehicle state information. The host vehicle is additionally equipped with automotive radar and camera systems for remote sensing of the surrounding objects. The performance of the proposed approach for driving environment perception is evaluated at varying relative distances between the two test vehicles. The positioning accuracies at different longitudinal and lateral target vehicle positions are computed for each remote sensing system as well as for the proposed fusion method. The performance of the trajectory prediction and risk assessment is evaluated in different cut-in driving scenarios where the remote vehicle performs a lane-change maneuver in front of the host vehicle.
The rest of the paper is organized as follows. In Section 2, the overall architecture of the proposed driving environment perception system is described and background information about automotive remote sensors and V2X communications is provided. In Section 3, the proposed method for object state estimation and risk assessment is presented. The experimental results are reported and discussed in Section 4, and conclusions and directions for future research are presented in Section 5.

Overall Design of the Proposed System
The overall design of the proposed cooperative environment perception approach based on on-board sensors and vehicular wireless communications is illustrated in Figure 1. The system is designed to serve as a flexible platform that enables reliable vehicle driving environment recognition and to provide appropriate road safety applications. Automotive remote sensors (e.g., radars and cameras) are often connected to the controller area network (CAN) bus such that the data from these sensors along with other data generated by in-vehicle electronic control units (ECUs) are collected in the form of CAN messages. The DSRC messages received with an on-board unit (OBU) are collected via Ethernet interfaces. For the enhancement of GNSS positioning performance, RTCM corrections, which contain differential corrections for the GNSS as defined by the Radio Technical Commission for Maritime Services (RTCM) Special Committee 104, can be obtained either through Networked Transport of RTCM via Internet Protocol (NTRIP) when an Internet connection is available, or through DSRC RTCM messages as defined in SAE J2735 [29] when it is possible to utilize roadside units (RSUs) capable of RTCM applications.
The overall design of the proposed cooperative environment perception approac based on on-board sensors and vehicular wireless communications is illustrated in Figur 1. The system is designed to serve as a flexible platform that enables reliable vehic driving environment recognition and to provide appropriate road safety application Automotive remote sensors (e.g., radars and cameras) are often connected to the controlle area network (CAN) bus such that the data from these sensors along with other dat generated by in-vehicle electronic control units (ECUs) are collected in the form of CAN messages. The DSRC messages received with an on-board unit (OBU) are collected v Ethernet interfaces. For the enhancement of GNSS positioning performance, RTCM corrections, which contain differential corrections for the GNSS as defined by the Radi Technical Commission for Maritime Services (RTCM) Special Committee 104, can b obtained either through Networked Transport of RTCM via Internet Protocol (NTRIP when an Internet connection is available, or through DSRC RTCM messages as defined i SAE J2735 [29] when it is possible to utilize roadside units (RSUs) capable of RTCM applications. For cooperative relative positioning, and particularly for the fusion of the on-boar sensors and V2X communications, it is necessary to transform the GNSS positionin information (i.e., latitude and longitude) of the host vehicle and the remote targets from the World Geodetic System 1984 (WGS 84) reference frame to the host vehicle referenc frame. For this, WGS-84 coordinates are converted to earth-centered, earth-fixed (ECEF coordinates and then to east-north-up (ENU) coordinates. The GNSS course heading o the host vehicle is utilized to finally rotate the remote target positions and find the relative positions in the host vehicle reference frame. The relative positions and othe dynamic information on remote targets acquired from V2X communications can be use in conjunction with the measurements from on-board sensors for more accurate an reliable target state estimation, classification, and trajectory prediction. Some of the safet applications that can be offered with this cooperative approach are shown in the safet applications block in Figure 1.

Automotive Remote Sensors
The types of sensors used for driving environment perception and object tracking i recent years include radars, cameras, and lidars. These sensors perform detection o For cooperative relative positioning, and particularly for the fusion of the on-board sensors and V2X communications, it is necessary to transform the GNSS positioning information (i.e., latitude and longitude) of the host vehicle and the remote targets from the World Geodetic System 1984 (WGS 84) reference frame to the host vehicle reference frame. For this, WGS-84 coordinates are converted to earth-centered, earth-fixed (ECEF) coordinates and then to east-north-up (ENU) coordinates. The GNSS course heading of the host vehicle is utilized to finally rotate the remote target positions and find their relative positions in the host vehicle reference frame. The relative positions and other dynamic information on remote targets acquired from V2X communications can be used in conjunction with the measurements from on-board sensors for more accurate and reliable target state estimation, classification, and trajectory prediction. Some of the safety applications that can be offered with this cooperative approach are shown in the safety applications block in Figure 1.

Automotive Remote Sensors
The types of sensors used for driving environment perception and object tracking in recent years include radars, cameras, and lidars. These sensors perform detection of objects in the vehicle surroundings and provide information on the object state (e.g., relative position and relative speed) and object classification. Some of the most common advanced driver assistance system (ADAS) applications enabled by the environment perception technology include forward collision warning (FCW), automatic emergency braking (AEB), adaptive cruise control (ACC), lane departure warning (LDW), and lane keeping assist system (LKAS). For this study, we equipped our test vehicle with radar and camera sensors that have already been incorporated into production vehicles.
Automotive radar, which is an active ranging sensor designed for detecting and tracking remote targets in the surrounding environment, is one of the most used ranging sensors for ADAS functions these days. Automotive radars are capable of providing the relative position and speed information about the objects located within the sensor FOV, and they operate even in adverse weather conditions such as rain, fog, and snow. Two frequency bands, 24 GHz and 77 GHz, have been predominantly exploited for automotive radars [30][31][32]. The 24-GHz band is mainly used for short-range radar systems with a detection range up to 30 m [30,33], while the 77-GHz band is mainly used for long-range radar systems with a detection range up to 250 m [31]. A frequency modulated continuous wave (FMCW) radar is the most common type of automotive radars used for remote sensing of surroundings [31]. FMCW radars transmit a frequency-modulated signal in a continuous manner. The frequency of the signal changes linearly with time, and this enables accurate time measurement based on the frequency difference between the transmitted and received signals, which can be converted into range information. The specification of Delphi ESR 2.5, the multimode radar system utilized for the driving experiments in this study, is shown in Table 1. Computer vision techniques are increasingly utilized for automotive safety applications. Camera systems with computer vision methods perform detection and localization of the objects that have been captured in the camera images. It is also possible to extract valuable information on the driving environment, such as lane marking positions and road curvature, from the images obtained with the cameras on board the vehicle. A comprehensive review on computer vision techniques for vehicle detection, tracking, and behavior characterization is presented in [34]. The most notable camera system that has been widely incorporated into production vehicles is that of Mobileye [35]. The longitudinal distance to the surrounding vehicle can be determined based on the image position of the target and the detected target width or height [36][37][38]. Based on [36], the accuracy of the range estimated with the camera system decreases quadratically as the distance to the object increases, while the error percentage increases linearly, such that a 5% error in range is expected at 45 m and a 10% error in range is expected at 90 m. The characteristics of the Mobileye 630 camera system that was installed on our test vehicle are summarized in Table 2. Lidar stands for light detection and ranging, and as the name suggests, it measures the relative distance and angle to a target within its FOV by transmitting laser light and measuring reflected light. Near-infrared light with a wavelength of 905 nm is typically used for automotive lidars, and they are capable of higher localization performance compared with other automotive ranging sensors. Lidar sensors attracted much attention of the automated driving research community since they were used by many groups who participated in the DARPA Grand Challenges [39][40][41]. Despite the advantage of lidar sensors, most automakers are yet to incorporate them into production vehicles largely due to their drawbacks such as the high manufacturing cost and bulky form factor. Moreover, high-performance lidars generate millions of data points per second [42]. Processing lidar data points is often computationally expensive and the perception and tracking performance based on lidar measurements can vary depending on the algorithm used [43][44][45]. The use of lidar sensors is outside the scope of this paper, and perception approaches based on multiple remote sensors including lidars will be investigated in future work.

V2X Communications
There exists a range of wireless communication technologies that can be employed for CAV systems and intelligent transportation system (ITS) applications (e.g., 5.9 GHz DSRC, cellular communications, Wi-Fi, Bluetooth, satellite radio, and visible light communications) [20,46]. The V2X communications based on 5.9 GHz DSRC is a mature technology that is the most widely tested and commercially available [46][47][48]. The DSRC-based V2X communication technology is based on the IEEE 802.11p and the IEEE 1609 series of standards, which are collectively known as wireless access in vehicular environments (WAVE) standards [46]. The 5.9 GHz frequency band is divided into seven 10-MHz channels, which include one control channel (CCH) and six service channels (SSHs) [49]. As an alternative to the DSRC-based V2X communications, the cellular V2X (C-V2X) communications based on the 3GPP standards has attracted significant attention in recent years. The C-V2X communications utilizes the cellular network infrastructure (e.g., LTE and 5G networks) to enable V2X applications. For the upper layers, the C-V2X technology is expected to leverage existing standards (e.g., IEEE, SAE, ISO, and ETSI) and utilize common ITS message types, such as the basic safety message (BSM) of IEEE WAVE and the cooperative awareness message (CAM) and the decentralized environmental notification message (DENM) of ETSI ITS-G5. Some of the advantages of the C-V2X technology over 5.9 GHz DSRC include a much larger coverage area, higher throughput, lower latency, and more robust scalability [47,48]. Despite the advantages, the standardization work of C-V2X communications is still in process and the availability of C-V2X commercial hardware equipment is limited at this stage. In this work, we utilized 5.9 GHz DSRC for V2X communications, and the test vehicles were equipped with Cohda MK5 OBUs for the exchange of information among vehicles. The characteristics of DSRC-based V2X communications are shown in Table 3. The vehicle state information is broadcast and shared among the vehicles equipped with DSRC devices by exchanging the BSM, which is defined in the SAE J2735 message set dictionary [29]. The BSM contains data obtained from the vehicle CAN bus and the GNSS receiver, which include safety critical state information such as the vehicle position, heading, speed, and yaw rate. The BSM is typically transmitted with a period of 100 ms on the dedicated safety channel (i.e., Channel 172 in the U.S.) [29,46]. The BSM Part I data contain the BSM core data frame that shall be included when broadcasting a BSM, while the BSM Part II data contain optional additional information (e.g., event flags, path history, path prediction, and exterior lights). The contents of the BSM core data frame are described in Table 4. Table 4. Data description for the basic safety message (BSM) core data frame.

Content Description
Message count Sequence number for the same type of messages originated from the same sender.
Temporary ID Device identifier that is modified periodically for on-board units (OBUs). This value may be fixed for roadside units (RSUs).

DSRC second
Milliseconds within a minute that typically represents the moment when the position was determined.

Position
Geographic latitude, longitude, and height.
Position accuracy Semi-major axis (length and orientation) and semi-minor axis (length) of an ellipsoid representing the position accuracy. Brake system status Status of the brake and other control systems (i.e., traction control, ABS, stability control, brake boost, and auxiliary brake). Vehicle size Vehicle width and length.

State Estimation and Prediction
A Kalman-filter-based fusion approach that was previously described in [23] is employed in this study for state estimation and trajectory prediction of the remote targets in the vehicle surroundings. At each time step, the measurements from the on-board remote sensors as well as the BSMs received from the remote targets are collected and processed with Kalman filter algorithms, which reduce the measurement noise and output the state and error covariance for each track. The types of information we used in this study to estimate the state and future trajectory of the remote target include the following: Position, speed, heading, yaw rate, and size information from the V2X communications; range and azimuth information from the radar system; and longitudinal and lateral distance information from the camera system. The state estimates from different tracks are then associated and fused together, where the weight for each track is determined based on the error covariance. The future trajectory of the remote target detected in the perception stage is estimated with the constant turn rate and velocity (CTRV) motion model. For performance evaluation in the context of safety applications, the future trajectory of the remote target is compared with the future trajectory of the host vehicle and an appropriate warning is generated when a possible collision is detected.

Kalman Filtering
Kalman filtering [50,51] is a recursive algorithm that estimates the state of a system as well as the estimation uncertainty based on the prior state and the noisy measurements. The operation of the Kalman filter is described in Figure 2. In the prediction step, the statê x k|k−1 and the error covarianceP k|k−1 are projected with the state transition matrix A from the previous statex k−1|k−1 and the corresponding error covarianceP k−1|k−1 . The random variable w k is the process noise, which is assumed to be normally distributed with the process noise covariance Q k , such that w k ∼ N(0, Q k ). The process noise covariance Q k assumed here to be constant, but it may be changed during the filter operation for adjust-Sensors 2021, 21, 1860 8 of 29 ment to different dynamics. In the update step, the error covarianceP k|k−1 is used along with the measurement matrix H and the measurement error covariance R k to compute the Kalman gain K k . The measurement matrix H maps the state vector x k to the measurement vector z k such that with the process noise covariance , such that ~ 0, . The process nois covariance assumed here to be constant, but it may be changed during the filte operation for adjustment to different dynamics. In the update step, the error covarianc | is used along with the measurement matrix and the measurement erro covariance to compute the Kalman gain . The measurement matrix maps th state vector to the measurement vector such that The random variable is the measurement noise, which is assumed to be normall distributed with the measurement error covariance , such that ~ 0, . Finally the state | and the error covariance | are updated based on the Kalman gain and the measurement vector obtained at time step . The relative positions and the motion equations of the remote targets in the vehic surroundings are typically given in Cartesian coordinates, and it is necessary to perform a polar-to-Cartesian transformation when the ranging measurements obtained from remote sensors (e.g., radar sensors) are in polar coordinates. An extended Kalman filte (EKF) is often utilized to handle such nonlinear systems; however, the linea The random variable v k is the measurement noise, which is assumed to be normally distributed with the measurement error covariance R k , such that v k ∼ N(0, R k ). Finally, the statex k|k and the error covarianceP k|k are updated based on the Kalman gain K k and the measurement vector z k obtained at time step k.
The relative positions and the motion equations of the remote targets in the vehicle surroundings are typically given in Cartesian coordinates, and it is necessary to perform a polar-to-Cartesian transformation when the ranging measurements obtained from remote sensors (e.g., radar sensors) are in polar coordinates. An extended Kalman filter (EKF) is often utilized to handle such nonlinear systems; however, the linear approximation of a nonlinear system may result in highly unstable performance, and the derivation of the Jacobian matrices is often nontrivial in many applications [52]. More advanced nonlinear filtering approaches such as sequential Monte Carlo methods, also known as particle filters, have been introduced [53], but the computation complexity becomes enormous for high dimensional problems and the use of particle filters should be determined based on the degree of the system nonlinearity [54]. In this study, the unbiased converted measurement Kalman filter algorithm as presented in [55,56] is employed to perform the coordinate transformation without bias and to obtain the correct covariance. This converted measurement approach yields nearly optimal estimates and provides higher estimation accuracy than the EKF [57].
The operation of the unbiased converted measurement Kalman filter is described in Figure 3. The filtering process includes additional steps to compute the covariance of the unbiased conversion compared with the linear Kalman filtering described above. The range measurement r m and the azimuth measurement θ m are defined as where r and θ are the true range and azimuth of the remote target, respectively, and v r and v θ are the associated measurement noise with an error standard deviation of σ r and σ θ , respectively. The unbiased converted measurements x m and y m are computed by taking into account the bias compensation factor such that Sensors 2021, 21, x FOR PEER REVIEW 10 of 3

Data Fusion
A track-to-track fusion approach is employed in this study for combining high-leve data from multiple sources. The data processing for each sensor system is performe individually at the sensor level in a high-level fusion system. Each sensor system output one or more tracks based on the sensor measurements, and the state estimates from multiple sensor tracks are associated and combined with a track-to-track fusion algorithm A high-level fusion of multiple sensors has been successfully implemented in man studies dealing with automotive applications [58][59][60][61][62]. Important advantages of the high level fusion approach lie in spatial and temporal alignment, modularity, an communication overhead [63,64]. A simple block diagram for the high-level fusion system architecture [63,65] is shown in Figure 4. The computation of the unbiased converted measurements as well as the associated covariance requires the compensation factors λ θ and λ θ , which are determined from The rest of the steps for obtaining the covariance of the unbiased conversion are as shown in Figure 3, and the statex k|k and the error covarianceP k|k are updated according to the Kalman gain K k and the measurement z k .

Data Fusion
A track-to-track fusion approach is employed in this study for combining high-level data from multiple sources. The data processing for each sensor system is performed individually at the sensor level in a high-level fusion system. Each sensor system outputs one or more tracks based on the sensor measurements, and the state estimates from multiple sensor tracks are associated and combined with a track-to-track fusion algorithm. A high-level fusion of multiple sensors has been successfully implemented in many studies dealing with automotive applications [58][59][60][61][62]. Important advantages of the high-level fusion approach lie in spatial and temporal alignment, modularity, and communication overhead [63,64]. A simple block diagram for the high-level fusion system architecture [63,65] is shown in Figure 4.
one or more tracks based on the sensor measurements, and the state estimates from multiple sensor tracks are associated and combined with a track-to-track fusion algorithm. A high-level fusion of multiple sensors has been successfully implemented in many studies dealing with automotive applications [58][59][60][61][62]. Important advantages of the highlevel fusion approach lie in spatial and temporal alignment, modularity, and communication overhead [63,64]. A simple block diagram for the high-level fusion system architecture [63,65] is shown in Figure 4. One of the most widely used algorithms for track-to-track fusion is the convex combination algorithm [65][66][67][68], and it has been used extensively for its simple implementation. Two state estimates and with the corresponding covariance and , respectively, can be fused to obtain the state estimate by One of the most widely used algorithms for track-to-track fusion is the convex combination algorithm [65][66][67][68], and it has been used extensively for its simple implementation. Two state estimates x i and x j with the corresponding covariance P i and P j , respectively, can be fused to obtain the state estimatex by whereP is the covariance associated with the fused estimate, which is given by

Trajectory Prediction and Risk Assessment
The future trajectory of the remote target in the vehicle surroundings is estimated with a CTRV model. The state space at time step k is defined as where X k is the relative distance in the longitudinal direction, Y k is the relative distance in the lateral direction, v k is the target speed, ψ k is the relative course heading, and ω k is the yaw rate. The state transition equation for the prediction of the state at time step k + 1 is given by For the purpose of risk assessment, the circle model described in [23] is employed and a possible collision event is predicted based on the future trajectories of the remote vehicle and the host vehicle as shown in Figure 5. The host vehicle radius R HV and the remote vehicle radius R RV are given by where W HV and L HV denote the width and the length of the host vehicle, respectively, and W RV and L RV denote the width and the length of the remote vehicle, respectively. A possible collision is determined if the following inequality is true: For the purpose of risk assessment, the circle model described in [23] is employed and a possible collision event is predicted based on the future trajectories of the remote vehicle and the host vehicle as shown in Figure 5. The host vehicle radius and the remote vehicle radius are given by where and denote the width and the length of the host vehicle, respectively, and and denote the width and the length of the remote vehicle, respectively. A possible collision is determined if the following inequality is true:  The detection of a possible collision leads to the generation of a collision warning to the host vehicle. Based on the time-to-collision (TTC) estimate, four different collision warning messages are provided. Following the collision warning stages discussed in [69], the warning messages provided by the proposed system include "no threat" for no possible collision, "threat detected" for TTC > 2.6 s, "inform driver" for 1.6 s < TTC ≤ 2.6 s, and "warn driver" for TTC ≤ 1.6 s. The description for each warning message is described in Table 5. The conditions for the collision warning messages defined here are similar to those of Daimler PRE-SAFE [70] and Mobileye FCW [71]. The PRE-SAFE and Mobileye systems warn the driver approximately 2.6 s and 2.7 s before the expected collision, respectively. In the case of the PRE-SAFE system, an additional warning is provided at approximately 1.6 s before the expected collision.

Experimental Evaluation
The proposed system for cooperative driving environment perception was evaluated experimentally in test-track environments. Two test vehicles were utilized for the experiments where both vehicles were equipped with V2X communication devices such that the exchange of the BSM between the two test vehicles was enabled. In order to evaluate the benefits of introducing V2X communications to the currently available environment perception systems, the host vehicle in this work was additionally equipped with radar and camera systems that have already been adopted in production vehicles.

Vehicle Configuration
Two Kia Soul cars, one in white and the other in yellow as shown in Figure 6a, were used for the experiments. The experiment setup consisted of a combination of hardware and software components that enabled the host vehicle to gather information on the vehicle surroundings by means of on-board remote sensors and V2X communications. As shown in Figure 6b, the host vehicle (Kia Soul in white) was equipped with a Delphi ESR 2.5 radar and a Mobileye 630 camera for detection of surrounding objects, and each test vehicle was equipped with a Cohda MK5 OBU, a DSRC antenna, a u-blox F9P GNSS receiver, and a GNSS antenna, all of which together enabled the vehicles to exchange BSM via V2X communications and share vehicle position information with a lane-level accuracy. The u-blox F9P device is a low-cost GNSS module with built-in support for standard RTCM corrections. For real-time data acquisition and processing, the host vehicle was equipped with a dSPACE SCALEXIO AutoBox, which is an in-vehicle prototyping system with a real-time processor and interfaces for CAN and Ethernet communications. The signals transmitted from the radar and camera systems as well as the in-vehicle ECUs were acquired via CAN interfaces, and the signals transmitted from the DSRC OBU were acquired via Ethernet interfaces. RTCM corrections were acquired with an NTRIP client running on a smartphone and streamed via a Bluetooth connection to the u-blox C099-F9P application board. In order to obtain ground truth positions for the test vehicles, both vehicles were equipped with an Ascen AKT980R, which is a highly accurate GNSS receiver that provides GNSS carrier phase measurements with a horizontal positioning accuracy of 0.008 m and a heading accuracy of 0.09 deg with a 2-m baseline. The overview of the hardware interface for the proposed system is presented in Figure 7. The driving experiments were conducted at the proving ground available at Korea Automotive Technology Institute which is shown in Figure 8. For real-time data acquisition and processing, the host vehicle was equipped with a dSPACE SCALEXIO AutoBox, which is an in-vehicle prototyping system with a real-time processor and interfaces for CAN and Ethernet communications. The signals transmitted from the radar and camera systems as well as the in-vehicle ECUs were acquired via CAN interfaces, and the signals transmitted from the DSRC OBU were acquired via Ethernet interfaces. RTCM corrections were acquired with an NTRIP client running on a smartphone and streamed via a Bluetooth connection to the u-blox C099-F9P application board. In order to obtain ground truth positions for the test vehicles, both vehicles were equipped with an Ascen AKT980R, which is a highly accurate GNSS receiver that provides GNSS carrier phase measurements with a horizontal positioning accuracy of 0.008 m and a heading accuracy of 0.09 deg with a 2-m baseline. The overview of the hardware interface for the proposed system is presented in Figure 7. The driving experiments were conducted at the proving ground available at Korea Automotive Technology Institute which is shown in Figure 8.

Experimental Environment
In order to obtain relative position estimates of the remote vehicle from the rada camera and also from V2X communications, driving experiments were conducted u the scenarios where the remote vehicle is driven ahead the host vehicle. Fo performance evaluation of the proposed data fusion approach for different relative t locations, the relative distance between the two test vehicles was gradually increase shown in Figure 9, the first set of experiments was carried out under a scenario w both test vehicles drove in the same lane, and the second set of experiments conducted under a scenario where the remote vehicle drove in the adjacent lane. Th driving scenarios, which are referred to as the "normal driving scenarios" here summarized in Table 6. For these two normal driving scenarios, dynamic dr maneuvers such as lane change maneuvers were not performed. For both dr scenarios, the host vehicle was driven at a speed of about 20 km/h while the remote ve was driven at about 25 km/h. The red and blue dashed lines in Figure 9 indicate the of the camera and radar sensors, respectively. The narrower set of dashed lines in indicates the long-range radar coverage, whereas the wider set indicates the mid-r radar coverage. The relative positions of the target vehicle obtained from the r camera, and V2X communications are plotted in different colors. The red dot indicate camera measurement while the blue dot indicates the radar measurement. The d magenta indicates the relative position obtained based on V2X communication data finally black dot denotes the ground truth position. All of these dots represen estimated distance to the rear center of the remote vehicle from the origin, which center of the front bumper of the host vehicle. The magenta bounding box is created rotated based on the width, length, and heading information obtained from

Experimental Environment
In order to obtain relative position estimates of the remote vehicle from the ra camera and also from V2X communications, driving experiments were conducte the scenarios where the remote vehicle is driven ahead the host vehicle. performance evaluation of the proposed data fusion approach for different relativ locations, the relative distance between the two test vehicles was gradually incre shown in Figure 9, the first set of experiments was carried out under a scenari both test vehicles drove in the same lane, and the second set of experime conducted under a scenario where the remote vehicle drove in the adjacent lane. driving scenarios, which are referred to as the "normal driving scenarios" h summarized in Table 6. For these two normal driving scenarios, dynamic maneuvers such as lane change maneuvers were not performed. For both scenarios, the host vehicle was driven at a speed of about 20 km/h while the remot was driven at about 25 km/h. The red and blue dashed lines in Figure 9 indicate of the camera and radar sensors, respectively. The narrower set of dashed lines indicates the long-range radar coverage, whereas the wider set indicates the m radar coverage. The relative positions of the target vehicle obtained from th camera, and V2X communications are plotted in different colors. The red dot indi camera measurement while the blue dot indicates the radar measurement. Th magenta indicates the relative position obtained based on V2X communication d finally black dot denotes the ground truth position. All of these dots repre

Experimental Environment
In order to obtain relative position estimates of the remote vehicle from the radar and camera and also from V2X communications, driving experiments were conducted under the scenarios where the remote vehicle is driven ahead the host vehicle. For the performance evaluation of the proposed data fusion approach for different relative target locations, the relative distance between the two test vehicles was gradually increased. As shown in Figure 9, the first set of experiments was carried out under a scenario where both test vehicles drove in the same lane, and the second set of experiments was conducted under a scenario where the remote vehicle drove in the adjacent lane. The two driving scenarios, which are referred to as the "normal driving scenarios" here, are summarized in Table 6. For these two normal driving scenarios, dynamic driving maneuvers such as lane change maneuvers were not performed. For both driving scenarios, the host vehicle was driven at a speed of about 20 km/h while the remote vehicle was driven at about 25 km/h. The red and blue dashed lines in Figure 9 indicate the FOV of the camera and radar sensors, respectively. The narrower set of dashed lines in blue indicates the long-range radar coverage, whereas the wider set indicates the mid-range radar coverage. The relative positions of the target vehicle obtained from the radar, camera, and V2X communications are plotted in different colors. The red dot indicates the camera measurement while the blue dot indicates the radar measurement. The dot in magenta indicates the relative position obtained based on V2X communication data, and finally black dot denotes the ground truth position. All of these dots represent the estimated distance to the rear center of the remote vehicle from the origin, which is the center of the front bumper of the host vehicle. The magenta bounding box is created and rotated based on the width, length, and heading information obtained from V2X communications at a given time step. Finally, the black bounding box represents the ground truth position of the remote vehicle.
communications at a given time step. Finally, the black bounding box represents the ground truth position of the remote vehicle.      As presented in Tables 7 and 8, the performance of the proposed data fusion method is evaluated by computing the root mean squared error (RMSE) and the standard deviation of the error, and compared with those calculated for other remote sensing systems. In order to determine the positioning accuracy at varying relative distances, the results are grouped in separate 10-m bins (in the longitudinal direction). For the driving scenario with both vehicles in the same lane, the longitudinal and lateral localization accuracies of the proposed method in terms of the RMSE are found to be 0.22 m and 0.13 m, respectively, when taking into account all the results that correspond to the range of longitudinal distances between 0 and 70 m.  Table 8. Relative positioning accuracy in the lateral direction for the measurements obtained during the normal driving scenario 1. The results obtained when the target vehicle is at a relative longitudinal distance of 70 m or longer are not used for this performance evaluation, considering that the lateral position accuracy of the GNSS system used for ground truth degrades at such long distances due to the limited heading accuracy such that it may not be appropriate to be utilized as a reference system. In addition, the Mobileye camera used in this work appears to suffer from significant degradation in longitudinal accuracy for remote targets located at such distances.

RMSE (m) SD (m) RMSE (m) SD (m) RMSE (m) SD (m) RMSE (m) SD (m)
The ground truth distance to the remote vehicle during the driving scenario where the remote vehicle is in the adjacent lane is shown in Figure 11a,b. As previously explained, the relative positions of the remote vehicle acquired from separate test runs are concatenated and presented together, and the time step between two consecutive frames from the same test run corresponds to 0.1 s. The separation between the two test vehicles gradually increases while the remote vehicle stays in the adjacent lane. The absolute error for the longitudinal and lateral distances to the remote vehicle is presented in Figure 11c,d. Similar to the aforementioned case with both vehicles in the same lane, the proposed fusion method reliably estimates the remote target position at varying distances even at times when the accuracy of individual remote sensors becomes severely degraded. The performance of the individual remote sensing systems and the proposed cooperative environment perception approach is evaluated by computing the RMSE and the standard deviation of the error at different relative distances as shown in Tables 9 and 10. Similar to the previous scenario, the results are grouped in separate 10-m bins (in the longitudinal direction) to determine the positioning accuracy at varying relative distances for the driving scenario with the remote vehicle driving in the adjacent lane. Taking into account all of the position estimates that fall between 0 and 70 m in the longitudinal direction, the longitudinal and lateral localization accuracies of the proposed method in terms of the RMSE are found to be 0.33 m and 0.09 m, respectively.   Figure 12 shows the relative positioning accuracy in the longitudinal and lateral directions for the combined sets of the measurements from the two normal driving scenarios. In the longitudinal direction, the total RMSE of the relative position estimated with the proposed fusion approach is 0.27 m, which corresponds to an improvement of 86% and 52% compared to the total RMSE of the relative position estimated with the camera and radar sensors, respectively. In the lateral direction, the total RMSE of the relative position estimated with the proposed fusion approach is 0.12 m, which corresponds to an improvement of 27% and 56% compared to the total RMSE of the relative position estimated with the camera and radar sensors, respectively.

Experimental Environment
The performance of the proposed cooperative environment perception approach was also evaluated in cut-in driving scenarios where the remote vehicle performs a lane-change maneuver in front of the host vehicle. Three different cut-in scenarios were considered in this work as described in Table 11. The host vehicle traveled at 40-45 km/h at the start of the cut-in maneuver in all three cut-in driving scenarios, while the remote vehicle originally driving in the adjacent lane cut in at a distance of 15-20 m in front of the host vehicle. The speed of the remote vehicle was set differently for each cut-in scenario in order to vary the level of collision threat, such that the scenario 1 presents the lowest level of threat while the highest level of threat is expected in the scenario 3.
Sensors 2021, 21, x FOR PEER REVIEW 21 corresponds to an improvement of 27% and 56% compared to the total RMSE of relative position estimated with the camera and radar sensors, respectively.

Experimental Environment
The performance of the proposed cooperative environment perception approach also evaluated in cut-in driving scenarios where the remote vehicle performs a l change maneuver in front of the host vehicle. Three different cut-in scenarios w considered in this work as described in Table 11. The host vehicle traveled at 40-45 k at the start of the cut-in maneuver in all three cut-in driving scenarios, while the rem vehicle originally driving in the adjacent lane cut in at a distance of 15-20 m in front o host vehicle. The speed of the remote vehicle was set differently for each cut-in scen in order to vary the level of collision threat, such that the scenario 1 presents the low level of threat while the highest level of threat is expected in the scenario 3.

. Performance Evaluation
The ground truth relative position of the remote vehicle and the absolute error the relative position during the cut-in driving scenario 1 are shown in Figure 13. longitudinal and lateral relative distance measurements obtained from separate test r are concatenated and shown together. The start and the duration of the cut-in events be conveniently observed in Figure 13b

Performance Evaluation
The ground truth relative position of the remote vehicle and the absolute error for the relative position during the cut-in driving scenario 1 are shown in Figure 13. The longitudinal and lateral relative distance measurements obtained from separate test runs are concatenated and shown together. The start and the duration of the cut-in events can be conveniently observed in Figure 13b, where the lateral distance changes from the center of the adjacent lane towards the center of the lane that the host vehicle is positioned. The TTC results and the corresponding levels of the collision warning obtained from the proposed method for trajectory prediction and risk assessment are presented in Figure 14. Possible collision events are successfully predicted for all four cut-in maneuvers performed by the remote vehicle. The TTC results for the collision detection shown are above 4 s except the third cut-in case. The levels of these threats other than the third one are minor, resulting in level-1 warnings only for a short duration of time. A level-2 warning is given for the third cut-in event, which is due to the more abrupt change in the longitudinal distance just prior to and during the lane-change maneuver. Figure 15 shows the ground truth position and the absolute position error in longitudinal and lateral directions during the cut-in driving scenario 2. The data collected from separate test runs are concatenated as presented. A total of five cut-in events take place in this scenario, and the start and the duration of these events can be observed in Figure 15b. The TTC results and the corresponding collision warning levels are presented in Figure 16. For all five cut-in maneuvers performed by the remote vehicle in the scenario 2, possible collision events are correctly predicted and corresponding collision warnings are provided.  Figure 15 shows the ground truth position and the absolute position error in longitudinal and lateral directions during the cut-in driving scenario 2. The data collected from separate test runs are concatenated as presented. A total of five cut-in events take place in this scenario, and the start and the duration of these events can be observed in Figure 15b. The TTC results and the corresponding collision warning levels are presented in Figure 16. For all five cut-in maneuvers performed by the remote vehicle in the scenario 2, possible collision events are correctly predicted and corresponding collision warnings are provided.  Figure 17 presents the ground truth relative position of the remote vehicle and the absolute error for the relative position during the cut-in driving scenario 3. The data obtained from separate test runs are concatenated and presented together. A total of eight cut-in events can be recognized in Figure 17b. An unusual remote vehicle maneuver in the lateral direction can be noticed before the initiation of the fourth cut-in maneuver. This was caused by an incident where our driver of the remote vehicle steered away to avoid another vehicle that was used by a different group in the proving ground. Despite the higher fluctuations of the position estimates from the individual remote sensing systems as shown in Figure 17c,d, the proposed cooperative approach provides accurate and reliable positioning results throughout the execution of cut-in maneuvers. The TTC estimates and the corresponding warning levels are presented in Figure 18. The results show that the proposed system for trajectory prediction and risk assessment enables reliable and timely warning to cope with the sudden lane change maneuvers performed in the cut-in driving scenario 3.  Figure 17 presents the ground truth relative position of the remote vehicle and the absolute error for the relative position during the cut-in driving scenario 3. The data obtained from separate test runs are concatenated and presented together. A total of eight cut-in events can be recognized in Figure 17b. An unusual remote vehicle maneuver in the lateral direction can be noticed before the initiation of the fourth cut-in maneuver. This was caused by an incident where our driver of the remote vehicle steered away to avoid another vehicle that was used by a different group in the proving ground. Despite the higher fluctuations of the position estimates from the individual remote sensing systems as shown in Figure 17c,d, the proposed cooperative approach provides accurate and reliable positioning results throughout the execution of cut-in maneuvers. The TTC estimates and the corresponding warning levels are presented in Figure 18. The results show that the proposed system for trajectory prediction and risk assessment enables reliable and timely warning to cope with the sudden lane change maneuvers performed in the cut-in driving scenario 3.  Figure 17c,d, the proposed cooperative approach provides accurate and reliable positioning results throughout the execution of cut-in maneuvers. The TTC estimates and the corresponding warning levels are presented in Figure 18. The results show that the proposed system for trajectory prediction and risk assessment enables reliable and timely warning to cope with the sudden lane change maneuvers performed in the cut-in driving scenario 3.

Conclusions
In this paper, we presented the experimental design and performance evaluation of the driving environment perception system based on the fusion of multiple on-board

Conclusions
In this paper, we presented the experimental design and performance evaluation of the driving environment perception system based on the fusion of multiple on-board sensors and V2X communications. The two test vehicles used for the driving experiments were each equipped with DSRC equipment and a low-cost GNSS receiver for the exchange of BSM data. The host vehicle was additionally equipped with radar and camera sensors that have already been adopted in production vehicles. The performance of the proposed fusion approach in terms of relative positioning accuracy was evaluated at varying longitudinal and lateral distances between the two test vehicles. In the longitudinal direction, the total RMSE of the relative position estimated with the proposed fusion approach was 0.27 m, whereas those estimated with the camera and radar sensors were 1.91 m and 0.57 m, respectively, which correspond to an improvement of 86% and 52%. In the lateral direction, the total RMSE of the relative position estimated with the proposed method was 0.12 m, while those estimated with the camera and radar were 0.16 m and 0.26 m, respectively, corresponding to an improvement of 27% and 56%. The performance of the trajectory prediction and risk assessment was evaluated in different cut-in driving scenarios where the remote vehicle performs a lane-change maneuver in front of the host vehicle. The test results showed that the proposed method provides accurate target localization and facilitates reliable target trajectory prediction and detection of potential collision during the events when a remote vehicle driving in an adjacent lane cuts in front of the host vehicle.
Although the proposed cooperative approach for driving environment perception proved to be successful in the driving scenarios considered in this study, there still exists a number of challenges to be addressed. The scope of this paper has been limited to driving scenarios where a single remote target is present in the surroundings of the host vehicle. As part of future work, the benefits of the cooperative environment perception approach will be further investigated in scenarios that involve multiple surrounding objects for more diverse use cases. In addition, various factors that could adversely affect the accuracy and reliability of on-board sensors (e.g., high-curvature roads and adverse weather conditions) and V2X communications (e.g., large separations between vehicles, urban environments) will be examined so that the data quality levels that can be expected for each sensor track and for V2X communications can be more accurately determined in continuously changing driving conditions. It is also important to note that the trajectory prediction performed in the scope of this study was based on the physics-based motion model, which performs well for short-term prediction but degrades in its performance when the prediction horizon is extended. Furthermore, this trajectory prediction approach cannot anticipate the state changes caused by varying road curvature, traffic signals, or future driving maneuver execution. Therefore, future work should investigate interaction-aware motion models and also incorporate the map data and the signal phase and timing message, which can be acquired via V2X communications, for accurate and reliable longer-term trajectory prediction that takes into account inter-vehicle interaction, road configuration, and traffic signal dependencies.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.