A Study on the Evaluation Method of Autonomous Emergency Vehicle Braking for Pedestrians Test Using Monocular Cameras

: Tra ﬃ c accidents continue to increase in Korea as tra ﬃ c increases, and the resulting loss of life is also on the rise. According to data surveyed by the South Korean National Police Agency, 45,921 pedestrian tra ﬃ c accidents were reported in 2019, resulting in 1487 deaths and 46,400 injuries. Due to the increased interest in tra ﬃ c accident safety, the Advanced Driver Assistance System (ADAS) concept is rapidly developing and playing a signiﬁcant role in coping with activities that are not recognized by the driver. Autonomous Emergency Braking (AEB), a representative ADAS system, is a system that is useful for preventing and mitigating accidents by braking vehicles in emergencies. For the study of AEBs’ safety evaluation methods for pedestrians, a distance measurement method using a monocular camera with excellent accessibility, and measurement equipment required to validate data on the movement of vehicles, and a dummy to replace pedestrians are used. Based on the evaluation scenario considering the proposed Korea road environment, the relative distance obtained from equipment like DGPS and the relative distance using a monocular camera is compared and analyzed to verify safety. Comparative analysis shows that the minimum deviation is 2.3 cm, the third test result of 30 km / h of Car-to-Pedestrian Nearside Child (CPNC), and the maximum deviation is 25 cm, the ﬁrst test result of 25 km / h of Car-to-Pedestrian Nearside Adult (CPNA). The main factor in error generation is that the lane recognition in the camera image is not accurate, and the perception of small children is slow, which is why emergency braking is considered to have been slow. It is deemed that a safety assessment in weather conditions of adverse conditions will be required in the future.


Introduction
As the volume of domestic traffic increases, traffic accidents continue to increase, leading to increased casualties. According to data surveyed in 2019 by the Korea National Police Agency, the death rate due to traffic accidents has been decreasing every year as of 1991, but the number of injuries continues to increase.
A total of 45,921 pedestrian traffic accidents were reported in 2019, with 1487 deaths and 46,400 injuries. Among them, children under the age of 13 were very vulnerable to traffic accidents, with 34 deaths, and 12,543 injuries [1].
Safety concerns are also increasing due to traffic accidents. As a way to solve this problem, the Advanced Driver Assistance System (ADAS) concept, which helps vehicles prevent or mitigate accidents and reduces drivers' fatigue by responding to incidents that the driver does not recognize, is developing rapidly and at the same time playing an important role.
A typical ADAS function is the Autonomous Emergency Braking (AEB) system that performs braking in an emergency. By using car-sensing devices (radars, cameras, and sensors), the AEB system can recognize the surrounding driving environment [2,3]. This system operates when the preceding vehicle suddenly brakes due to an accident, or when there is an object that is suddenly entering the driveway of the vehicle in the road. The AEB system can effectively improve the driving safety of vehicles, reduce the occurrence of collision traffic accidents [4], and the working intensity of drivers. For that reason, it is evaluated as a useful system for preventing and mitigating accidents [5].
After looking at the research trends of ADAS tests and pedestrian AEB, Woo et al. [6] proceeded with the design and simulation of a vehicle test bed based on intelligent transport systems (ITS). The ITS-based intelligent vehicle test bed was constructed to meet the growing demand for test and verification for ADAS and ITS systems. This test bed is carefully designed to meet the requirements of the International Organization for Standardization (ISO)/Technical Committee (TC)204 standards. Yang et al. [7] presented a driver study on longitudinal active collision avoidance of the AEB-P system. The lower-layer controller of the Autonomous Emergency Braking-Pedestrian (AEB-P) system was designed based on the Proportional Integral Derivative (PID) controller theory, which realizes the conversion of the expected speed reduction to the pressure of a vehicle braking pipeline. The relevant pedestrian test scenarios were set up based on the China-New Car Assessment Program (C-NCAP) test standards. The CarSim and Simulink co-simulation model of the AEB-P system was established, and a multi-condition simulation analysis was performed. Lenard et al. [8] studied typical pedestrian accident scenarios for the development of autonomous emergency braking test protocols. This paper aimed to contribute to the development of relevant test conditions by describing pedestrian accidents' typical circumstances. Cluster analysis was applied to two large British databases, and both highlighted an urban scenario in daylight and fine weather where a small pedestrian walks across the road, especially from the near curb, in clear view of a driver who is traveling straight ahead. For each dataset, the main test configuration was defined to represent the conditions of the most common accident scenario along with test variations to reflect the characteristics of less common accident scenarios. Hamdane et al. [9] studied issues and challenges for pedestrian active safety systems based on real world accidents. The purpose of this study was to analyze real crashes involving pedestrians in order to evaluate the potential effectiveness of AEB in pedestrian protection. These were assessed to determine their impact on pedestrian safety. The influence of the detection and the activation of the AEB system were explored by varying the field of view (FOV) of the sensor and the level of deceleration. A FOV of 35 • was estimated to be required to detect and react to the majority of crash scenarios. Siddiqui et al. [10] analyzed an empirical study of the braking performance of AEB-P. The Insurance Institute for Highway Safety (IIHS) has performed tests on eleven such vehicles; these data are publicly available and were analyzed for this study. This study's first objective was to compare forward collision warning (FCW) engagement distance to a target, AEB-P brake application time, and incidences of impact across different manufacturers. The second objective was to analyze the brake application characteristics of AEB-P and how it varies across the different vehicles tested. Kwon et al. [11] conducted a study on the test evaluation method of AEB (vehicle to pedestrian), Considering the road environment in Korean and The European New Car Assessment Program (Euro NCAP) Test Protocol V3.0.1. In Euro NCAP, the AEB vulnerable road user (VRU) test protocol V3.0.2 was established. This paper analyzes the scenario of Euro NCAP VRU test protocol V3.0.1, which will be established in 2020, and proposes test conditions according to the Korean road traffic law. In addition, the reliability of the proposed scenario and test conditions were verified by comparing and analyzing the proposed theoretical evaluation formulas and actual test results.
In addition, Jeppsson et al. [12] studied the real life safety benefits of increasing brake deceleration in car-to-pedestrian accidents using a simulation of vacuum emergency braking (VEB). The objective of this study was to predict the real-life benefits, namely the number of injuries avoided rather than the reduction in impact speed, offered by a vacuum emergency brake added to an AEB-P system. Lubbe et al. [13] studied drivers' comfort boundaries in pedestrian crossings in a study on driver braking characteristics as a function of pedestrian walking speed. In this study, driver discomfort can be inferred from brake onset, which refers to the start of brake pedal depression, as the most intuitive way for a driver to resolve a conflicting situation. Using the Euro NCAP AEB VRU test protocol v3.0.2. [14] to support the driver in avoiding when possible or mitigating such crashes, car manufacturers offer avoidance technology that reacts to the situation by autonomous braking and at higher speeds may issue warnings to alert the driver. Systems that specifically look for and react to vulnerable road users like pedestrians and cyclists are called AEB VRU systems. This protocol specifies the AEB VRU test procedure for both AEB pedestrians and AEB cyclists, which are part of the vulnerable road user protection scheme. Shi et al. [15] analysed pedestrian-to-ground impact injury risks in vehicle-to-pedestrian collisions based on rotation angles. Due to the diversity of pedestrian-to-ground impact (secondary impact) mechanisms, secondary impacts always result in more unpredictable injuries as compared to initial vehicle-to-pedestrian collisions (primary impact). This study investigates the effects of frontal vehicle structure, vehicle impact velocity, and pedestrian size and gait on pedestrian-to-ground impact injury risk. Murmu et al. [16] studied relative velocity measurements using a low cost single camera-based stereo vision system. This article presents a novel low cost, low-resolution stereo vision system based on a single camera, which may be used to measure the position and relative velocity of a moving object. This new system is very simple in design, low cost, and capable of measuring the depth and relative velocity with desirable accuracy. Mudassar et al. [17] studies a framework for estimating the distance and dimension attributes of pedestrians in real-time environments using a monocular camera. This work describes a method that uses simple mathematical estimations to automatically discover the distance and dimensions (height and width) of a moving pedestrian lying at distant locations from the camera. The proposed system is confined to immovable monocular camera environments. Outcomes are compared to the found results of existing methods as well as with the real measurements. The results show the robustness of the proposed framework with noteworthy delay rates. Zaarane et al. [18] studied a measurement system for autonomous vehicles using a stereo camera.
In the current paper, we propose an inter-vehicle distance measurement system for self-driving vehicles based on image processing. The proposed system uses two cameras mounted as one stereo camera system in the host vehicle behind the rear-view mirror. Bae et al. [19] conducted a study on a distance calculation method with a vehicle using a forward-facing single camera. This paper calculates the lane width from the image with the lane being detected from the image of the monocular camera, and based on geometric information. A theoretical formula was proposed to calculate the distance from the vehicle in front.
The general AEB test evaluation has many restrictive conditions because of the expensive measurement equipment and advanced utilizable technology. However, in the case of a monocular camera, it has relatively good accessibility and non-restrictive conditions, so in this study AEB research on pedestrian safety using a monocular camera was conducted, because it was judged to be insufficient and required. Therefore, in this study, an AEB safety evaluation method using a monocular camera is proposed and verified for the study of AEB evaluation method for pedestrian safety using a monocular camera by comparing and analyzing the result of measuring the relative distance using a monocular camera and the result of measuring the relative distance using measurement equipment based on the test evaluation scenario in the domestic road environment proposed in the previous study [11,19].

Theory of Distance Measurement Using a Monocular Camera
In a previous study, a distance calculation method using geometric information of lanes, vanishing points, and preceding vehicles detected in the camera image mounted on the vehicle was proposed. The method is as follows [10]: First, the lane width in the image may be calculated by using the lane and vanishing point detected in the camera image. The detected lanes and vanishing points are shown in Figure 1. Where L 1 is the width of the lane for the contact portion of the rear tire of the preceding vehicle with the ground, L 2 is the width of the lane from the top of the car hood, and H 1 is the vertical distance between L 1 and L 2 . H 2 is the vertical distance from the vanishing point to L 1 . Figure 2 shows the principle of distance calculation using geometric information with the preceding vehicle. Equation (1) can be obtained using a proportional expression in Figure 2: where h 1 is the height of the camera from the ground, h 2 is the height from the hood to the camera, b is the distance from the camera to the top of the hood, and c is the distance from the camera to the front bumper. The term a is the distance from the camera, which is the length of the hood of the car to the front bumper. It can be calculated from the camera's installation height, installation angle, and vehicle specifications. θ can be calculated as the difference of a from the angle from the camera to the rear wheel of the vehicle, and l is the height visible from the image to the rear wheel of the preceding vehicle in the ground covered by the hood of the vehicle. l * can be calculated as the difference of l from the camera installation height. Therefore, it can be expressed by Equations (2)-(4) using the width of the lane calculated in the image: The method to obtain the distance from the image by using Equations (1)-(4) is defined by Equation (5): d image is a distance calculated from the lane height in the image. However, it is assumed that the specifications of the preceding vehicle (rear overhang (d O.H )) are known from the image.
The relative distance from the preceding vehicle can be defined based on geometrical information. It is shown in Equation (6): where d real is the distance from the preceding vehicle and can be expressed by a, Equations (1)- (5). Equation (6) is in the curve, the distance to the leading vehicle or pedestrian is expressed in a straight line. Also, in the case without d O.H , it is expressed as the distance to the rear tire of the leading vehicle, not the value for the end of the bumper, but for pedestrians, d O.H is 0.

Test Scenario
The AEB recognizes pedestrians as well as vehicles and obstacles in front and plays an essential role in reducing traffic accidents between vehicles and pedestrians. Euro NCAP acknowledges this role and is enacting additional scenarios for V2P each year to reflect new trends [20].
The scenario proposed in the previous study suggested a scenario considering the speed limit of 30 km/h in the child protection zone under Article 19 of the Enforcement Decree of the Road Traffic Act regarding Euro NCAP AEB VRU test protocol v3.0.2. [11,14].
The AEB V2P scenario added a scenario (including the longitudinal and transverse movement of traffic vulnerabilities) involving children running to the scenario of the AEB VRU test protocol v3.0.2, and utilizing the suggested scenario.
Among the suggested scenarios, there are three scenarios in which the vehicle test was conducted, and each one is shown in Figures 3-5.    Figure 3 shows a car-to-pedestrian nearside adult (CPNA) scenario and is the proposed scenario 1. It is an evaluation scenario for an adult walking sideways in front of a moving subject vehicle (SV). At this time, SV recognizes the walking adult and checks whether the AEB is operating normally. The test conditions are shown in Table 1.  Figure 4 shows a car-to-pedestrian farside child (CPFC) scenario and is the proposed scenario 2. It is an evaluation scenario for a child running sideways in front of a driving SV. At this time, SV recognizes the running child and checks whether the AEB is operating normally. The test conditions are shown in Table 1. Figure 5 shows a Car-to-Pedestrian Nearside Child (CPNC) scenario and is the proposed scenario 3. This is an evaluation scenario in the case where a child jumps in front of LV 1, and LV 2 stopped on the shoulder and appeared laterally in front of the driving SV. At this time, SV recognizes the running child and checks whether the AEB is operating normally. The test conditions are shown in Table 1.

Vehicle for Vehicle Test
To verify the actual functional safety of AEB, the Genesis G90 was selected based on vehicle tests on a real road. It is shown in Figure 6. The Genesis G90 is equipped with an Active Safety System, based on sensors such as camera sensors and radar sensors which is considered excellent.

Place for Vehicle Test
The place where the AEB safety evaluation test was conducted is a vehicle-road link test intersection among driving test roads of the Korea Intelligent Automobile Parts Promotion Institute (KIAPI)', and is shown in Figure 7. Since the friction coefficient and the width of the real road are applied in the driving test road, it was determined that the driving test road was suitable for the test site. At this time, 'skid-resistance' (STANLEY LONDON) was used to confirm that the average friction coefficient of the test road was 1.08. The friction coefficient at least 0.9 or more of the general road was judged to be suitable for the test environment.

Condition for Vehicle Test
The equipment necessary to acquire data through the vehicle test was attached to the SV. Figure 8 shows the equipment that is mounted in the test vehicle. Differential global positioning system (DGPS), data acquisition (DAQ) and camera modules were used, and the equipment specifications are summarized in Table 2.   Figure 9 shows the dummies of adults and children to implement the proposed scenario. The characteristics are listed in Table 3.   Figure 10 is a monocular camera that is mounted in the test vehicle and was used to calculate the relative distance in Equation (6). The specifications of the monocular camera are summarized in Table 4.  The DGPS was mounted on the vehicle center of gravity (CG) point, and the distance from the right side of the vehicle to the DGPS, the distance from the vehicle bumper to the DGPS, and the distance from the road surface to the DGPS were measured and corrected. In addition, the accuracy was set to 3 cm or less by triangular correction using an additional antenna installed in KIAPI and DGPS mounted on the test vehicle. The DAQ was connected to the DGPS, and Placed in the rear seat of the test vehicle, and corrected the initial value of CAN and Bluetooth communication. The camera module was mounted on the test vehicle's dashboard to record the front to check the scenario in the data analysis process. The monocular camera was mounted on the top center of the front glass to record the front in the same measurement location in the theoretically proposed measurement location. After power was supplied to the test equipment before the test, the location was corrected. Furthermore, the data was measured by conducting three repeated tests, after the vehicle was designated as 0 points in contact with the dummy. Also, it was set again each time an event such as a collision occurred due to AEB being non-activated during the test.

Distance Measurement Results Using A Monocular Camera
The vehicle test for distance measurement using a monocular camera was conducted to acquire images of each scenario.
Tables 5-7 are data of the images recorded during the vehicle tests with a monocular camera, which can be calculated using Equation (6).
By measuring L 1 and L 2 , d image , a were theoretically calculated, and d real of CPNA was calculated from a minimum of 173 cm to a maximum of 288 cm. CPFC was calculated from a minimum of 148 cm to a maximum of 237 cm, and CPNC was calculated from a minimum of 51 cm to a maximum of 286 cm.

Vehicle Test Results
The results of the test for speed, acceleration, and relative distance for CPNA conducted at the vehicle-road link test intersection of the KIAPI were shown in Figure 11, CPFC is shown in Figure 12, CPNA is shown in Figure 13, respectively. The graphs of Figures 11-13 show the results of measurements using measurement equipment. The measured value using DGPS were measured as a relative distance by setting the point to 0 m because the target is in the defined position. The test was repeated three times for each scenario. Besides, Table 8 shows the relative distances measured by measurement equipment for each test. By measuring L 1 and L 2 , d image , a were theoretically calculated, and d real of CPNA was calculated from a minimum of 173 cm to a maximum of 288 cm. CPFC was calculated from a minimum of 148 cm to a maximum of 237 cm, and CPNC was calculated from a minimum of 51 cm to a maximum of 286 cm.   Only a few of the 10 scenarios proposed in the preceding study were conducted. The reason is that AEB function was not activated when the driver steered the vehicle for sale. Even the user's manual of the vehicle warns that the AEB's performance may be deteriorated and it may not brake. The test was impossible as the AEB function was released when the driver operated the vehicle following a radius of rotation.

Results Comparative Analysis
Tables 9-11 are vehicle tests for scenarios by dummy and speed. The deviation of the obtained relative distances, such as the relative distance of the monocular camera and DGPS, were compared and organized. The minimum deviation of CPNA was 13 cm in the test result of 30-3, and the maximum deviation was 25 cm in the test result of 25-1. The minimum deviation of CPFC was 2.3 cm in the test result of 30-3, and the maximum deviation was 21.9 cm in the test result of 25-1. The minimum deviation of CPNC was 3 cm in the test results of 25-3, and the maximum deviation was 12 cm in the test results of 20-2, 20-3, and 25-1.
The cause of the maximum deviation in each scenario is due to the inaccurate lane recognition in the camera image. In particular, an additional reason for significant deviations in scenario 20-1, 25-1, is believed that the emergency braking activated late, as the recognition of the small child dummy was made late.

Conclusions
In this paper, the relative distance was measured, compared, and analyzed to verify the safety evaluation method of the AEB function utilizing a monocular camera after the AEB was operated with a monocular camera and DGPS equipment using the pedestrian evaluation scenario suggested in the previous study. The KIAPI conducted vehicle testing, and the following conclusions were drawn: (1) In a previous study, using a vanishing point detected in the video taken by the monocular camera and geometric information that calculates the lane width from a lane, a theoretical formula was proposed to calculate the relative distance from the vehicle in front. (2) A vehicle test was conducted using a pedestrian safety emergency braking scenario tailored to the domestic road environment. (3) The vehicle test was conducted by a real vehicle at the vehicle-road link test intersection of the KIAPI. It was repeated three times with the same equipment and the driver to ensure objectivity, dummies of adults and children were used for pedestrians, and the vehicle used was a Genesis G90. (4) In the same test vehicle, the relative distance measured by measurement equipment like precise DGPS and relative distances using a monocular camera was compared. In the results of CPNA, the minimum deviation was 13 cm in 30-3, and the maximum deviation were 25 cm in 25-1. In the results of CPFC, the minimum deviation was 2.3 cm in 30-3, and the maximum deviation was 21.9 cm in 25-1. In the results of CPNC, the minimum deviation was 3 cm in 25-3, and the maximum deviation was 12 cm in 20-2, 20-3 and 25-1. (5) The cause of each scenario and the maximum deviation is because the lane recognition in the camera image was not accurate. In particular, in the scenario CPFC 20-1, 25-1, it is judged that the emergency braking operation was delayed due to the belated recognition of the small children's dummy.
In this study, to verify pedestrians' safety evaluation method using a monocular camera, the distance measurement method using the monocular camera of the previous study was applied to the AEB evaluation scenario.
The result of comparative analysis by acquiring the relative distance through the vehicle test showed a minimum deviation of 2.3 cm in 30-3 of the CPFC, and the maximum deviation is 25 cm of the CPNA.
Accordingly, the evaluation results of The AEB using a monocular camera in the driving state of the real vehicle was seen that the stopping distance is slightly longer compared to the evaluation results of the existing DGPS. The reason was judged on the measurement error when measuring the variables required for Equation (6).
However, the AEB applied to the current mass-produced car is based on expensive sensors like rider radar, so there is a considerable disadvantage to the cost. It is considered that the cost burden can be reduced by applying AEB with a monocular camera only. It was also found that it is possible to use it as a safety evaluation method in an environment where a vehicle test is impossible. In the future, it is considered that a safety evaluation in bad weather is necessary.