Next Article in Journal
Physical Education and Gender Differences in Physical Activity, Sedentary Behavior Related to Academic Success of Science-Related Courses for Children in the State of Qatar
Previous Article in Journal
Classification of Exergames with Different Types: Perceptual and Physiological Responses in College Students
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 13
Issue 19
10.3390/app131910770
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Identification of Critical Scenario Components Based on Driving Database Analysis for Safety Assessment of Automated Driving Systems

by
Hiroshi Yoshitake
1,* and
Motoki Shino
2
1
Department of Human and Engineered Environmental Studies, The University of Tokyo, Tokyo 113-8654, Japan
2
Department of Mechanical Engineering, Tokyo Institute of Technology, Tokyo 152-8550, Japan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(19), 10770; https://doi.org/10.3390/app131910770
Submission received: 15 August 2023 / Revised: 19 September 2023 / Accepted: 26 September 2023 / Published: 27 September 2023
(This article belongs to the Section Transportation and Future Mobility)
Browse Figures
Versions Notes

Abstract

:
A thorough safety assessment of an automated driving system (ADS) is necessary before its introduction into the market and practical application. Scenario-based assessments have received significant attention in research. However, identifying sufficient critical scenarios for ADSs is a major challenge, especially for complex urban environments with a variety of road geometries, traffic rules, and traffic participants. To identify the critical scenarios in this complex environment, it is essential to understand the environmental factors that lead to safety-critical events (e.g., accidents and near-miss incidents). Thus, this study proposes a method for identification of critical scenario components by analyzing near-miss incident data and extracting environmental factors that induce driver errors. In this study, we applied the proposed method to a scenario, in which an ego vehicle makes a right turn at a signalized intersection with an oncoming vehicle approaching the intersection in left-hand traffic, as a case study. The proposed method identified two components (dynamic occlusion caused by oncoming right-turn vehicles and change in traffic lights) that were both critical and challenging for ADSs. The case study results showed the usefulness of the identified components and the validity of the proposed method, which can extract critical scenario components explicitly.

1. Introduction

According to a report by the World Health Organization on road safety, the number of road fatalities worldwide exceeded 1.35 million in 2016 [1]. Automated driving systems (ADSs) with the Society of Automotive Engineers (SAE) automation levels [2] of level 3 or higher, also called highly automated driving systems (HADSs), are expected to reduce these fatalities. Furthermore, vehicles equipped with such HADSs are expected to reduce traffic congestion and fuel consumption and provide better transportation. Many companies and research institutes are conducting research and development on these systems.
In HADSs, the responsibility lies with the system and not the driver. Thus, a thorough safety assessment of the system is required, particularly when introducing the system to the market. Conventionally, mileage-based assessments have been performed to evaluate the safety of these systems. This assessment calculates the average driving distance between two accidents by ensuring a sufficient driving distance for system tests [3]. However, the driving distance required to test the safety of HADSs is predicted to be billions of kilometers or more [4,5], which is not feasible. The driving distance for mileage-based assessments of ADSs becomes unfeasibly long because they need to encounter various critical scenarios in public traffic, including rare ones, and assess the system in those scenarios.
In contrast to this mileage-based assessment, scenario-based assessment has been employed in various studies [6,7,8,9]. Scenario-based assessment is a method in which critical scenarios for ADS are intentionally generated and the response of the systems is evaluated in virtual simulations and real-world tests [10,11]. If critical scenarios are identified in advance, this method makes it possible to evaluate the performance of a system in randomly generated critical scenarios that cannot be considered in mileage-based assessments without driving long distances. Therefore, this method can significantly reduce the driving distance and effort required to test an ADS compared to conventional mileage-based assessments.
Identifying sufficient critical scenarios inside the operational design domain of ADS is a significant challenge for conducting a scenario-based assessment efficiently. Menzel et al. [12] classified scenarios into three levels, functional, logical, and concrete, according to the granularity of information. Functional scenarios are abstract scenarios described in a language format that humans can easily understand. The scenarios contain information on the sequence and behavior of the scenario components. Logical scenarios have parameter degrees of freedom described in a machine-processable form. Information on models and parameters representing elements and behaviors is assigned, and parameters are provided in ranges, distributions, or constraints between the parameters. Concrete scenarios are those with specific parameters, which satisfy the range and distribution in logical scenarios. These scenarios are used for safety assessment. A five-layer model provides a structured description of the environmental factors in a scenario [13].
The complex urban environment is deemed challenging for ADSs [14], and their critical scenarios need to be identified. In urban environments, there are differences in highways in terms of road geometry (e.g., intersections), traffic regulation (e.g., traffic lights), and traffic participants (e.g., pedestrians and cyclists). Road geometry, traffic regulations, and traffic participants correspond to Layers 1, 2, and 4 of the five-layer model, respectively. These urban environmental features must be considered in test scenarios. Thus, it is essential to understand the environmental factors (scenario components) that lead to safety-critical events (e.g., accidents and near-miss incidents) to identify the critical scenarios.
In this study, we propose a methodology to identify critical scenario components in urban environments by analyzing a human driving database with an error analysis method. Our focus is to extract environmental factors that trigger safety-critical events based on driver error analysis. The proposed method identifies components that consist of functional scenarios, which serve as a reference for identifying test cases (i.e., concrete scenarios) for ADSs in urban environments. In this study, “critical factor” refers to an environmental factor that triggers a dangerous situation, in which the ADS may make an unsafe decision that leads to an accident. The contribution of this study is to provide a methodology that can identify critical factors explicitly in urban environments through driving data analysis. The originality of this study is to utilize human driving data and driver error analysis to identify the critical factors explicitly.
The remainder of this paper is organized as follows. Section 2 briefly reviews the related works on critical scenario identification. Section 3 introduces the proposed method, including the description of the driving database and the driver error analysis method. Section 4 presents the data analysis and component identification results for a target driving scenario as a case study, and Section 5 discusses the results. Section 6 concludes the paper.

2. Related Works

Several studies have proposed methods to identify critical concrete scenarios [15,16,17,18,19,20,21,22,23,24,25,26]. Ponn et al. [15] and Paardekooper et al. [16] proposed similar methods using actual driving data. Both methods identify logical scenarios based on driving data and then identify critical scenarios by sampling the combination of scenario parameters and assessing them with certain criticality metrics. Different criticality metrics have been used in other studies to identify critical scenarios. For example, Akagi et al. [17] used the “Risk Perception” [27] to evaluate the risk of cut-in maneuvers, and Klischat and Althoff [18] used a metric that considers where an automated vehicle can drive safely. Zhang et al. [19] utilized the maximum inverse time-to-collision to divide the parameter space into safe and hazardous zones. Nalic et al. [20] proposed a method based on time-to-react measurement and incorporating the concept of variable criticality thresholds to assess the scenario’s overall criticality. Stepien et al. [21] proposed a method to generate test cases by applying heuristics to naturalistic driving data. Thal et al. [22] further developed the method in [21] by adding boundary functions and a random sampling approach to improve the criticality and coverage of the generated test cases. Khastgir et al. [23] proposed a method to identify test scenarios as an extension of the systems theoretic process analysis. In addition to using real driving data, some studies utilized simulations [24,25] and traffic accident data [26] to identify concrete scenarios that are critical for ADSs.
In contrast to the aforementioned studies, which directly identify test cases for ADSs, there are studies that focus on identifying critical functional/logical scenarios [28,29,30,31,32,33,34,35,36,37,38]. Kramer et al. [28] identified hazardous scenarios based on environmental conditions that may trigger a hazard, considering the system specifications of ADS. Similarly to [28], Ponn et al. [29] considered laws and regulations, in addition to system specifications. Weber et al. [30] proposed a framework to define safety-related scenarios based on a potential collision between an ego vehicle and an object that forces it to change its planned action. de Gelder et al. [31] proposed a method to determine sufficient parameters to describe the scenarios and a metric to assess the representativeness of a scenario in real-world scenarios derived from driving data. Nitsche et al. [32] identified pre-crash scenarios at road junctions using car-crash data and a clustering method. Hu et al. [33] used the accident database IGLAD, which contains accident data from different countries, to identify and compare the critical scenarios in different countries. Kang et al. [34] customized a vision transformer that classifies critical situations around traffic accidents using an accident dataset and proposed a framework to derive scenarios automatically. Other studies identified scenarios using police reports of traffic accidents [35,36], accident data included in a naturalistic driving study (NDS) [37], and near-miss incidents in a microscopic traffic simulation [38].
In most of the methods that identify concrete scenarios, the factors considered in the scenario are provided, and the methods search for a critical scenario inside a fixed parameter space. Regarding the methods for identifying functional/logical scenarios based on a specific data source (e.g., traffic accident data), the description of the identified scenario is limited to the content present in the data source.
When identifying critical scenarios in urban environments, it is essential to consider environmental factors that may trigger safety-critical events in complex environments. However, it is difficult to consider scenarios with these factors in most existing methods because the scenario space is a given parameter, or the data source used in the methods limits the scenario space. The method in [28] considers these factors systematically when identifying the scenarios; however, methods for extracting or identifying scenario components are not mentioned. Thus, the method proposed in this study provides a method to explicitly identify environmental factors that are critical in urban environments.

3. Methodology

3.1. Method for Identification of Critical Scenario Components

3.1.1. Overview

Figure 1 illustrates the framework of the method for identification of critical scenario components proposed in this study. This method, which can be divided into two steps, utilizes a human driving database. In Step 1, the near-miss incident database (NMI-DB) of the Tokyo University of Agriculture and Technology (TUAT) was used to analyze the driver errors in the near-miss incident (NMI) data on a target driving scenario. Critical factors were extracted in this step using the driving reliability and error analysis method (DREAM) [39]. The details of Step 1 are described in Section 3.1.3. In Step 2, the NMI data were analyzed in-depth, and environmental components related to the critical factors extracted in Step 1 were identified. The details of Step 2 are described in Section 3.1.4.
A human driving database and a driver error analysis method were utilized to identify critical factors for ADS, which is the originality of this study. Configurations and specifications of ADSs vary between systems. Thus, it is difficult to set critical factors from the system perspective. The various ADSs and human drivers will exist inside the same mixed traffic and share the same driving situations. ADSs and human drivers will not react identically to situations, and ADSs are considered superior to human drivers. However, ADSs and human drivers share similar limitations, such as observing the surroundings and controlling only the vehicle. Therefore, human driving data were adopted in the proposed method. Due to this setting, this proposed method cannot identify the most critical scenario components for ADS. Still, it can identify critical factors expected to be handled by ADSs that are expected to be superior to human drivers.

3.1.2. Driving Database

In recent years, data on daily driving behavior have been collected on a large scale by many naturalistic driving studies (NDSs) [40,41,42]. In an NDS, vehicle signals, such as speed and acceleration, and images from cameras that show the front view of the vehicle and driver are constantly recorded. Therefore, when a driver causes an accident or in the case of a near-miss incident, it is possible to understand the behavior of the driver and surrounding environment leading to the event. However, although the constantly recorded data collected in the NDS contain long-term records of daily driving, accidents and near-miss incidents are rare.
In contrast to databases of constantly recorded data in NDSs, some databases contain driving data recorded in response to events, such as emergency braking and steering using an event recorder. Unlike the database collected in an NDS, this database contains many accidents and near-miss incidents. Similarly to the data recorded in the NDS, the vehicle signals and camera images are recorded. However, the data are recorded only for a limited time before and after the event. Both databases have advantages and disadvantages in identifying critical scenarios. In this study, the latter type was adopted to understand critical scenario components from accident and near-miss incident data.
The near-miss incident database managed by the Smart Mobility Research Center at TUAT was adopted as the driving database collected with an event recorder. This database contains accident and near-miss incident data of taxis recorded by an event-type drive recorder. Recording starts automatically when the deceleration value in the front/rear direction exceeds 0.45 G. Data from 10 s before and 5 s after the start trigger are recorded. The database contains vehicle information, such as speed and acceleration, and camera images of the vehicle’s front view and driver’s face. In addition, experienced evaluators annotated the data with information about the driver who caused the accident/near-miss incident and the surrounding environment. A near-miss incident is not an accident, but a case in which a vehicle almost collided with another traffic participant. Thus, the critical factors leading to accidents can be observed. Previous studies have shown that the ratio of accident types collected in the database and traffic accidents in Japan are identical, and the near-miss incident data in the database are effective for factor analysis of traffic accidents [43,44].

3.1.3. Extraction of Critical Factors Using DREAM (Step 1)

DREAM is a factor analysis method developed in a Swedish research project to understand the causes of traffic accidents, tracing back the factors involved in the accident from its occurrence. This method considers the parallelism of human cognitive information processing and organizes the relationships among critical events (phenotypes) and contributing factors (genotypes) as a chain. Phenotypes are observable phenomena resulting from traffic adaptation failures that lead to accidents or near-miss incidents. Phenotypes are expressed in physical dimensions, such as time and space. Phenotypes can be either general or specific (Table 1); when detailed information is available, a specific phenotype is used. Genotypes describe the factors that bring about the phenotypes (i.e., critical events), which are identified based on data analysis. Genotypes include drivers, vehicles, traffic environments, and organizations (Table 2). The manual for DREAM [45] provides pre-defined lists of phenotypes and genotypes and pre-defined links between genotypes and phenotypes or other genotypes. Analysts choose phenotypes and genotypes that correspond to a certain accident case considering pre-defined items to analyze the causation of the accident. The results of the analysis of each case can be integrated owing to the features of the pre-defined items (phenotypes and genotypes) used in this method. The integrated results enable us to understand accident causation, which is described as a chain of contributing factors that lead to an accident.
In this study, DREAM was mainly used to analyze near-miss incidents. The near-miss incidents of a target driving scenario are manually extracted from the database by analysts. Each extracted incident is analyzed according to the DREAM 3.0 manual [45] to describe the relationships between the phenotypes and genotypes. Frequent genotypes in the data are extracted in this step as critical factors.

3.1.4. Identification of Critical Scenario Components (Step 2)

Environmental components related to the critical factors extracted in Step 1 are identified by in-depth analysis of NMI data with the extracted critical factors. Details of the critical factors are investigated, and environmental components contributing to the factors are extracted from each NMI data. Extraction results for the components are aggregated in the NMI data with the critical factors. Frequent environmental components are identified in this step as critical scenario components in a target driving scenario.

3.2. Validation of Proposed Method

3.2.1. Overview

The purpose of the proposed method is to enable identification of common and critical test scenarios for the safety assessment of ADSs in a certain target driving scenario. The commonness of the identified scenario components can be evaluated by analyzing their frequency in naturalistic driving (ND) data. This frequency indicates the possibility of the identified components being encountered in daily driving. The criticality of the identified components can be evaluated by comparing the ratio of their frequency in NMI data with that in ND data. The NMI data are more critical driving data compared to ND data. Thus, it is assumed that if the identified components are critical, their frequency in NMI data will be higher than that in ND data.
The proposed method is applied, and critical scenario components are identified in a scenario where the ego vehicle makes a right turn at a signalized intersection as a case study. The details of this target driving scenario are described in Section 3.2.2. The validity of the proposed method is examined by the aforementioned methodology, analyzing ND data and comparing them with NMI data. ND data on the target driving scenario are extracted from an ND database. The details of the ND database are described in Section 3.2.3.

3.2.2. Target Driving Scenario

This study focuses on a scenario in which an ego vehicle, in left-hand traffic, makes a right turn at a signalized intersection while another vehicle approaches the intersection (Figure 2). The accident in this scenario is a collision between a right-turning ego vehicle and an oncoming vehicle.
In right turns (and left turns in right-hand traffic), drivers often make safety confirmation errors while driving [46], and the workload of drivers is higher than that in other scenarios [47]. In other words, the target scenario is one in which a driver error is more likely to occur. In addition, this scenario has been identified as a critical scenario in studies identifying the scenarios based on accident data [32,33,35] and the driving data of an NDS [37].
Furthermore, it is difficult for ADSs to plan their behavior in scenarios involving intersections, including right turns. This difficulty results from the presence of multiple vehicles, pedestrians, and other traffic participants. Moreover, the system must predict its behavior based on inaccurate measurements [48,49].

3.2.3. Naturalistic Driving Database

The ND database (ND-DB) of JARI was adopted as the driving database collected in an NDS. This database was collected during a project commissioned by the Ministry of Economy, Trade, and Industry in Japan. This database contains ND data for drivers who drive commercial vehicles daily in various traffic environments. This database contains camera images of the surrounding environment and the driver’s face as well as the vehicle’s position and speed. Thirty data-collection vehicles equipped with cameras and recording devices were manufactured. Approximately 100 male employees from cooperating companies, aged 20–60 years, were asked to use the vehicles for their business activities to collect driving data.

4. Results

This section first describes the results of the proposed method applied to the target driving scenario, right turns at signalized intersections (Section 4.1 and Section 4.2), as a case study. Next, this section describes the validation results for the proposed method (Section 4.3).

4.1. Extraction of Critical Factors

From the near-miss incident database, data points on 150 incidents corresponding to the target scenario were extracted. The phenotypes and genotypes were analyzed according to the analysis procedure described in Section 3.1.3. Figure 3 shows the relationship between the phenotypes and genotypes of all of the extracted incidents. The phenotype “A1.1 Too early action” accounted for approximately 95%. As for the highest-level genotypes, “C2 Misjudgment of situation” accounted for approximately 97%. In the lower-level genotypes, “K1 Temporary obstruction of view” and “F2 Expectance of certain behaviours” accounted for more than one-third of the extracted incidents. By analyzing near-miss incident data for right turns at signalized intersections using DREAM, we extracted frequent genotypes of K1 and F2.
The definition of genotype K1 is “Objects (e.g., driven or parked vehicles, gatherings of people) in the traffic environment cause temporary obstruction of view” [45]. Figure 4 shows a sample of the near-miss incident classified as K1. When the vehicle stopped at the intersection, waiting for oncoming vehicles to pass, there was an occlusion created by the vehicle proceeding from the opposite direction to turn right. The driver could not see the lane where oncoming vehicles were moving. The driver started to turn right even though he could not see through the lane. An oncoming vehicle appeared from the occlusion area, and the driver applied emergency braking to avoid a collision.
The definition of genotype F2 is “Expecting other road users to behave in certain ways following praxis (e.g., brake gently, stop for stop signs and red-lights, give way when driving on a non-priority or minor road and comply with the speed limits)” [45]. Figure 5 shows a sample of a near-miss incident classified as F2. When the vehicle stopped at the intersection waiting for oncoming vehicles to pass, the traffic light changed from green to yellow and red with a green arrow. The driver saw this light change and began to turn right, expecting the oncoming vehicle to stop at the red light. However, the oncoming vehicle did not stop, and the driver applied emergency braking to avoid a collision.

4.2. Identification of Critical Scenario Components

As for NMI data points in which the genotype was K1, objects that caused a temporary obstruction of view were investigated as critical scenario components. Table 3 shows the investigation results for the occlusion-causing objects. Oncoming vehicles coming from the opposite direction and preceding vehicles running in front of the ego vehicle caused most of the temporary occlusion. Among the objects, oncoming right-turning vehicles accounted for approximately 67% of the objects that obstructed the driver’s view and dynamically created an occlusion.
In the case of NMI data points with the genotype F2, the expectation of drivers was first investigated. Table 4 shows the investigation results for the driver’s expectation in the right-turn scenario. Most of the expectations considered the driving behavior of the oncoming vehicle. Drivers often expected that oncoming vehicles would not enter the intersection under yellow or red lights, leading to errors. Thus, when the vehicle turns right at the intersection, the change in traffic lights is considered as an environmental factor leading to the expectation of certain behaviors.
Analysis results identified two critical scenario components related to genotypes K1 and F2, respectively. The first environmental component was dynamic occlusions caused by oncoming right-turning vehicles. The second component was changing traffic lights. These scenario components were identified explicitly through analysis of driving data, as shown above.

4.3. Validation of Proposed Method

Three hundred right-turn data points at signalized intersections were extracted from the ND-DB. For each data point, dynamic occlusions caused by oncoming right-turning vehicles and changes in traffic lights were analyzed. Right turns, where an oncoming right-turning vehicle is present without an occlusion, and right turns, where no oncoming straight-ahead vehicle is approaching the intersection when the lights changed, were excluded. Figure 6 compares the right-turn ratio with the critical factors between the NMI and ND data points. In the ND data, both factors were encountered in more than 10% of the total data points (light gray bars in Figure 6). This result indicates that these factors can be encountered daily and are common scenario components. In contrast with the ratio of ND data points, both of the factors accounted for more than 25% of the NMI data points (dark gray bars in Figure 6). Statistical test results clarified that the right-turn ratio of NMI data points was significantly higher than that of ND data points (p < 0.001). These results suggest that driver errors are more likely to occur in scenarios where these two factors are present and the identified scenario components are critical.

5. Discussion

5.1. Usefulness of the Identified Critical Scenario Components

Two critical-scenario components were identified using the proposed method: dynamic occlusion caused by oncoming right-turning vehicles and change in traffic lights when an oncoming vehicle is approaching the intersection. The usefulness of these critical components was examined in terms of the safety assessment of ADSs. ADSs vary widely in their configuration and functionality, depending on the entity developing them. Therefore, it is difficult to evaluate the validity of the identified components and the method proposed in this study focusing on a specific ADS. Therefore, we roughly classified ADSs into two types, an autonomous system and a cooperative system, and examined the safety assessment of ADSs between these two systems in a scenario with the identified components. The former is a system that drives automatically based on the information obtained from the sensors installed on the ego vehicle. The latter utilizes the information obtained from other sources (e.g., other vehicles and infrastructure) using communication technology and the information used by an autonomous-type system. The usefulness of the components was examined by comparing the system behavior of these two types of ADS in the scenarios with the identified components. The following sections present the results for each component.

5.1.1. Dynamic Occlusion

The autonomous-type system uses sensors, such as LiDAR devices, and cameras installed on the ego vehicle to detect objects. Thus, it can only detect objects that can be directly observed by the ego vehicle. Therefore, if an oncoming vehicle enters the occluded area formed by oncoming right-turning vehicles, it cannot be detected. Suppose the system decides to turn based on the acquired information, even though an oncoming vehicle is present. In this case, that action may result in a collision with the oncoming vehicle. In contrast, information on the oncoming vehicle can be shared with a cooperative-type system by the oncoming vehicle itself or other vehicles and infrastructure, thereby avoiding incorrect decisions that could lead to a collision. In summary, the scenario with dynamic occlusion caused by oncoming right-turning vehicles is critical for an autonomous-type system because it requires planning that considers unobservable objects hidden in the occluded area.

5.1.2. Change in Traffic Lights

In an autonomous-type system, an oncoming vehicle approaching the intersection and the status of the traffic lights can be detected through the sensors mounted on the ego vehicle. The system is expected to plan a right turn at an intersection based on the traffic rules and detection results, which change with time. For example, in a situation where the traffic light for the oncoming vehicle’s lane is red and the ego vehicle’s lane is green, the ego vehicle is expected to turn right because the oncoming vehicle should not enter the intersection according to traffic rules. However, suppose that an oncoming vehicle ignores the red light and enters the intersection. In this case, a collision may occur because the ego vehicle’s planner does not consider the decision error of the oncoming vehicle. The autonomous-type system can detect an oncoming vehicle and predict the possibility that the oncoming vehicle will enter the intersection, even with a red light, and avoid a collision with it. However, suppose that the ego vehicle waits for the oncoming vehicle to pass. In this case, the phase of the traffic lights might change, and the ego vehicle might be left inside the intersection, possibly blocking the traffic in the intersecting lane. In a cooperative-type system, unlike an autonomous-type system, it is possible to obtain information about the oncoming vehicle and the timing of light changes from others. However, although the information used in the system is different, planning under the traffic rules while adapting to the situation changing with time is the same as in the autonomous-type system, and the cooperative-type system will face the same issue. As described earlier, this scenario with a change in traffic lights is critical for ADSs because it requires planning that adapts to situations where multiple factors (e.g., approaching oncoming vehicle and traffic light status) change with time.
The examinations conducted show that each identified scenario component presents challenges for the ADS, and it may end up in a collision if it cannot overcome these challenges. Therefore, the usefulness of the identified critical scenario components was confirmed. Furthermore, the proposed method used to identify these scenario components based on driver error analyses is valid.

5.2. Contribution of the Proposed Method

Two critical scenario components were extracted using the proposed method: dynamic occlusions and changes in traffic lights. Other studies have also highlighted the importance of considering dynamic occlusion in traffic scenarios [30,33], which indicates that the proposed method can extract reasonable factors. In [30], the authors mentioned dynamic occlusion as a factor that will make the scenario more challenging and explained the mechanism to express dynamic occlusion in a scenario. In [33], the authors claimed that the accident data used to identify the scenarios lacked information about the obstruction in the visual field owing to obstacles, including dynamic occlusions. Both highlight the importance of considering dynamic occlusions based on the knowledge that occlusions lead to accidents. However, the selection of factors based on prior knowledge may not be feasible in unknown or complex scenarios. Compared with the others, the results of the proposed method show the importance of considering this factor with evidence. Therefore, it is suggested that the proposed method can contribute to explicitly extracting critical scenario components in urban environments if driving data on near-miss incidents or accidents can be obtained.
The problems faced by autonomous-type ADSs are similar to those faced by a human driver. ADSs are superior to human drivers in terms of object detection range, accuracy, and parallel processing power, and they can achieve higher driving performance. However, because autonomous-type systems and drivers are similar in observing the surroundings around the vehicle and controlling only the vehicle itself, they share the same limitations. Information from other sources is necessary to solve the problems caused by these limitations. Thus, cooperative-type systems, which can obtain information from other sources (e.g., surrounding vehicles and infrastructure), are essential for implementing safe ADSs. The methodology proposed in this study, which utilizes driving databases of human drivers and extracts critical environmental factors based on database analyses, is considered useful for establishing the requirements for cooperative-type systems.

5.3. Limitations

This study used NMI and ND data collected from Japanese drivers in Japan. Therefore, the scenario components identified by the proposed method may reflect the characteristics of the traffic conditions and environment in Japan, as well as the driving characteristics of Japanese people. Thus, one limitation is that such components may not be directly applicable to other countries or regions. However, by applying the proposed method to the driving databases of different countries or regions, the critical environmental factors that must be considered in each country or region can be identified.
Although the objective was to identify critical scenario components for ADSs, the method proposed in this study utilized a driving database driven by human drivers, not ADSs. We adopted this database because of the lack of accident and near-miss incident data on ADSs in urban environments. Therefore, not all of the identified components may be critical for ADSs. However, it can be assumed that there is a certain overlap between the behavior of human drivers and ADSs, and the proposed method can be effective to a certain extent, as shown in this paper.
Another limitation is that the scenario was limited to one in which an ego vehicle makes a right turn at a signalized intersection. Therefore, future research should apply the methodology used in this study to other scenarios and examine its usefulness.

6. Conclusions

It is essential to understand the environmental factors that lead to safety-critical events to identify the critical scenarios for scenario-based assessments of ADSs in urban environments. This study proposes a method to identify the critical scenario components by extracting environmental factors that trigger safety-critical events based on driver error analysis. Using a database of near-miss incidents, we extracted critical environmental factors for a scenario in which an ego vehicle turns right at a signalized intersection. The findings of this study are as follows:
  • “Dynamic occlusion caused by oncoming right-turn vehicles” and “change in traffic lights” are critical scenario components in the right-turn scenario at signalized intersections, and the proposed method can explicitly identify critical scenario components.
  • The critical scenario components identified using the proposed method had challenging points for ADSs, demonstrating the usefulness of the identified components and indicating the validity of the proposed methodology.
Identifying critical scenario components with the proposed method will enable the labeling of critical scenarios, leading to efficient safety assessments of ADSs by prioritizing the innumerable test scenarios. Moreover, the critical scenario components identified by the proposed method have the potential to set requirements for cooperative-type ADSs used in urban environments.
This study was limited to the particular scenario of a right turn at a signalized intersection. Therefore, future plans include applying the same method to other driving scenarios to identify the critical scenario components for ADSs and to validate the proposed method further. Moreover, the proposed methodology can contribute to understanding the problems of autonomous-type systems and setting the requirements for cooperative-type systems. Therefore, we plan to use this methodology to investigate the advancement of ADSs.

Author Contributions

Conceptualization, H.Y. and M.S.; methodology, H.Y. and M.S.; validation, H.Y. and M.S.; formal analysis, H.Y.; investigation, H.Y.; resources, H.Y.; data curation, H.Y.; writing—original draft preparation, H.Y.; writing—review and editing, H.Y. and M.S.; visualization, H.Y.; supervision, M.S.; project administration, M.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Restrictions apply to the availability of these data. The near-miss incident database was obtained from the Smart Mobility Research Center of the Tokyo University of Agriculture and Technology and the database can be purchased from them. The naturalistic driving database was obtained from the Japan Automobile Research Institute and the database is only available for restricted organizations due to confidentiality concerns.

Acknowledgments

The authors would like to express gratitude to the members of UL Systems, Inc., which has been engaged in research activities regarding the safety assessment of ADSs for many years, for their advice on the appropriateness of the method used in this paper. The authors are also grateful to the Automobile Division, Manufacturing Industries Bureau, Ministry of Economy, Trade, and Industry of Japan, and the Japan Automobile Research Institute for their cooperation in using the JARI naturalistic driving database, and the Smart Mobility Research Center of the Tokyo University of Agriculture and Technology for their cooperation in using the TUAT near-miss incident database.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. World Health Organization. Global Status Report on Road Safety 2018; WHO: Geneva, Switzerland, 2018. [Google Scholar]
  2. SAE International. J3016_202104: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Available online: https://www.sae.org/standards/content/j3016_202104/ (accessed on 31 July 2023).
  3. Amersbach, C.; Winner, H. Defining required and feasible test coverage for scenario-based validation of highly automated vehicles. In Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 27–30 October 2019; pp. 425–430. [Google Scholar]
  4. Wachenfeld, W.; Winner, H. The release of autonomous vehicles. In Autonomous Driving: Technical, Legal, and Social Aspects; Maurer, M., Gerdes, J.C., Lenz, B., Winner, H., Eds.; Springer: Berlin, Germany, 2016; pp. 425–449. [Google Scholar]
  5. Kalra, N.; Paddock, S.M. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp. Res. Part A Policy Pract. 2016, 94, 182–193. [Google Scholar] [CrossRef]
  6. Pütz, A.; Zlocki, A.; Küfen, J.; Bock, J.; Eckstein, L. Database approach for the sign-off process of highly automated vehicles. In Proceedings of the 25th International Technical Conference on the Enhanced Safety of Vehicles, Detroit, MI, USA, 5–8 June 2017; pp. 1–8. [Google Scholar]
  7. Saigol, Z.; Peters, A. Verifying automated driving systems in simulation: Framework and challenges. In Proceedings of the 25th ITS World Congress, Copenhagen, Denmark, 17–21 September 2018; pp. 17–21. [Google Scholar]
  8. Elrofai, H.; Paardekooper, J.P.; de Gelder, E.; Kalisvaart, S.; den Camp, O.O. Scenario-Based Safety Validation of Connected and Automated Driving; Technical Report; Netherlands Organization for Applied Scientific Research, TNO: Helmond, The Netherlands, 2018. [Google Scholar]
  9. Antona-Makoshi, J.; Uchida, N.; Yamazaki, K.; Ozawa, K.; Kitahara, E.; Taniguchi, S. Development of a safety assurance process for autonomous vehicles in Japan. In Proceedings of the 26th International Technical Conference on the Enhanced Safety of Vehicles, Eindhoven, The Netherlands, 10–13 June 2019; pp. 1–18. [Google Scholar]
  10. Junietz, P.; Wachenfeld, W.; Klonecki, K.; Winner, H. Evaluation of different approaches to address safety validation of automated driving. In Proceedings of the 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA, 4–7 November 2018; pp. 491–496. [Google Scholar]
  11. Amersbach, C.; Winner, H. Functional decomposition—A contribution to overcome the parameter space explosion during validation of highly automated driving. Traffic Inj. Prev. 2019, 20 (Suppl. S1), S52–S57. [Google Scholar] [CrossRef]
  12. Menzel, T.; Bagschik, G.; Maurer, M. Scenarios for development, test and validation of automated vehicles. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 1821–1827. [Google Scholar]
  13. Bagschik, G.; Menzel, T.; Maurer, M. Ontology based scene creation for the development of automated vehicles. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 1813–1820. [Google Scholar]
  14. Nitsche, P.; Mocanu, I.; Reinthaler, M. Requirements on tomorrow’s road infrastructure for highly automated driving. In Proceedings of the International Conference on Connected Vehicles and Expo (ICCVE), Vienna, Austria, 3–7 November 2014; pp. 939–940. [Google Scholar]
  15. Ponn, T.; Breitfuß, M.; Yu, X.; Diermeyer, F. Identification of challenging highway-scenarios for the safety validation of automated vehicles based on real driving data. In Proceedings of the 15th International Conference on Ecological Vehicles and Renewable Energies, Monte-Carlo, Monaco, 10–12 September 2020; pp. 1–10. [Google Scholar]
  16. Paardekooper, J.P.; van Montfort, S.; Manders, J.; Goos, J.; de Gelder, E.; den Camp, O.O.; Bracquemond, A.; Thiolon, G. Automatic identification of critical scenarios in a public dataset of 6000 km of public-road driving. In Proceedings of the 26th International Technical Conference on the Enhanced Safety of Vehicles, Eindhoven, The Netherlands, 10–13 June 2019; pp. 1–8. [Google Scholar]
  17. Akagi, Y.; Kato, R.; Kitajima, S.; Antona-Makoshi, J.; Uchida, N. A risk-index based sampling method to generate scenarios for the evaluation of automated driving vehicle safety. In Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 27–30 October 2019; pp. 667–672. [Google Scholar]
  18. Klischat, M.; Althoff, M. Generating critical test scenarios for automated vehicles with evolutionary algorithms. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Paris, France, 9–12 June 2019; pp. 2352–2358. [Google Scholar]
  19. Zhang, P.; Zhu, B.; Zhao, J.; Fan, T.; Sun, Y. Performance evaluation method for automated driving system in logical scenario. Automot. Innov. 2022, 5, 299–310. [Google Scholar] [CrossRef]
  20. Nalic, D.; Mihalj, T.; Orucevic, F.; Schabauer, M.; Lex, C.; Sinz, W.; Eichberger, A. Criticality assessment method for automated driving systems by introducing fictive vehicles and variable criticality thresholds. Sensors 2022, 22, 8780. [Google Scholar] [CrossRef]
  21. Stepien, L.; Thal, S.; Henze, R.; Nakamura, H.; Antona-Makoshi, J.; Uchida, N.; Raksincharoensak, P. Applying heuristics to generate test cases for automated driving safety evaluation. Appl. Sci. 2021, 11, 10166. [Google Scholar] [CrossRef]
  22. Thal, S.; Wallis, P.; Henze, R.; Hasegawa, R.; Nakamura, H.; Kitajima, S.; Abe, G. Towards Realistic, Safety-Critical and Complete Test Case Catalogs for Safe Automated Driving in Urban Scenarios. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Anchorage, AK, USA, 4—7 June 2023; pp. 1–8. [Google Scholar]
  23. Khastgir, S.; Brewerton, S.; Thomas, J.; Jennings, P. Systems approach to creating test scenarios for automated driving systems. Reliab. Eng. Syst. Saf. 2021, 215, 107610. [Google Scholar] [CrossRef]
  24. Weber, N.; Frerichs, D.; Eberle, U. A simulation-based, statistical approach for the derivation of concrete scenarios for the release of highly automated driving functions. In Proceedings of the AmE 2020—Automotive Meets Electronics; 11th GMM-Symposium, Dortmund, Germany, 10–11 March 2020; pp. 1–6. [Google Scholar]
  25. Nonnengart, A.; Klusch, M.; Müller, C. CriSGen: Constraint-based generation of critical scenarios for autonomous vehicles. In Formal Methods. FM 2019 International Workshops. FM 2019. (Lecture Notes in Computer Science); Springer: Cham, Switzerland, 2020; Volume 12232, pp. 233–248. [Google Scholar]
  26. Xinxin, Z.; Fei, L.; Xiangbin, W. CSG: Critical scenario generation from real traffic accidents. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 23–26 June 2020; pp. 1330–1336. [Google Scholar]
  27. Kondoh, T.; Yamamura, T.; Kitazaki, S.; Kuge, N.; Boer, E.R. Identification of visual cues and quantification of drivers’ perception of proximity risk to the lead vehicle in car-following situations. J. Mech. Syst. Transp. Logist. 2008, 1, 170–180. [Google Scholar] [CrossRef]
  28. Kramer, B.; Neurohr, C.; Büker, M.; Böde, E.; Fränzle, M.; Damm, W. Identification and quantification of hazardous scenarios for automated driving. In Model-Based Safety and Assessment (Lecture Notes in Computer Science); Zeller, M., Höfig, K., Eds.; Springer: Cham, Switzerland, 2020; Volume 12297, pp. 163–178. [Google Scholar]
  29. Ponn, T.; Diermeyer, F.; Gnandt, C. An optimization-based method to identify relevant scenarios for type approval of automated vehicles. In Proceedings of the 26th International Technical Conference on the Enhanced Safety of Vehicles, Eindhoven, The Netherlands, 10–13 June 2019; pp. 10–13. [Google Scholar]
  30. Weber, H.; Bock, J.; Klimke, J.; Roesener, C.; Hiller, J.; Krajewski, R.; Zlocki, A.; Eckstein, L. A framework for definition of logical scenarios for safety assurance of automated driving. Traffic Inj. Prev. 2019, 20 (Suppl. S1), S65–S70. [Google Scholar] [CrossRef]
  31. de Gelder, E.; Hof, J.; Cator, E.; Paardekooper, J.P.; den Camp, O.O.; Ploeg, J.; De Schutter, B. Scenario parameter generation method and scenario representativeness metric for scenario-based assessment of automated vehicles. IEEE Trans. Intell. Transp. Syst. 2022, 23, 18794–18807. [Google Scholar] [CrossRef]
  32. Nitsche, P.; Thomas, P.; Stuetz, R.; Welsh, R. Pre-crash scenarios at road junctions: A clustering method for car crash data. Accid. Anal. Prev. 2017, 107, 137–151. [Google Scholar] [CrossRef] [PubMed]
  33. Hu, W.; Xu, X.; Zhou, Z.; Liu, Y.; Wang, Y.; Xiao, L.; Qian, X. Mining and comparative analysis of typical pre-crash scenarios from IGLAD. Accid. Anal. Prev. 2020, 145, 105699. [Google Scholar] [CrossRef]
  34. Kang, M.; Lee, W.; Hwang, K.; Yoon, Y. Vision transformer for detecting critical situations and extracting functional scenario for automated vehicle safety assessment. Sustainability 2022, 14, 9680. [Google Scholar] [CrossRef]
  35. So, J.J.; Park, I.; Wee, J.; Park, S.; Yun, I. Generating traffic safety test scenarios for automated vehicles using a big data technique. KSCE J. Civ. Eng. 2019, 23, 2702–2712. [Google Scholar] [CrossRef]
  36. Gambi, A.; Huynh, T.; Fraser, G. Generating effective test cases for self-driving cars from police reports. In Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia, 26–30 August 2019; pp. 257–267. [Google Scholar]
  37. Kim, Y.; Tak, S.; Kim, J.; Yeo, H. Identifying major accident scenarios in intersection and evaluation of collision warning system. In Proceedings of the IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, 16–19 October 2017; pp. 1–6. [Google Scholar]
  38. Yue, B.; Shi, S.; Wang, S.; Lin, N. Low-cost urban test scenario generation using microscopic traffic simulation. IEEE Access 2020, 8, 123398–123407. [Google Scholar] [CrossRef]
  39. Habibovic, A.; Tivesten, E.; Uchida, N.; Bärgman, J.; Aust, M.L. Driver behavior in car-to-pedestrian incidents: An application of the Driving Reliability and Error Analysis Method (DREAM). Accid. Anal. Prev. 2013, 50, 554–565. [Google Scholar] [CrossRef]
  40. Dingus, T.A.; Klauer, S.G.; Neale, V.L.; Petersen, A.; Lee, S.E.; Sudweeks, J.; Perez, M.A.; Hankey, J.; Ramsey, D.; Gupta, S.; et al. The 100-Car Naturalistic Driving Study, Phase II—Results of the 100-Car Field Experiment; National Highway Traffic Safety Administration: Washington, DC, USA, 2006; DOT HS 810 593.
  41. Benmimoun, M.; Pütz, A.; Zlocki, A.; Eckstein, L. eurofot: Field operational test and impact assessment of advanced driver assistance systems: Final results. In Proceedings of the FISITA 2012 World Automotive Congress, Beijing, China, 27–30 November 2012; pp. 537–547. [Google Scholar]
  42. Blatt, A.; Pierowicz, J.; Flanigan, M.; Lin, P.S.; Kourtellis, A.; Lee, C.; Jovanis, P.; Jenness, J.; Wilaby, M.; Campbell, J. Naturalistic Driving Study: Field Data Collection; SHRP 2 Rep. S2-S07-RW-1; Transportation Research Board of the National Academies: Washington, DC, USA, 2015. [Google Scholar]
  43. Fujita, M.; Michitsuji, Y.; Shino, M.; Kamata, M.; Nagai, M. Research on near-miss incident analysis using drive recorders (Second report)—Analysis methodology by collected data and database construction. Trans. JSAE 2007, 38, 145–150. [Google Scholar]
  44. Nagai, M. Present status of drive recorder database and its application potential. J. Soc. Automot. Eng. 2013, 67, 47–53. [Google Scholar]
  45. Warner, H.W.; Aust, M.L.; Sandin, J.; Johansson, E.; Björklund, G. Manual for DREAM 3.0, Driving Reliability and Error Analysis Method, Deliverable D5.6 of the EU FP6 Project SafetyNet, TREN-04-FP6TRSI2.395465/506723, 2008. Available online: https://publications.lib.chalmers.se/records/fulltext/80432.pdf (accessed on 31 July 2023).
  46. Gstalter, H.; Fastenmeier, W. Reliability of drivers in urban intersections. Accid. Anal. Prev. 2010, 42, 225–234. [Google Scholar] [CrossRef] [PubMed]
  47. Hancock, P.A.; Wulf, G.; Thom, D.; Fassnacht, P. Driver workload during differing driving maneuvers. Accid. Anal. Prev. 1990, 22, 281–290. [Google Scholar] [CrossRef] [PubMed]
  48. Hubmann, C.; Becker, M.; Althoff, D.; Lenz, D.; Stiller, C. Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA, 11–14 June 2017; pp. 1671–1678. [Google Scholar]
  49. Shu, K.; Yu, H.; Chen, X.; Chen, L.; Wang, Q.; Li, L.; Cao, D. Autonomous driving at intersections: A critical-turning-point approach for left turns. In Proceedings of the IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece, 20–23 September 2020; pp. 1–6. [Google Scholar]
Applsci 13 10770 g001
Figure 1. Framework of the proposed method for identification of critical scenario components using a driving database and a driver error analysis method.
Figure 1. Framework of the proposed method for identification of critical scenario components using a driving database and a driver error analysis method.
Applsci 13 10770 g001
Applsci 13 10770 g002
Figure 2. Target driving scenario (an ego vehicle making a right turn at a signalized intersection with an oncoming vehicle approaching the intersection in left-hand traffic).
Figure 2. Target driving scenario (an ego vehicle making a right turn at a signalized intersection with an oncoming vehicle approaching the intersection in left-hand traffic).
Applsci 13 10770 g002
Applsci 13 10770 g003
Figure 3. Relationship between phenotypes and genotypes in near-miss incidents with oncoming vehicles when making a right turn at a signalized intersection (N = 150).
Figure 3. Relationship between phenotypes and genotypes in near-miss incidents with oncoming vehicles when making a right turn at a signalized intersection (N = 150).
Applsci 13 10770 g003
Applsci 13 10770 g004
Figure 4. Sample near-miss incident classified as “K1 Temporary obstruction of view” (ID: 659524).
Figure 4. Sample near-miss incident classified as “K1 Temporary obstruction of view” (ID: 659524).
Applsci 13 10770 g004
Applsci 13 10770 g005
Figure 5. Sample near-miss incident classified as “F2 Expectance of certain behaviours” (ID: 661582).
Figure 5. Sample near-miss incident classified as “F2 Expectance of certain behaviours” (ID: 661582).
Applsci 13 10770 g005
Applsci 13 10770 g006
Figure 6. Component ratio of right turns with the identified scenario components among near-miss incidents and naturalistic driving data points.
Figure 6. Component ratio of right turns with the identified scenario components among near-miss incidents and naturalistic driving data points.
Applsci 13 10770 g006
Table 1. General and specific critical events (phenotypes) [45].
Table 1. General and specific critical events (phenotypes) [45].
General Critical EventsSpecific Critical Events
TimingToo early action; Too late action; No action
SpeedToo high speed; Too low speed
DistanceToo short distance
DirectionWrong direction
ForceSurplus force; Insufficient force
ObjectAdjacent object
Table 2. Three categories of contributing factors (genotypes) [45].
Table 2. Three categories of contributing factors (genotypes) [45].
DriverVehicle and EnvironmentOrganization
ObservationTemporary HMI problemsOrganization
InterpretationPermanent HMI problemsMaintenance
PlanningVehicle equipment failureVehicle design
Temporary personal factorsWeather conditionsRoad design
Permanent personal factorsObstruction of view due to object
State of road
Table 3. Occlusion-causing objects in NMI data with the genotype “K1 Temporary Obstruction of View” (N = 60).
Table 3. Occlusion-causing objects in NMI data with the genotype “K1 Temporary Obstruction of View” (N = 60).
No.ObjectFrequency
1Oncoming vehicle making a right turn40
2Oncoming vehicle making a left turn6
3Oncoming vehicle going straight5
4Preceding vehicle going straight4
5Preceding vehicle making a right turn4
6Crossing vehicle1
Table 4. Driver’s expectations in incident data points of genotype “F2 Expectance of Certain Behaviours” (N = 49).
Table 4. Driver’s expectations in incident data points of genotype “F2 Expectance of Certain Behaviours” (N = 49).
No.Driver’s ExpectationsFrequency
1An oncoming vehicle going straight will not enter the intersection at a red light.25
2An oncoming vehicle going straight will not enter the intersection at a yellow light.17
3An oncoming vehicle going straight will pass through the intersection without slowing down.2
4An oncoming vehicle will not come straight.2
5An oncoming vehicle will not change its direction.1
6An oncoming vehicle will run a different path.1
7A stopping vehicle will not enter the intersection.1
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Yoshitake, H.; Shino, M. Identification of Critical Scenario Components Based on Driving Database Analysis for Safety Assessment of Automated Driving Systems. Appl. Sci. 2023, 13, 10770. https://doi.org/10.3390/app131910770

AMA Style

Yoshitake H, Shino M. Identification of Critical Scenario Components Based on Driving Database Analysis for Safety Assessment of Automated Driving Systems. Applied Sciences. 2023; 13(19):10770. https://doi.org/10.3390/app131910770

Chicago/Turabian Style

Yoshitake, Hiroshi, and Motoki Shino. 2023. "Identification of Critical Scenario Components Based on Driving Database Analysis for Safety Assessment of Automated Driving Systems" Applied Sciences 13, no. 19: 10770. https://doi.org/10.3390/app131910770

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop