Synthesis of Autonomous Vehicle Guideline for Public Road-Testing Sustainability

Abstract

Autonomous vehicles have the potential to reduce the risk of accidents as they eliminate the element of human error from driving. Lack of attention, poor judgement, or physical limitations may lead to road incidents. Thus, the development and deployment of autonomous vehicles should be a priority. However, before being publicly available, autonomous vehicles must be tested to ensure their viability and safety by conducting public road testing. Autonomous vehicles have been designed and tested since the early 1900s; however, deployment of fully autonomous vehicles on public roads only started in the 2000s. Numerous countries have developed guidelines for public road testing, but those rules are not uniform, and discrepancies occur between nations. Issues such as vehicular safety, registrations, authority, insurance, cybersecurity, and infrastructures weigh differently in each country. Synthesizing these diverse national regulations into global guidelines would promote the safety and sustainability of autonomous vehicle testing and benefit all parties interested in autonomous vehicles.

1. Introduction

Since the first appearance of road vehicles, automation functionality has continuously been developing and evolving. The use of autonomous vehicles (AVs) in road transport is increasing because of their contribution to road safety and the potential to lower the number of casualties due to the elimination of human error. Besides safety, AVs could also help reduce traffic congestion, increase productivity, and provide transportation accessibility. Nevertheless, automating the driving process is a complex task involving numerous challenges that need to be tackled to facilitate the widespread adoption of AVs. Apart from the technological issues, the psychological, policy, and regulatory challenges should be tackled simultaneously. Psychological concerns, which may be the main barrier to mass adoption of AVs, refer primarily to the trust issues and public concern for personal safety and security [1]. Additional studies in the public acceptance and willingness may identify the necessary measures that will contribute to the widespread adoption of AVs. Besides the availability, transport policy can help shape positive public opinion on AVs. Increasing the funding for initiatives involving education, marketing, advertising, compatibility, and process simplification can result in new, effective transportation policies that can stir the public sentiment towards widespread acceptance [2].

Additionally, adaptive regulations must be developed incorporating both the latest knowledge and safety requirements and technological advancement [3]. Under the new regulations, testing standards must address a wider range of real-world scenarios, such as the absence of connectivity. Currently, it is unclear whether an AV is capable of avoiding crashes without connectivity with surrounding vehicles (V2V) or infrastructure (I2V) [4]. Thus, reaching the goal of widespread use of AVs requires not only advancement in AV technology but also other AV-interlinking areas/technologies.

AV must be able to analyse the surrounding area and manoeuvre safely with minimum or no human interaction [5]. Referring to the on-road motor vehicles, six levels of automation have been defined by the Society of Automotive Engineers (SAE), published in SAE J3016. Figure 1 illustrates these six levels, starting from Level 0, which requires full human attention and input, to fully automated Level 5 [6]. Level 2 is the highest level of vehicle automation currently in production, although recently, Honda (Tokyo, Japan) became the first to produce a road vehicle equipped with Level 3 self-driving technology [7]. Levels 4 and 5 are yet to be achieved through testing, research, and development in the coming years.

As of 2021, most manufacturers involved with AVs are conducting tests on public roads. Findings and inputs from these tests can be valuable in revising and developing the infrastructures, regulations, and public opinion. Companies such as Waymo (Mountain View, CA, USA) are conducting testing in public areas permitted by the government [8]. Germany has started testing AVs up to Level 4 on public roads with speed limits of up to 130 km/h that require a human driver’s oversite [9]. Facilitating AV testing on the public road requires multigroup involvement, including manufacturers and testing organisations, government, various safety authorities, and the public.

With numerous countries undergoing AV public road testing, there must be some common ground where their regulations and guidelines overlap. Hence, to formulate a global and complete set of guidelines, a study of the relevance of individual regulations and their adaptability must be conducted. This is because public testing is required in order to allow AVs’ deployment in the real world. Since AVs are expected to have minimal or no driver’s input, validating their usage in the real world is critically important. Traditionally, test results act as indicators of a vehicle’s road safety; however, that is not the case with AVs where the safety assurance is acquired in real-mileage testing [10]. Deploying AVs in a simulated environment may be insufficient as it is still essentially conducted under controlled conditions. Thus, the question of concise and complete legislation enabling AV real-road testing arises.

Different countries have different strategies and component requirements for public road-testing guidelines. Some aspects considered important in one set of guidelines may not even be mentioned in another, potentially leading to safety hazards when testing is executed. Emphasizing safety is key in legislation and policy making and is crucial for conducting successful testing practices. In case of accidents, the testing organisation would be responsible for liabilities. Ideally, synthesising various policies would potentially result in a concise and complete set of guidelines acting as a global framework for countries developing AV testing guidelines. Hence, countries could draw upon and adapt these regulations based on their suitability within their own context. National authorities would benefit greatly from relying on this global standard as an underlying framework already in place, leaving only suitability issues to be considered. Resources could then be diverted to researching suitability and supporting infrastructural aspects of AV deployment. Testing companies could use this global tool as a preliminary reference prior to consulting the local authorities. It would also serve as a good basis to educate the community on AV public road testing.

For this study, an initial database search was conducted focusing on publicly available AV testing guidelines data in the English Language, excluding policy, regulation, procedure, and standard documents. The initial database search result was screened for guidelines published by government/agency/authority in public road-testing applications only. Considering the differences in policies and regulation between the countries and regions, this study finally selected Australia, Canada, and the United Kingdom for the review, representing Asia Pacific, North America, and Europe. The purpose of this research was to identify and highlight the regulatory differences in their AV testing guideline and to draft a synthesised framework that could be referred to and adapted by many countries.

2. Background

In the early 1900s, Norman Bel Geddes deployed the first self-driving car, an electric vehicle guided by radio-controlled electromagnetic fields generated with magnetized metal spikes embedded in the roadway, during the General Motors exhibition in 1939 [11]. Further advancements in AV were apparent in the 1950s when RCA Laboratories and the state of Nebraska tested an AV on a 122-m stretch of road with embedded metal wires [12]. In the 1960s, United Kingdom’s Transport and Road Research Laboratory and Citroen tested a Citroen Ds model, which drove autonomously 130 km/h using magnetic cables embedded in the road [13]. In the following decade, manufacturers started to include advanced technologies such as neural networks in AV control, the precursors of the technologies currently used in AV. The Prometheus project by the Daimler group used a Mercedes-Benz van as a test vehicle and managed to autonomously drive up to 63 km/h on traffic-less roads [14]. The Defense Advanced Research Projects Agency (DARPA) also tested an autonomous land vehicle using lidar, computer vision, and automated control [15].

Based on the previous development, the United States Congress approved the Intermodal Surface Transportation Efficiency Act of 1991 (ISTEA) that guides the United States Department of Transport to conduct AV system testing and highway road testing in 1997. This Act led to a demonstration of 20 AVs in San Diego, California [16]. In addition, Daimler-Benz successfully tested autonomous driving in free lanes, convoy driving, and lane changes with autonomous passing under normal traffic conditions, with human supervision and inputs only in certain circumstances [14]. Within the same decade, numerous institutions and organisations started conducting tests on vehicles with different levels of autonomy over long distances. In the 2000s, DARPA conduct tests in urban environments to simulate traffic congestions and other real-world conditions [17]. In 2009, Google conducted their first AV testing that was publicly announced only in 2012 in Nevada 2012 [18]. Later in 2010, the Institute of Control Engineering of the Technische Universität Braunschweig publicly tested AV driving in Germany [19]. Due to the increasing number of testing activities in the real world, California’s Department of Motor Vehicles (DMV) noted that their 2010 regulations had not considered AVs, and therefore, certain aspects of the regulations may not be relevant due to the advancement of AV technologies [20]. This indicated the need for developing the regulatory framework for AV testing. As a result, Nevada became the first state to pass a law on AV testing on public roads in 2011, which came into effect in 2012. In the same year, Google registered their AV with the Nevada DMV, which requires a driver to be present in the AV to monitor the testing [21,22]. Florida and California became the next two states to allow public AV testing [23].

Testing of partially autonomous vehicles or driver assistance systems became widespread among large manufacturers throughout the 2010s. In 2014, SAE International published a classification system for AV with levels ranging from manual to fully autonomous systems called the J3016 [6]. In August 2016, Singapore launched the first self-driving public taxi service, provided by nuTonomy, a spin-off company of the Massachusetts Institute of Technology (MIT) [24]. In the same year, the province of Ontario, Canada, officially allowed AVs to be tested on public roads with a driver present to monitor the testing [25]. In 2017, Canada’s first fully AV was publicly tested within a closed section of a public road [26]. As the global number of AV testing is growing and is expected to significantly increase in the near future, a synthesized testing guideline would seem a welcomed and valuable reference tool to all the parties attempting to implement public road testing worldwide. Figure 2 illustrates the aforementioned milestones of AV road testing history, from a radio-controlled vehicle of the 1930s to the existing AV testing procedures and policies of the 2010s.

Similar to other technology development, AV testing must be conducted to confirm and validate their ability and reliability, especially in terms of safety under real-world conditions. The safest AV testing method is to conduct the test within a controlled environment, such as in a dedicated area with simulated traffic, replicating the real-world road conditions. However, testing within controlled environments should not be the primary approach because it lacks many factors found in real-world conditions. Controlled or simulated environments could not replicate the uncertainties and unpredictability characteristic of uncontrolled situations. For example, the reactions of other road users may depend on factors such as emotions, habits, current physical and mental conditions and circumstances. In the simulated environment, almost all variables are controlled, bringing into question the validity of the results obtained within “fake city” environments. Hence, the need for AV testing to be conducted in the real world and the deployments of many testing organisations on public roads.

Ultimately, the synthesised global AV testing framework would contribute to designing the national guidelines for AV public road testing. Countries covered in this research include Australia, Canada, and United Kingdom. Australia’s national guideline, the “Guidelines for Trials of Automated Vehicles in Australia 2020”, aims to guide the testing organisations and local authorities in the upkeep of testing safety and requirements. The guideline also acts as encouragement in innovation and, at the same time, maintains the safety of AV testing in Australia [27]. “Canada’s Safety Framework for Automated and Connected Vehicles” is a multilayered framework that covers three areas of general testing, the jurisdictional authorities, and the testing organisation themselves. Such a flexible system allows for customising policies that suit different environments. The United Kingdom’s guideline, “The Pathway to Driverless Cars: A Code of Practice for testing”, is a framework helping the testing organisation by providing clear guidelines and recommendations for implementing key actions and procedures during the pretesting, testing, and post-testing phase [28]. Figure 3 presents Australia, Canada, and the United Kingdom guidelines in a timeframe, among other AV-related documents. Following the release of the SAE J3016 standard in 2014, the United Kingdom released their guideline in 2015, Australia in 2017, and Canada in 2018. It is expected that the existing guidelines will be regularly updated, and many more will be established worldwide.

3. AV Public Road-Testing Guidelines

AV testing guidelines provide recommendations to testing organisation and authorities on standards and practices they should consider and apply when planning and conducting public road testing. As mentioned earlier, this study examined the guidelines for AV public road testing from Australia, Canada, and United Kingdom, with a particular focus on (1) Preparation (preliminary activities before the testing), (2) Testing, (practices during the testing) and (3) Post-Testing (reporting and data management). Highlights of guidelines from Australia, Canada, and the United Kingdom, are listed in

Table 1. Highlights of guidelines from Australia, Canada, and the United Kingdom.

Subject	Australia	Canada	United Kingdom
Approvals and permits	Follow the regulations of local authorities and the policies mentioned in the guideline. May conduct trial without requiring permit which depends largely on the local authorities.	Obtained appropriate authorisation from the local authorities. Compliance with each jurisdiction is required when testing.	Obey all traffic laws such as MOT test, Road Traffic Act 1988, Highway Code.
Safety requirements	Map out routes and describe Operational Design Domain (ODD). Explain technology being tested to authorities. Sufficient closed-track testing of technologies. Establish a Traffic Management Plan.	Sufficient closed-track testing of technologies. Communication plan for the public. Notification to local authorities. Automation and manual mode switch.	Can be tested anywhere on public roads. Stability of technology must be tested and proven. Sufficient closed track testing. Automation and manual mode switch.
Data collection, storage, and security	Information such as time, date, location, automation status, traffic conditions, vehicle information, sensor detection, and the vehicle operator during the incident must be accessible.	Availability of data recording device that records technical information about the AV status and operations.	Fitted with a data recording device with a minimum frequency of 10 Hz. Include data such as driving modes, vehicle speed, steer and braking, and objects surrounding the vehicle. May include video and audio storage.
Test driver/operator	Does not need an operator in the vehicle, but real-time remote monitoring is necessary.	Must always require driver or remote operators.	Must always require driver or remote operators.
Insurance	Compulsory third-party insurance. Comprehensive vehicle insurance. Public liability insurance. Product liability insurance. Self-insurance. Work or occupational health and safety insurance.	The minimum insured value of CAD 5 million in liability insurance, in the form and manner required by the MTA authority. Additional liability insurance for large seating capacity.	Anyone conducting tests of automated vehicles on public roads or in other public places must hold appropriate insurance or otherwise comply with the statutory requirements.
Reporting	Provide an initial report of the incident within 24 h. The full report, including the relevant data and information, must be provided within a week. Must submit a report to relevant agencies monthly for minor incidents.	Preliminary report submitted within 24 h of an accident.	Give full cooperation with relevant authorities in the event of an investigation.

. The findings from the review of these guidelines are used to develop a synthesis.

3.1. Australia

The nationwide AV recommendations in Australia have been compiled into a national guideline called “Guidelines for Trials of Automated Vehicles in Australia 2020”, which was released in 2020 for AV testing, but not for large-scale commercial deployments. The National Transport Committee of Australia (NTC) and Austroads, an organisation supporting the traffic and transport agencies, played a part in forming this guideline [27]. The national guideline aims to ensure the testing organisations have similar testing conditions in all states and territories in Australia, allowing cross-border testing. The national guideline also serves as a platform to share information and research outcomes. The 2020 guideline stipulates that road transportation agencies must ensure that the AV testing is conducted safely under given conditions. The organisations conducting safety tests must adequately manage the safety risks, and AV testing without human drivers is possible only if testing organisations have shown sufficient safety planning. Accidents and injuries caused by or suffered during tests must be compensated and investigated. All organisations planning to conduct AV testing need to provide relevant information on the proposed evaluation, including a safety management plan, insurance, and available testing data to the authorities. The mandatory test and safety management plan play a critical role in addressing the areas that the testing organisation need to have in place beforehand. Upon submitting the application, the road transportation agency will determine the number of vehicles approved for the test and the time limit for the testing. There are six areas highlighted under the guideline, as shown in Figure 4, consisting of application, trial management, safety management plan, insurance, data and information, and implementation.

3.1.1. Preparation

The testing organisation is advised to initiate the application process by contacting the relevant road transportation agency to determine the permit requirements and possible exemptions. Road transport agencies use the guidelines as conditions to grant permits and exemptions. For testing that does not require a permit, the testing organisation is still encouraged to use the guideline as a basis and reference in the case of accidents.

Referring to the guideline, the testing organisation is required to address the key criteria of test and safety management in the application. In the case where criteria are deemed irrelevant, the testing organisation will be required to explain the reason for their dismissal. Testing organisations must ensure the AV application complies with the Motor Vehicle Standards Act 1989. The testing organisation is required to have an AV that is roadworthy and meets all relevant vehicle requirements to conduct the testing. Whether imported or locally manufactured, the AVs must follow the Australian Design Rules (ADR), which mandate AVs compliance with the laws pertaining to emission control, occupant protection and lighting, among other rules. The AVs must comply with all road and traffic regulations, vehicle standards, and privacy laws. Testing organisations should especially be aware of the specific regulations governed by the local authorities of where the testing will be conducted, as the permissibility of testing depends largely on the local authorities.

The testing organisation must address and explain their test management plan. The guideline addresses eight specific and essential criteria—purpose, location, technology description, traffic management, infrastructure requirement, engagement, and accessibility—that testing organisations must include in their application submission. The submission of the test management plan provides insights into the testing activities and understanding for the road transportation agency to consider when deciding whether to grant permits or exemptions.

Apart from the test management, a safety management plan is equally important for the testing organisations to prepare and submit in the application. It is also one of the key factors the road transportation agency considers in making decisions to grant permits or exemptions. The guideline requires the testing organisations to address and explain the test safety risks and mitigations strategies based on key criteria listed in the guideline. The testing organisation is also encouraged to refer to available standards such as ISO 26262 (Road vehicles—Functional safety series) and ISO/TC 241 (Road traffic safety management systems) in the preparation of their safety management plan.

In the case of AVs that fall under the heavy vehicle category, testing organisations need to extend the safety consideration due to the additional risks associated with heavy vehicles. This also includes consideration in network access, community consultation, and testing engagement.

The testing organisation is required to describe the data that will be recorded and stored during testing and describe how the data will be provided to the relevant parties.

Insurance is compulsory for AV testing; thus, in their application, testing organisations must show they have attained appropriate insurance that includes compulsory third-party insurance, comprehensive vehicle insurance, public liability insurance, product liability insurance, self-insurance, and occupational health safety insurance.

3.1.2. Testing

The testing organisation must always comply with the relevant Australian laws and local jurisdictions during testing. They are expected to adhere to their test management and safety management plans as basic rules of conduct during the testing period.

The testing organisation must submit monthly reports on near-miss incidents, system transitions from automated to human driving, and public complaints. In a case of an accident, the testing organisation is required to follow the reporting requirements of the state or territory where the testing is conducted. Information on time, date, location, automation status, weather conditions, traffic conditions, sensor detection, and the vehicle driver/operator during the incident must be made accessible to all relevant parties. Testing organisations must be able to collect and prepare the data to make possible a reconstruction of the accident that will facilitate the investigation. An initial accident report is expected to be submitted within 24 h and a full report within seven days of the incident.

3.1.3. Post Testing

After the trial has ended, the testing organisations must submit an end-of-test report based on their testing outcomes. The report must be made with a high-level summary consisting of testing achievements, challenges and lessons learned in relation to the purpose of the testing submitted in the application. However, the report does not need to include any commercial, sensitive information. Miscellaneous events should also be reported and, if needed, will be investigated further. The data submitted from all parties involved will be confidential, and no personal information will be revealed. The road transportation agency will uphold the confidentiality of the information and intellectual property.

3.2. Canada

The voluntary AV testing guideline, “Testing Highly Automated Vehicles in Canada Guidelines for Trial Organisations”, by Transport Canada, was released in 2018 to guide testing organisations in conducting safe AV testing on Canada’s public roads and to inform the roles and responsibilities of federal, provincial, and territorial representatives in Canada. The guideline has set the minimum safety requirement for the testing organisation to follow. AV testing request by an organisation requires approval by the provincial and territorial governments based on their laws and regulations. Testing organisations hold the responsibility to conduct safe AV testing and follow the government guideline requirements during all testing activities. Testing organisations must ensure the AV testing complies with the laws and regulations of the federal, provincial, and territorial regulations for motor vehicle safety and safe vehicle use, as well any relevant recommendations that may not be included in the official guideline. Furthermore, the municipal authorities have a role in vehicle/traffic movements, infrastructure, and public transportation. Therefore, testing organisations are expected to engage with the municipal authorities to consider the local traffic conditions and infrastructure availability for the AV testing. It is also important for the testing organisation to engage with the local emergency services in preparation for possible emergencies during the testing period. There are six key categories under the Canadian guidelines, shown in Figure 5, consisting of authorities, safety requirements, AV requirements, insurance, trial/remote driver, and reporting/information sharing.

3.2.1. Preparation

The guideline requires the testing organisation to consider six areas: compliances, safety requirements, AV system requirements, insurance, cross-border test, and safety declarations before the testing. The testing organisation is required to ensure that the planned testing complies with the federal, provincial and territorial requirements, including the Federal Motor Vehicle Safety Act (MVSA). The use of wireless technologies in the testing must comply with relevant procedures, certifications, and licensing requirements. In addition, the testing organisation is required to obtain authorisation or a permit for the testing location from the province or territory. In cross-border circumstances, the testing organisation is required to notify and comply with the rules, registration, licensing, and insurance regulations imposed by all provincial and territorial authorities involved. They should obtain appropriate insurance with adequate liability, as the provincial and territorial authorities require.

The testing organisation should ensure the AV has completed sufficient closed-course testing, on-road, or any other applicable testing methods prior to the testing. Moreover, they should include the environment, road, and traffic conditions expected for the planned testing. All AV testing should incorporate the capability to record testing data and information on the AV status and operation. The collected data and information may be required by relevant authorities for investigatory purposes or to address road safety-related issues.

For safety purposes, the provincial and territorial authorities may require the testing organisation to label their AVs and inform the public about the AVs presence on public roads during testing. Furthermore, the provincial and territorial authorities may also require them to inform the federal and regional/municipal officials, law enforcement, and emergency services about their presence and the details of their testing activity to ensure that these parties are prepared for possible emergency situations.

The testing organisation may conduct tests with SAE Levels 4 and 5 without the presence of a driver/operator in the AV, provided that the AV has been previously tested with a communication link between the vehicle and the personnel, equipped with continuous monitoring and warning system. The AV should also be capable of performing dynamic driving task fallback. In the test with a driver, the AV should be able to safely transit from an automated to a manual mode and should be equipped with a warning system to effectively alert the driver of the upcoming system transition from an automated to a manual mode. The test driver must hold a valid driver’s license for the appropriate class of the AV being tested and must be appropriately trained by the testing organisation to operate the AV during the test. Provincial and territorial authorities may request the training records and all additional information related to the test driver.

As part of the approval process, the provincial and territorial authorities may request a testing organisation to provide a declaration stating they have given thoughtful, thorough consideration to all critical testing areas and have taken the best appropriate measures, protocols, and equipment redundancies to address possible safety issues. The authorities may request the testing organisation to provide additional information on those measures and protocols.

3.2.2. Testing

During the test, the driver must hold a valid driving license according to the class of the AV. Test driver must safely execute their responsibilities and continuously monitor the test environment. If necessary, the test driver must also anticipate the need to intervene in the automated mode for safety reasons.

The testing organisation is encouraged to share information and related findings obtained during the test through documents or witness accounts in order to inform and educate authority officials.

In the case of unplanned disengagements and accidents during the test, the testing organisation is required to notify and submit reports to the provincial and territorial authorities. For accidents involving injury or death, the testing organisation should submit a preliminary report within the first 24 h and immediately cease all testing until receiving further instructions from the related authorities. The testing organisation must then cooperate with the authorities and provide the recorded data and information for the investigation.

3.2.3. Post-Testing

After successfully conducting the test, the testing organisation is encouraged to share their experience and lesson learnt during the test as feedback to the federal, provincial, and territorial authorities in an effort to further improve the safety regulations and procedures. Submission of the end-of-test report covering the research and test outcomes is also encouraged and would be used to help guide future policies and programs related to AVs.

3.3. The United Kingdom

The “Pathway to Driverless Cars: A Code of Practice for Testing” was released in 2015 by the Department of Transport, the United Kingdom, that aims to guide testing organisations wishing to conduct safety testing of highly or fully automated vehicle technologies on public roads or in other public places in the United Kingdom [28]. The Code of Practice is nonstatutory, designed for responsible AV testing in public places around the United Kingdom, involving a wide range of vehicles, including pods, shuttles, and heavy vehicles. The Code of Practice provides clear guidelines in safety measures for United Kingdom public road testing. However, the guideline is not intended for testing in private test tracks or inaccessible public areas, but the testing organisation may consider referring to the guideline. The Code of Practice addresses the requirements in safety, insurance, engagements, infrastructures, driver/operator, vehicle, technology maturity, and data recording, among others. The Code of Practice requires the AV to be roadworthy and always obey United Kingdom Road traffic laws for the testing. The test driver/operator is responsible for supervising the AV while observing the road traffic laws to ensure the test is safe and must be able to take over the automated operation if necessary. The testing organisation should use the Code of Practice and knowledge in legal, regulatory, and technological areas. The testing organisation is expected to provide relevant data and cooperation to the authorities in the event of an investigation. The testing organisation that fails to follow the Code of Practice during public road testing in the United Kingdom may be held liable for legal actions. There are six areas highlighted under the guideline, as shown in Figure 6, consisting of safety requirements, authorities, engagement, insurance, test driver/operator, and vehicle requirements.

3.3.1. Preparation

Testing organisations must ensure that their AVs are roadworthy, comply with all vehicle requirements, and are compatible with existing UK road traffic laws and regulations. The testing organisation must ensure the MOT is valid and hold appropriate insurance according to the statutory requirements. They should also prepare a risk management procedure that ensures the AVs have successfully completed closed-area testing before proceeding to tests on the public road. Such pretesting should establish sufficiently necessary capabilities for the AVs’ sensors and control systems to operate on public roads successfully. With the automated system, the AVs must be able to perform manual braking and steering and should be equipped with a quick and easy system allowing the test driver/operator to transition from automated to manual modes, which has already been proven in pretesting. The AV should also be equipped with a data recording device to capture and store data from vehicles, sensors, and control systems during testing. The testing organisation may also install a video and recording system into the AVs. The device should be able at minimum to record the automated and manual mode data, vehicle speed, steering commands, braking command, vehicle lights/indicators, warnings, sensors data and remote commands.

As parts of the AVs’ security system, all automated prototype controllers must already have been built at appropriate levels of security to manage any possible risk, particularly related to unauthorized access. The Code of Practice recommends testing organisations to consider standards such as the BSI PAS754 (Software Trustworthiness—Governance and management—Specification) as a security measure reference. In terms of the software used for the test, the testing organisation should ensure the software has been pretested on the bench in simulation before proceeding onto public road testing.

The test driver/operator who supervises the test must have an appropriate driving license with several years of driving experience. The testing organisation should develop training procedures for the test driver/operator competencies to understand AV characteristics, capabilities, and limitations. The test driver/operator should also be trained to respond and take necessary actions, especially in taking over the automated operation.

The testing organisation must engage with the relevant transport and highway authorities of the areas where the AV testing will be conducted. The testing organisations need to inform relevant authorities of any infrastructure required for the test performance and obtain their permission. They should also engage with the local emergency services such as the police and fire services to ensure their readiness to facilitate emergency and investigative services, if necessary, at the time of testing. Reaching out to engage the public is also recommended to provide reassurance and address public concerns related to testing risks and mitigations.

3.3.2. Testing

The testing organisation should be well-informed on managing risk during testing according to their risk management strategies, and all AV testing protocols should follow all relevant road traffic laws. The testing organisation should document and record the software levels and revisions during the testing. The captured test data and information must comply with the data protection legislation under the Data Protection Act 1998.

The test driver/operator must hold a valid appropriate driving license. The test driver/operator should be qualified, trained, and capable of operating the AV, its systems and technologies, and capable of controlling the transition from automated to manual modes. During the testing, the test driver/operator should be aware of how the AV appropriately moves on public roads compared to conventional driving. In terms of driving behaviour, the test driver/operator should pay attention to any warning alerts triggered by the AV, and if necessary, intervene and respond accordingly. They should always abide by the standard prohibitions, such as usage of hand-held devices and exceeding road speed limits, even in automated mode. As the testing may be conducted extensively, the testing organisation should also ensure that the test driver/operator does not suffer from fatigue due to long working hours. The limits should be set on the maximum hours of testing operations per driver/operator to ensure peak attentiveness. In addition, in certain testing conditions, an assistant should be considered to assist in monitoring the AV’s operation, information, and system feedback during the test, which may also increase the testing safety.

3.3.3. Post Testing

The Code of Practice does not specify any reports to be made or submitted after the testing has been complete. However, the testing organisation is expected to cooperate with relevant authorities and, if required, provide the assembled data for the purpose of investigations.

4. Synthesis of Guidelines and Recommendations

AVs are considered the future of transport; thus, public road testing is the best platform to assess their actual performance in preparing for the real world while prioritizing safety above all. Part of the effort to safely conduct public road testing is establishing guidelines that are not only related to the testing organisation but also to the related authorities and communities. Australia, Canada, and United Kingdom have released guidelines that share a common safety goal, among others. The purpose of AV guidelines is to facilitate making policies and standards that all testing organisations must comply with. Hence, a concise set of guidelines would give the testing organisation a clear indicator of appropriate steps to be taken, covering multiple overlapping areas. To facilitate AV testing and deployments on public roads, several clarifications are required for all parties: testing organisations, authorities, and communities. Based on the review of guidelines from Australia, Canada, and the United Kingdom, this study has synthesized the guidelines into seven topics, as shown in Figure 7.

4.1. Testing Processes

Guidelines from Australia and Canada were developed by the federal authority, but the jurisdiction for the approval and permits of AV public road testing lies with the local authorities. The testing organisation should engage with the relevant local authorities collaborating on the specific requirement related to testing processes, the requirements, and the test reporting guidelines. The procedures among the authorities may vary, and the existing guidelines provide only the general information for the testing organisation to follow and for the local authorities to consider adopting. To ease the testing challenges, the local authorities should agree to implement common procedures and guidelines that clearly state the common purposes and necessary steps to be taken by the testing organisations and all involved parties, resulting in more effective and efficient testing management procedures.

4.2. Testing Location

Although the testing of AVs is conducted on a public road, some limitations are apparent. Based on the reviewed guideline, testing locations are not restricted to particular areas, and testing organisations are required to obtain location authorisations. As AVs need a connection to infrastructures, the testing organisation should conduct, in advance, a thorough study to identify a suitable testing route and area before submitting their location permit applications. Identifying road types and environmental conditions benefits the authorities as well as it highlights the capabilities of the AV being tested. Certain areas do not cater to the infrastructure required to operate AV. Countries such as Australia, Canada, and the United Kingdom require the testing organisation to describe the Operational Design Domain (ODD) to the local authorities. A guideline should address the need for the ODD to include the geographical location, road type, traffic conditions, environmental conditions, time of testing, and level of technology being used, among others. Local authorities would be able to advise on the suitability of the proposed areas or suggest other locations based on the information contained in the ODD to facilitate the testing and address any other constraints such as weather and traffic abnormalities. Challenges related to location suitability may lead to mishaps and accidents that could be avoided by identifying and exchanging information between the testing organisation and local authorities on the suitability and capabilities of any given location. Responsibility for selecting a suitable testing location should be equally shared by both the testing organisation and the local authorities.

4.3. Regulations and Authorities

The current state of AV testing worldwide requires the testing organisation to refer to different authorities depending on the testing location. Although this is necessary as different states may have different rules, it is recommended that the organisation has a single contact point for testing. Australia has its guidelines issued by NTC Australia; however, local authorities must be contacted to obtain an approval or a permit that allows testing. For instance, testing in the Northern Territory requires consultation with the Department of Infrastructure, Planning, and Logistics Registrar of Motor Vehicles. In contrast, testing in Queensland requires consultation with the Department of Transport and Main Roads of Queensland. Similarly, in Canada, the testing guidelines for the organisation is a joint effort by Transport Canada and the Canadian Council of Motor Transport Administrators (CCMTA), pointing to the jurisdictional overlap. The guidelines for each nation should clearly identify the required authority to be consulted. Centralizing the contact point for the testing organisation would facilitate testing management and AV deployment. Forming a single regulatory board to represent the local authorities would be beneficial and would ultimately ease the AV testing and deployment. Listing the links to related standards, laws, regulations, and policies in the guideline also ensures the testing organisation would refer to the correct documents. Collectively, all AVs must comply with the existing vehicular laws, and the vehicles must be registered with the relevant authorities. Additionally, authorities should be notified that the vehicle is an AV testing unit which allows emergency responders and authorities to prepare for AV specific scenarios.

4.4. Test Driver/Operator

Ensuring public safety is essential for all parties involved; thus, the requirement of driver/operator presence must be addressed. The driver would act as the last fallback in case the autonomous mechanism fails. Some countries require a driver/operator to be always present in the AV, whereas Australia is more lenient and has no o such requirement. However, all countries encourage the driver/operator to be present during the testing. Existing requirements of driver/operator are in line with the customary traffic laws, such as having a valid driver’s license. Liability issues would sometimes be the driver’s responsibility, provided that the driver/operator is not attentive even though the system has indicated a possible autonomous failure.

Hence, it is a common requirement that the driver/operator should be trained in the AV’s technologies and system not only to conduct the test but also to manage failures and incidents. It is important for the training to include a standard protocol for the driver/operator to react and respond accordingly in possible emergency events such as fire, major accident and injury. This is to ensure that the driver/operator at hand would be able to minimize the damages. The guideline should specify the general technical competencies that the testing organisation can adapt into their driver’s training. Apart from the technical competencies, the driver/operator should also be trained to have skills related to safety and ethics. With sufficient training, the driver/operator would be able to manage damages, incidents, and situations effectively and diligently during the testing.

4.5. Insurance

All countries require some form of insurance. However, their coverage differs slightly depending on the depth of guidelines. For instance, Australia’s guidelines specifically mention the types of insurance policies that testing organisations should consider taking, such as compulsory third-party insurance, comprehensive vehicle insurance, public liability insurance, product liability insurance, self-insurance, and work or occupational health and safety insurance. However, in the United Kingdom, the insurance guidelines state only that the “relevant” insurance is required, which could be considered a general term. The insurance guidelines should address the liabilities involved and the minimum coverage amount that should be sufficient to cover the vehicle, product, public, occupational health and safety, and any other related risks. Furthermore, according to the guideline recommendations, obtaining relevant insurance is a priority that must be addressed before any testing begins.

4.6. Vehicle Safety Requirements

Assessment and mitigations are among the essential guideline requirements as preventive acts to reduce safety risk while simultaneously preparing to manage possible incidents. As many approaches are available, Failure Mode Effect Analysis (FMEA) would be a standard engineering practice to adopt. Nevertheless, a guideline should address the need for the testing organisation to conduct and present the assessment and mitigations to the related authorities. A guideline should require AVs to be tested in a closed circuit/area until the system is deemed satisfactory. The authorities and emergency service providers would then be able to provide recommendations and advice to further reduce the risk in any possible ways.

A system in the form of an audible warning on top of visual aid to inform testing operators of an autonomous malfunction should be compulsory. When testing operators are notified of a failure, operators could engage a physical switch that enables the driver/operator to instantly transit into manual driving mode. Therefore, a guideline should address for the AV to be equipped with an automated–manual mode transition switch to act as a reliable fallback mechanism to avoid incidents. Where the countries allow complete AV testing with a remote operator, the vehicles should be equipped with a real-time monitoring system capable of remotely taking control of the vehicle at any given moment.

As some of the AVs use existing production vehicles, integrating the AV system requires modifications. The available guidelines do not address the expected quality of the modifications. A guideline should note that any modifications performed on an existing production vehicle must refer to available standards or be carefully conducted not to cause system conflicts or secondary impairment of the vehicle performances that may affect the vehicle critical safety features such as airbag deployment.

By considering the possible incidents that may occur, a guideline should also recommend installing safety equipment: fire suppression equipment, emergency stop switch, and vehicle identifiers on the AV to further increase safety. The fire suppression equipment would be helpful for any fire events in the vehicle, while the emergency stop switch would be useful as the last response for the driver/operator to switch off the whole AV in an emergency. The vehicle identifiers in the form of signage or beacon light on the AV alert other road users to be cautious, thus increasing the safety of the other road users.

Specifying common critical vehicle safety requirements through the guideline could achieve uniformity of vehicle safety. A method to consider for countries is to form a body to conduct an AV safety inspection as part of the requirements before a trial is approved.

Reporting, Data Collection and Communication. None of the reviewed guidelines highlighted the need to assess devices used in AV. Imperfections of data produced by devices are unavoidable [29] due to interference, interactions, material properties, testing process and environment during the actual testing operation. Such imperfections and uncertainties may lead to data inaccuracy and lead to AV system failure, damage, and crash. As the level of devices in AV becomes more complex and advance over time to produce an ideal AV, the reliability of the devices becomes more critical. The testing organisation should mitigate the imperfections and verify the data to further increase the reliance on AV public road testing. Studies such as [29] can be used as an example of how the imperfections of the device can be addressed and reported.

All of the reviewed guidelines state the recommended information to be recorded. The Event Data Recorder (EDR) in vehicles must record pre, during, and post incidents in order to facilitate an investigation. Data types such as speed, time, weather condition, operating mode, forced manual takeover, and sensor data are mentioned as valuable data to be acquired. In addition, data should be accessed easily, and all information should be private and used only for investigatory and policymaking purposes.

The frequency of reporting between the guidelines varies insignificantly. All countries emphasize reporting, but guidelines by Australia and Canada highlight the frequency of the reports. A preliminary report must be submitted within 24 h in cases of serious accidents, and an end-of-the-test report is mentioned for both countries. A full report within seven days, including reports of traffic violations, is mentioned for Australia, while the United Kingdom does not propose such time restraint.

Therefore, to amalgamate a concise, clear guideline, the frequency of reports should be noted so that the testing organisation has a reliable baseline reference. All relevant information should be recorded as best practice that is already being implemented throughout all the guidelines.

5. Conclusions

Despite countries having different approaches and criteria for implementing AV testing, most guidelines concentrate heavily on AV test preparation. This is because preparing the vehicles, test areas, and permits, informing the authorities and public, preparing technology and infrastructure are keys to preventing accidents. Thus there is a lack of guidelines on testing procedures that would look into how the AV respond when acting on measures implemented in the preparation section. Post-test considerations focus on reporting and AV data analysis, which is considered satisfactory in most cases. Some guidelines’ complexity is more general than others, and discrepancies often result from countries having two to three guidelines from different jurisdictions addressing the same single issue. This may cause problems for the testing organisation, which must comply with the requirements of multiple authorities. Some guidelines put less emphasis on the post-testing procedures and do not focus on the frequency of assessments and reports provided to the authorities. This may be a concern since test reports are crucial in evaluating the performance safety of AVs. Theoretically, the synthesis of the guidelines from multiple countries allows for a concise and comprehensive global framework of regulatory measures concerning public AV testing, covering test preparation, testing, and post-testing stage. Rather than relying on a single national standard as a reference, an amalgam of guidelines from different countries allows a more holistic and measured view of AV testing practices. Thus, drafting a framework highlighting the seven synthesized elements discussed here will be a viable reference for national planning and sustaining the AV testing on public roads.