Testing Methodology of Vehicle Pedestrian Notification Systems

Transport Canada’s ecoTECHNOLOGY for Vehicles Program tests and evaluates the safety and environmental performance of advanced vehicle technologies. One area of investigation, in collaboration with Transport Canada's Motor Vehicle Safety Standards and Regulations Group, is audible alert systems for electric vehicles. BEVs and HEVs can be significantly quieter than conventional vehicles at low speeds. The inclusion of sound alert systems that emit a detectable minimum sound is currently being studied as one option to enhance pedestrian safety. This paper will provide an overview of TC’s testing to measure/assess the noise emissions from BEVs, HEVs and conventional vehicles, and various manufacturers’ noise emissions systems for HEVs / BEVs.


Introduction
The mandate of Transport Canada's ecoTECHNOLOGY for Vehicles (eTV) program is to test and evaluate advanced technology vehicles for safety and environmental performance. Results are used to support the development of relevant codes and standards, and support the development of safety and environmental regulations.
Results are disseminated in appropriate fora, such as EVS 28.
Blind and partially sighted pedestrians, among other vulnerable road users, rely upon motor vehicle sounds as a key input to navigate roadways. Thus, it was considered pertinent to gather data on the sound levels being emitted by quiet vehicles.
Transport Canada undertook a project to record the sounds emitted from several electric, plug-in hybrid and hybrid vehicles. Comparisons were made between these 'quiet vehicles' and their conventional internal combustion engine (ICE) counterparts under the same test conditions and test setup. The goal of this project was to provide data on quiet vehicle test scenarios to expand the body of knowledge on this topic.
The testing outlined in this report was carried out for Transport Canada's ecoTECHNOLOGY for Vehicles (eTV) program and Motor Vehicle Safety Directorate by Brüel & Kjaer and Sound Answers Inc. with Transport Canada engineering support. Transport Canada's eTV program is working in collaboration with governments, industry and academia to test and evaluate the safety and environmental performance of advanced vehicle technologies in Canada, including battery electric, plug-in hybrid, fuel cell, clean diesel and advanced gasoline vehicles. The eTV program's test results are helping to develop codes, standards and regulations that government and industry require to introduce these technologies in Canada in a safe and timely manner. Additional program details can be found at www.tc.gc.ca/eTV.

Method
Sound testing, measurements and data collection according to the SAE 2889-1 Measurement of Minimum Sound Emitted by Road Vehicles -May 2012 [2] was carried out on a total of 11 vehicles:

I.
At two separate facilities with programmable dynamometers within an anechoic chamber capable of meeting the performance requirements of SAE 2889-1.

II.
Each vehicle underwent a minimum of four trials at each test condition.

Measurement of Background Noise
The background noise (BGN) was measured and will be applied as a correction factor. The minimum sound emitted was calculated for each measurement, and averaged for each microphone individually. The minimum sound pressure level (SPL) as described in the standard was reported.
Additional baseline sound tests were performed at both facilities: Testing according to the current SAE 2889-1 [2] sound emissions standard at 10 kph was required. Each test condition was performed successfully a minimum of four times for a minimum of ten seconds at a sampling rate of 51.2 kHz.

Modified Testing as per SAE 2889-1 in an Indoor Facility
Testing per SAE 2889-1 additionally at speeds of 0, 20 and 30 kph but with the vehicles installed on a chassis dynamometer and elevated 2-4 inches above the dynamometer rolls. The microphones were elevated correspondingly; the test was run with the drive wheels spinning freely without load resistance from the chassis dynamometer.
All test vehicles were prepared for normal operating conditions and equipped with the manufacturers' recommended tires.

Test Setup
The hemi-anechoic chambers at the two facilities meet the ISO 3745 [3] specification for free-field conditions and have cut-off frequencies below 100 Hz.  The left and right microphones were positioned in-line with the front of the vehicle, then 2 m from the vehicle centre line and 1.2 m off the ground.
 One microphone will be placed near the location of the VPNS.
 One binaural head will be placed forward facing at the discretion of the sound analysis engineer.  Table 1 lists the test vehicles supplied for the indoor testing. Each vehicle with the exception of the Tesla Roadster and Zero motorcycles have an ICE 'counterpart'. For the purposes of testing, a 'counterpart' vehicle is being defined as a vehicle model equipped with an internal combustion engine which is of the same make, model and body style as its electric or hybrid equivalent.

Test Vehicles
It is agreed that any counterpart vehicle is not an identical comparison given that the vehicles are only cosmetically similar, though we believe this to be the best method when comparing an ICE engine's emitted sound to those of a BEVs and HEVs.  Table  4 & Table 5. A VPNS is designed to emit a series of sounds to alert pedestrians to the presence of an electric vehicle. The sounds emitted are not specific to any standard and are unique to each vehicle and manufacturer. The VPNSs that are equipped operated in a range from 0-24 kph at which point they would turn off. HEVs were always tested in EV mode only so as not include the sound emitted from their ICE engines, as this would be the quietest form of vehicle operation. One Hyundai Sonata was a prototype with a VPNS that was not yet in full production.

Test Facilities
The two test facilities used were located at International Automotive Components (IAC) in Plymouth, Michigan and the General Motors Milford Proving Grounds (MPG) in Milford, Michigan. The specifications for each facility equipped with both an anechoic sound chamber and chassis dynamometer are listed in Table 2.

Test Conditions
The test scenarios were developed in order to be able to account for the background noise of the dynamometer rolls, and the effect of the transmission. In addition a number of tests were performed with the vehicle elevated off of the dynamometer rolls in order to see if any appreciable differences were noted. With the vehicle elevated off of the rolls the transmission is not under any load and the test is counterpart to running the test in a traditional anechoic chamber. Table 3 lists the test conditions.

Results & Discussion
Due to the sheer number of tests and vehicles, and in order to not exceed the maximum page limit for this paper, the results section will be presented as a summary of results with the comparisons deemed most relevant highlighted.
Each vehicle was operated at several test conditions. For each condition the operational results were averaged from 4 successful runs with duration of 30 seconds each.      Figure 5 at 10 kph the ΔSPL difference between the vehicle with the VPNS in EV mode and the one without the VPNS in EV mode is < 1 decibel (dB). This is not realistically distinguishable with an overall level presentation. There is a < 2 dB difference ΔSPL difference between both EV scenarios and the ICE engine.
There is an approximate 6 dB increase in the vehicle operational level compared with vehicle being rolled in neutral at 10 kph.

Figure 6: Overall Comparison SPL (dBA) L CRS -Hyundai Sonata 20kph
At 20 kph Figure 6 shows there is an approximate 4 dB increase in the vehicle operational noise level compared with vehicle being rolled in neutral at 20 kph. Hyundai engineers described that the VPNS system was designed to turn off at 20 kph. It is unknown if this operating speed of 19-21 kph will always have the VPNS operating, though subjectively it does appear to operate for this speed setting.
At 20 kph it is interesting to note that while operating in EV mode with no VPNS and with the ICE engine running the ΔSPL difference between the two scenarios is negligible < 1 dB. Further, when the vehicle operates in EV mode with a VPNS the ΔSPL is approximately a 2 dB increase. At 30 kph Figure 7 shows kph results with the VPNS turned off as programmed by the manufacturer. Of note is that the two vehicles operating in EV mode were measured at slightly higher dB levels than the ICE test. Under all test scenarios the ΔSPL is approximately the same due to the increasing contribution of tire noise in each scenario.

Elevated Testing Results
Given the unique nature of the test facility equipped with both an anechoic chamber and a chassis dynamometer, Transport Canada wanted to examine if it was possible to evaluate the same test conditions with the vehicle elevated. If successful, this would allow the use of a much greater number of facilities world-wide to complete testing and validation of VPNSs. The vehicles were elevated using bottle jacks placed directly underneath the front suspension and raised the vehicles 2-4 inches (5-10cm). Table 6 presents results for elevated versus on dynamometer tests at 5 different test conditions including at 10, 20 and 30 kph. It can be seen that at 20 and 30 kph the fact that the Leaf is not under load results in significantly lower dB levels than when on the dynamometer and under load. This was not the case for the Volt, whose dB levels were similar for both on dynamometer and while elevated. While these results were not expected they were none the less performed in order to quantify what the difference would be in order to allow regulators and manufacturers the ability to assess the differences between a test performed in a traditional anechoic chamber and one performed in an anechoic chamber equipped with a dynamometer. Elevated test at 20 and 30 kph were not performed on the Versa and at 20 kph on the Cruze .   Figures 8 and 9 displays a colour map representation of the recorded observations between the Nissan Leaf in both elevated and dynamometer test scenarios. The VPNS tones are identified by arrows. It can be seen that in the elevated tests, without the background noise of the dynamometer operation, that the VPNS tones are easily distinguishable. However, in real world operation road and tire noise will exist.

Facility Comparison
Figures 10 and 11 present a summary of dB levels measured at the two test facilities in five separate test conditions; i) ambientdynamometer off, ii) ambient -dynamometer on, iii) ambient -dynamometer on -vehicle on, iv) SS 10 kph -vehicle in neutral and iv) SS 10 kph -vehicle in drive. By presenting a summary of these 5 test conditions at the two test facilities, determinations can be made as to the how much variability may be seen in separate facilities that both meet the requirements of SAE 2889-1. The 20 and 30 kph tests have not been included due to the similarity in their values.
As can be seen from Figures 10 and 11 the dB levels for both the Leaf and Versa across all 5 test conditions are similar except for the ambientdynamometer on -vehicle on measurements. This is capturing the noise difference between an idling ICE (Versa) and a BEV (Leaf) that is "on" but not in motion. In this case the difference is approximately 10 dB. Essentially the noise levels for a full BEV being on or off is often nil unless a coolant pump of or fan engages. Often an extra check must be performed to ensure whether a BEV is on or off. Additionally, it can be seen that the dynamometer operation of the IAC facility is approximately 5 dB(A)s higher than that of the GM proving grounds. However their differences quickly diminish at 10 kph with the GM proving grounds being slightly louder under load and slightly quieter with the vehicle in neutral.

Summary
Indoor testing according to the current provisions of SAE 2889-1 is challenging as only a very small number of facilities in North America meet the specifications for the anechoic chamber with a chassis dynamometer. However, it may be preferential to outdoor testing, as many challenges exist with the ambient noise levels experienced in the outdoors. Little measurable difference was seen between the results obtained from the two SAE 2889-1 rated facilities where the tests were performed.
The results showed that the rotation of the dynamometer rolls has an appreciable effect on the sound levels produced as little difference was seen between sound levels at 20 and 30 kph in both the conditions where the vehicle was in neutral (motive power from the dynamometer) and in drive (vehicle under load).
As expected, at the lower speed tests of 10 kph both BEVs and ICEs emit lower levels of sound and thus risk not being detected by a pedestrian.
In the case of the Leaf and Versa the Leaf was essentially silent on "idle" and emitted similar sound levels its ICE counterpart at 10 kph. For the case of the Volt and the Cruze the Volt was actually louder (74 dB vs. 60 dB) than the ICE at 10 kph showing that the design and installation of a VPNS can vary significantly by manufacturer.
Elevated testing was performed in order to obtain a data set to allow the sound community to assess the viability and acceptability of testing vehicles and their VPNSs in traditional anechoic chambers, which exist in much larger numbers than an anechoic chamber equipped with a dynamometer. It is up to sound experts to analyse the data and come to a conclusion.
Additionally it is up to regulators and sound experts to analyse the full data set and decide what the technical requirements of a VPNS should be for quiet vehicles, if any. We make no pronouncements in this area by choice and simply obtained a large data set of ICE vs. BEV/HEV counterparts to allow the discussion to occur from a position of knowledge and not conjecture.