Evaluation Method to Select Pure Electric Buses Based on Road Operation Tests

: In China, the development of pure electric buses is an essential means to address energy and environmental concerns. Proper evaluation and selection of pure electric buses is a crucial step prior to introducing the buses into actual operation. This paper presents a multi-objective method for the selection of pure electric buses based on road driving tests. An index system for evaluating the operating performance through the fuzzy analytic hierarchy process is established, and to ensure the method is more systematic than those in previous studies, indicators are classified into three categories: qualitative, semi-qualitative and quantitative indicators, of which the nineteen indexes have been comprehensively considered and designed. To verify the advantage of the evaluation method proposed in the article, two typical buses were selected as the assessed objectives regarding reliability, economic, security, environmental adaptability, etc. The assessing process indicates that the method is easily implemented and of high practical value. Additionally, the results show satisfactory agreement with the actual scenario. Thus, it can be assumed that the method detailed herein provides a basis for the design, selection, and evaluation of pure electric buses.

At present, China has developed relative standards for electric vehicles; however, major aspects still demonstrate a lack of the overall consideration, which mainly concern the supporting power system and ancillary facilities. Additionally, pure electric buses generally have lower operating efficiencies compared to other vehicle technologies owing to the restrictions imposed by the battery and charging technology, thus leading to limited driving range and long charging time. Therefore, the selection process involving pure electric buses is more complicated than that of traditional buses, indicating the significance of exploring a variety of performance measurements.
Previous research mainly focused on battery economic indexes and energy consumption indexes [6][7][8][9][10]; however, the actual operation of electric buses and the importance of various other indexes are not generally considered, which results in an incomplete evaluation with low availability and the selected buses are not capable of fully satisfying all inherent operational requirements. Therefore, to comprehensively evaluate the performance of pure electric buses in actual operation, a multi-level evaluation system is established herein by considering the reliability, economy, security, adaptability, and service aspects. Moreover, existing pure electric buses conform to the relevant technical standards, such as General Technical Conditions for Pure Electric City Bus [12]; however, various factors, such as vehicle equipment manufacturing, are not elucidated in these technical standards. Therefore, this study replenishes and verifies several specific evaluation indexes to increase the feasibility, integrity and reliability of the original selection model, as listed in Table 1.

Data Collection
At present, the three main methods used for testing the performance of new energy vehicles in China and abroad are computer simulation, bench simulation, and road operation [13]. Although computer simulations and bench simulations have the advantages of low costs and short testing periods, the systems of pure electric buses are relatively complicated, leading to differences between simulations and real operating conditions. Additionally, practical factors are prone to be ignored, and thus the results are not capable of accurately reflecting the actual performance and conditions. Road operation testing examines bus performance under actual operating conditions and makes full use of the data. This allows for the selection of pure electric buses that are most suitable for the actual demands, which provides significant advantages for bus operators.
Therefore, for the selection of vehicles, this paper has implemented road testing measures as the means of data collection, which included the daily operation data collected by bus companies on actual road operations, and questionnaire surveys. The collection of daily operational data included fuel consumption, failure frequency, and other information for the different models during certain time periods. A survey was undertaken that focused on the drivers and passengers to study subjective feelings toward the pure electric buses proposed in this study.
A three-month test was conducted, corresponding to the real day-to-day routes driven on the representative lines in Nanjing, China. Two different models of similarly sized buses from different manufacturers running on the same routes were selected as the testing targets.

Index Calculation and Data Level
The evaluation indexes are grouped into quantitative indexes, qualitative indexes, and semi-qualitative indexes according to their characteristics. These characteristics are established based on information from essential data provided by Nanjing Bus Company and obtained from the internet, including previous research reports, manufacturing consultations, and surveys; each index is divided into four data levels, and the achievement of a four-star rating is considered optimal.

Quantitative Indexes
The first index category includes the quantitative indexes. The quantitative data are derived and directly reflect the index levels. These indexes include the reliability, economic, and security indexes.

Battery capacity
The battery capacity component of the reliability index refers to the battery energy storage capacity, which is represented by various electrochemical technologies. The battery storage capacity is an essential performance index to consider when weighing the battery criterion. The battery capacities are classified into the grades presented in Table 2 [8]. 2. Power to weight ratio In general, buses have large power requirements, and specifically, pure electric buses are driven by electric motors. Thus, the power to weight ratio is defined as the ratio of the motor peak power to the full-load weight. This ratio reflects the comprehensive performance of pure electric buses, including their design, the materials employed, and their power consumption [14]. Buses that are capable of attaining higher speeds enhance the transportation efficiency. Thus, high-powered buses are capable of achieving shorter acceleration times and have greater grade-climbing capabilities. The power to weight ratio of pure electric buses is divided into the grades presented in Table 3 [15]. 3. Driving range For pure electric buses, the driving range is represented by the continuous driving mileage after the battery has been fully charged while considering ideal conditions (e.g., satisfactory roads, wind speed and temperature). The driving range is an important index for pure electric buses owing to its significance in the comprehensive evaluation of battery packs and their practicability. The driving ranges vary among the different manufacturers of pure electric buses. However, from a general market perspective, the majority of driving ranges are between 200 and 300 km [8]. In this paper, the driving range is divided into the evaluation grades presented in Table 4. 4. Total number of battery cycles The total cycle time of pure electric buses can be verified according to the Chinese national standard issued in 2015. According to market research, the total cycle time of batteries for pure electric buses varies from 500 to 2000 cycles. This study divides the total cycle time into the grades presented in Table 5.

Battery attenuation amplitude
The batteries used in pure electric buses attenuate normally with the increase of service time. According to the investigation results presented in CHINA EV100 SUMMER FORUM 2016 [16], the average attenuation of a new battery's capacity is 6.05% in the first year, 9.77% in the second year, and 14.87% in the third year. When the capacity attenuation reaches 20%, the battery should no longer be used. Table 6 presents the grades for attenuation used in this study. Table 6. Grade divisions of the attenuation amplitude index.

Calculation of the Economic Index
The life cycle costs of pure electric buses include the purchase cost, energy consumption cost, and maintenance cost. For long-term operation, the energy consumption and maintenance costs of pure electric buses have advantages compared to traditional fuel buses owing to their increased efficiencies [17].

Purchase cost
According to average market estimates, the average prices of fuel buses, plug-in hybrid buses, and pure electric buses are approximately ¥500,000, ¥1,000,000, and ¥600,000, respectively. Furthermore, the purchase costs of pure electric buses are often subsidized by central and local governments, leading to little difference between the purchase costs of pure electric buses and traditional fuel buses. In this paper, the calculation formula for the purchase cost of pure electric buses is defined by Equation (1). Additionally, the purchase costs are divided into the grades presented in Table 7.

Grade Division Grade 1 Grade 2 Grade 3 Grade 4 G (units: ¥10,000)
G ≤ 60 60 < G ≤ 80 80 < G < 120 G ≥ 120 2. Electricity consumption Similar to the key economic indicator of conventional fuel buses, i.e., fuel consumption per 1000 seats·kilometers, electricity consumption per 1000 seats·kilometers can be defined as the factor to assess the economic performance of the pure electric buses, whose value represents the amount of battery consumed every 1000 seats kilometers during bus operation. The calculation formula used for electricity consumption per 1000 seats kilometers is presented in Equation (2).
(2) q = electricity consumption per 1000 seats·kilometers (kWh/1000seats·km) Q = electricity consumption during testing (kWh) S = test mileage (km) n = the number of the seat (seats) In this study, the electricity consumption is evaluated according to previous testing experience [8,9,13], as presented in Table 8. Table 8. Grade divisions of the electricity consumption per 1000 seats·kilometers index.

Maintenance cost per 100 km
In addition to the electricity consumption costs associated with the daily operation of pure electric buses, the costs of regular inspection, repair, and maintenance must also be considered. However, the maintenance costs of pure electric buses are relatively low because pure electric buses do not contain the costly components of traditional fuel buses, such as the engine, retarder, and transmission [9]. In this paper, the calculation for the maintenance cost per 100 km is presented in Equation (3). Additionally, the maintenance cost per 100 km is divided into the grades presented in Table 9.
w = maintenance cost per 100 km (¥) W = maintenance cost during test (¥) S = test mileage (km) 1. Average temperature difference of the brakes Braking performance is essential to achieve acceptable safety performance, whose stability is closely related to vehicle safety. During braking, the kinetic energy of the bus is converted into heat energy due to the friction created by the brake material, resulting in the rise of the material's temperature.
Generally, the friction coefficient of the brake material decreases with an increase in the temperature of the brakes, leading to decreased braking performance. In this paper, the average temperature difference before and after applying the brakes is considered as the security index for the brake system. Lower average temperature difference represents more acceptable brake security.
T = average temperature difference after braking (°C) x = number of test samples (time) ∆T = temperature rise after braking (°C) Based on the road test data provided by the Nanjing Bus Company, the average temperature difference of the brake is divided into the grades presented in Table 10. 2. Bus failure rate The failure of pure electric buses can be evaluated from two aspects: the failure rate and severity of the fault. The failure rate refers to the frequency of failure occurrence during bus operation and is an important parameter for measuring the safety of buses.
In this paper, the number of failures that occur per 100 km is used as the failure rate index. The calculation formula for the failure rate is presented in Equation (5), and the failure rate is divided into the grades presented in Table 11.

Semi-Qualitative Indexes
Semi-qualitative indexes represent a type of qualitative index because they do not include supporting data that directly measure the characteristics; however, semi-qualitative indexes can be quantified after conducting certain analyses. A factor reflecting the characteristics of each index can then be selected, and the indexes can be divided according to the index scope, average market level, and other relevant information. Semi-qualitative indexes include the environmental adaptability and vehicle failure severity.

Calculation of the Environmental Adaptability Indexes
The environmental adaptability indexes of pure electric buses include the temperature adaptability, weather adaptability, route adaptability, vehicle initial capacity adaptability, and air conditioning adaptability. The environmental impact coefficient ε is selected as the evaluation factor, which indicates the average variation of the electricity consumption with respect to environmental changes. After selecting n (n ≥2), the power consumption per 100 km, 1 , 2 , … , , is considered in different environments for each influence factor. The calculation formula of the environmental adaptability is presented in Equation (6).
ε = environmental impact coefficient q = electricity consumption per 100 km (kWh/100 km) n = number of test samples According to relevant data and statistics [18,19], the environmental adaptability is divided into the grades presented in Table 12.

Failure Severity Calculation
The severity of the vehicle failure can be divided into four grades according to the degree of damage, as follows [20]: (1) Fatal failure refers to a failure that threatens the safety of individuals, leading to the destruction of major components, serious economic loss, or additional harm to the surrounding environment.
(2) Serious failure refers to a failure that causes damage to important components or major parts, affects driving safety, or requires extended maintenance (greater than 30 min) using spare parts and vehicle tools.
(3) Normal failure refers to a failure of secondary components that do not affect driving safety; the failure can be repaired within a short time period (less than 30 min) using spare parts and vehicle tools.
(4) Slight failure refers to a failure that has little influence on the normal operation of the bus and does not require replacement parts; the failure can be repaired within 5 min using only vehicle tools.
The average failure maintenance time is used as the evaluation factor to measure the failure severity, as defined by Equation (7). The grade divisions are presented in Table 13.

Qualitative Indexes
The third index category represents the qualitative indexes. Qualitative indexes cannot be directly evaluated through quantitative data and require assessing the characteristics through their description by collecting data through investigations and questionnaires, and then dividing the grades according to the average market level and other relevant information. The qualitative indexes generally feature service indexes.

Operating convenience
The grade divisions for the operating convenience index include four aspects: the driving seat layout, driving vision field, acceleration and deceleration, and starting/stopping. The maximum score is five points for each of the four aspects. The grade divisions of the operating convenience are established according to the total score of the four aspects, as shown in Table 14.

Comfort
The comfort index embraces the bus seats, vibration noise, and air conditioning comfort. For each aspect, the maximum score is five points, and the grade divisions for the comfort index are established according to the total score, as shown in Table 15.

After-sale service
The after-sale service index can be analyzed in the light of the warranty period, warranty cost, after-sale service attitude, and ability to solve an array of after-sale problems. The maximum score for each aspect is five points. The grade divisions of the after-sale service index are established according to the total score, as shown in Table 16. Table 16. Grade divisions of the after-sale service index.

Comprehensive Evaluation
In this paper, the indexes used to evaluate pure electric buses are comprehensive and incorporate bus performance during actual operation. Therefore, a comprehensive evaluation method based on the operational test data is required. Different types of comprehensive evaluation methods have been proposed, e.g., the analytic hierarchy process (AHP), principal component analysis, grey correlation analysis, and fuzzy comprehensive evaluation method.
The fuzzy comprehensive evaluation method is a comprehensive decision-making methodology applicable to multivariable problems that contain complex decisions [21]. The advice of experts and the weights assigned to the factors are both fully considered in the evaluation process. This study was conducted based on a combination of the AHP and fuzzy comprehensive evaluation models, termed the fuzzy hierarchy comprehensive evaluation method. This method combines the AHP and fuzzy comprehensive evaluation methods by using AHP to determine the weight of each index, and then the fuzzy comprehensive evaluation method assesses each index [22]. This model establishes a weighting set, factor set, and evaluation set, and determines the evaluation matrix and applies the fuzzy synthesis evaluation.

Determination of the Weights by the AHP Method
Using the AHP approach, the weights of each criteria group and index are assigned in terms of their importance. The weighting coefficients are confirmed according to the relative importance of each parameter.
A pairwise comparison is performed to determine the relative importance of the identified criteria; the decision-maker must provide a score for the preferences between each pair in the hierarchy. The pair-wise comparisons are performed in terms of which element dominates the other element in the pair (i.e., based on the relative importance of the elements). The resulting judgments are then expressed as integer values, as presented in Tables 17 and 18.

Numerical Rating Connotation 1
Factor i is equally important to factor j 3 Factor i is slightly more important than factor j 5 Factor i is clearly more important than factor j 7 Factor i is strongly more important than factor j 9 Factor i is extremely more important than factor j 2, 4,6,8 Intermediate values Reciprocal : factor i compared with factor j, : factor j compared with factor i = 1/ For the completely consistent matrix A, there is a unique non-zero eigenvalue, and the corresponding eigenvector is used to determine the weight vector, W, using Matlab 7.0 (MathWorks, Natick, MA). To avoid artificial errors and contradictions among the different factors, a consistency test must be conducted until a satisfactory condition is met. The consistency index (CI) and consistency ratio (CR) are used to check for the consistency associated with the comparison matrix, CI, based on the following two equations. RI is a random consistency index, which is obtained using a look-up table (Table 19). Normally, when CR < 0.1, the degree of inconsistency is acceptable, which states the consistency test is passed.

Determination of the Evaluation Set and Subjection Function
Based on the index data and data level, the evaluation set and subjection function can be generated while referencing the proposed fuzzy comprehensive evaluation model, thus providing a comprehensive evaluation. The subjection function is the fuzzy membership degree of the appraisal of index to grade k (determined using Table 20).

Quantitative indexes and semi-qualitative indexes
If the index parameters and levels show a positive correlation, the calculation method is as follows: If the index parameters and levels produce a negative correlation, the calculation method is as follows: 2. Qualitative indexes Based on the scores provided by experts to evaluate each index, the calculation method is as follows: = total number of experts who evaluate index = number of experts who give grade k to index Then, the appraisal of a single index (Class 1) is represented by : i = number of indicators of Class 1 j = number of indicators of Class 2 k = number of data levels of each indicator

Determination of the Fuzzy Evaluation Matrix
Based on the above factor and evaluation sets, quantitative research can be conducted with reference to the fuzzy comprehensive evaluation model to generate a comprehensive evaluation of the pure electric buses being considered. From the subjection function and weighting vector of the indexes, the fuzzy evaluation matrix can be constructed based on the data and evaluation set.
The first level of fuzzy evaluation matrix B is represented by the following: The second level of fuzzy evaluation matrix B is as follows:

Results and Discussion
After the comprehensive evaluation of each program, the appraisal results are obtained. For the purpose of comparing and selecting among the different buses, a number is developed to quantify the overall performance. The data levels of the indexes are given specific values, thus composing a vector P. The comprehensive evaluation value of different kinds of buses can be derived after multiplying vector P and matrix B of the fuzzy evaluation as follows.
The results of the comprehensive evaluation value can directly reflect the operational performance of each bus, providing a convenient means of comparison. Additionally, the proposed model can depict the evaluation process involving multi-factors and thus demonstrate the actual performance levels of pure electric buses.

Case Study
In this section, two pure electric bus models, NJL6100BEV30 (Bus 1) and GTQ6105BEVBT8 (Bus 2), were chosen as the case study for illustrating the application of the fuzzy-AHP comprehensive evaluation model based on road driving tests conducted in Nanjing. Both 10 m in length, the shapes of these two buses are similar, with the generally consistent internal configurations. The average operational data of the two buses are shown in Table 21. Scores were provided by 15 experts for the qualitative index evaluation, and the results are shown in Table 22.   Table 23, the second level evaluation matrices for the two buses are determined and presented below (Equation (13)).

Fuzzy evaluation matrices
The fuzzy evaluation matrices are calculated in this section, including the indicators of reliability, economic, adaptability, safety and service, which are respectively represented by the matrices R ' i1, R ' i2, R ' i3, R ' i4, R ' i5 (i = 1, 2).

Fuzzy evaluation matrices and final results
Take standard value vector of evaluation grade: where, the value 4, 3, 2,1 is respectively corresponding to evaluation set level 1, level 2, level 3 and level 4.
• The final performance score of Bus 1: • The final performance score of Bus 2: Because 6 > 3.493, the operational performance of Bus 1 is superior to that of Bus 2.

Conclusions
This paper presents a multi-objective selection method for pure electric buses based on road driving tests, and the work done in this research can be concluded as follows: (1) An analysis of the operation characteristics of pure electric bus was conducted, the key factors summarized, and a qualitative and quantitative evaluation index system of pure electric bus operation was built.
(2) The indicators for evaluating the pure electrical buses were carefully classified and processed; this could be divided into three categories: quantitative, semi qualitative and qualitative according to the characteristics of the indicators. The semi qualitative and qualitative indicators were also quantified respectively for the further calculation and assessment, based on the market average level and relevant data.
(3) On the basis of the classification and definition of the indicators, fuzzy-AHP was selected as an appropriate method for the comprehensive evaluation of the performance of two actual pure electric buses, and with the practical application of the index systems and evaluation method proposed in this article, the operation characteristics were easily reflected by the quantitative calculation, which means that the evaluation method is easily implemented and straightforward to analyze the performance of the pure electric buses.
Based on the aforementioned conclusions, it can be assumed that the method presented herein has several advantages: (1) The performance evaluation index system includes qualitative, semi-qualitative and quantitative indexes considering a wide range of performance factors.
(2) The data was collected based on road operation tests, which closely reflect the actual bus operating conditions, verifying the practicability of the aforementioned evaluation system.
(3) The qualitative indexes are quantified, and thus generate comprehensive and intuitive results by using the fuzzy comprehensive evaluation method. The results reveal that the proposed model successfully applies the multi-factor evaluation process and can provide results pertaining to the actual capabilities of pure electric buses.
Despite these considerable edges, this method still has a few shortcomings. Due to the involvement of determination when designing qualitative index, the index weights may be inevitably subjective, and the road operation tests can sometimes be affected by various factors, such as the driver, environment, and other unforeseen variables. These challenges may cause slight discrepancies in the test results and can be improved in subsequent studies.
In general, the method presented herein is original since it is more practical and comprehensive than previous evaluation methods. This method can pave a new way for bus companies to evaluate and select electric buses for use in cities in China, and even worldwide.