Measurements of the Emissions of a “Golden” Vehicle at Seven Laboratories with Portable Emission Measurement Systems (PEMS)

In the last years, the in-use emissions of vehicles are measured on the road with portable emissions measurement systems (PEMS). PEMS cannot measure as accurately as the laboratory grade equipment, and studies on their measurement uncertainty have continued since their appearance in the market. In this study we compared PEMS to laboratory grade equipment in Italian laboratories testing a diesel “Golden” (i.e., reference) vehicle for two consecutive years. The results showed equal means of PEMS and laboratory grade equipment for carbon dioxide (CO2), nitrogen oxides (NOx), and particle number (PN), with a variability of ±5 g/km for CO2, ±10 mg/km for NOx, and ±1 × 1011 p/km for PN, which further decreased in the second year. For carbon monoxide (CO), the PEMS were on average 5–20 mg/km higher than the bags (variability ±40 mg/km). The main conclusion of this study is that PEMS are accurate under controlled laboratory ambient conditions, without any indications of significant bias.


Introduction
Before entering the market, vehicles have to be type-approved, demonstrating that they fulfill the emission limits set in the regulations. The emissions under real-life operation, though, are typically assessed by tunnel measurements [1], chasing tests (i.e., following cars with a mobile laboratory), on-road measurements with portable systems, or even laboratory measurements simulating real-world routes [2]. The differences between typeapproval and on-road values of vehicles was a big topic a few years ago in the European Union (EU), not only for pollutants emissions [3,4], but also for fuel consumption and CO 2 [5]. The situation significantly improved with the assessment of the vehicles on the road [6]. As outlined in the European Green Deal, a 90% cut in emissions by 2050 is planned, delivered by a smart, competitive, safe, accessible, and affordable transport system. By 2030 at least 30 million zero-emission cars will be in operation on European roads, and by 2050 nearly all cars, vans, buses as well as new heavy-duty vehicles will be zero-emission. For transport to become sustainable, boosting the uptake of zero-emission vehicles, renewable and low-carbon fuels and related infrastructure are necessary, among other strategies, along with an improved urban mobility [7][8][9]. As announced in the recent Communication on the Sustainable and Smart Mobility Strategy, the Commission will strengthen the carbon dioxide standards for cars and vans and will propose more stringent air pollutant emissions standards for combustion-engine vehicles (Euro 7). The on-road assessment of the vehicle emissions with portable emissions measurement systems (PEMS) The inter-laboratory comparison exercises (ILCEs) with PEMS were conducted in 2018 (November 2018 to July 2019) and 2019 (September 2019 to June 2020) with the same diesel "Golden" reference vehicle. They were part of larger exercises with more participants measuring only with the regulated method (bags from the dilution tunnel), organized by CUNA (Commissione Tecnica di Unificazione nell'Autoveicolo). CUNA is the Italian standardization body for automotive, federated to UNI (Ente Nazionale Italiano di Unificazione), the Italian standardization body. CUNA, as a proficiency testing provider, organizes annually inter-laboratory activities among Italian laboratories. Table 1 summarizes the number of participating laboratories at the larger ILCEs (in 2018 and 2019) and the laboratories that additionally used their own PEMS. They were all located in Italy and included OEMs (original equipment manufacturer), institutes, and universities. Due to confidentiality issues, the critical topic of PEMS uncertainty, and the many laboratories involved, it took one year from the last test until the publication of results.

Test Protocol
All laboratories conducted three repetitions of the cold start worldwide harmonized light-duty test cycle (WLTC) with pre-defined road-loads and gear shift strategy, as described in the European Union (EU) regulation. The emissions were determined from bags collecting diluted exhaust gas from the dilution tunnel during the test cycle. For particle number emissions, real time analyzers have been used directly at the dilution tunnel. Some of the laboratories (7 in 2018 and 6 in 2019) connected their own PEMS at the tailpipe during the measurements. The Regulation (EU) 2017/1151 defines the permissible tolerances of the differences of gas PEMS to the bag results (e.g., for CO 2 the limits are 10 g/km or 10%, whichever is larger) and Regulation (EU) 2017/1154 for particle number (50% or 1 × 10 11 p/km, whichever is larger). This "validation" test is done optionally to confirm the proper installation and operation of the PEMS. To verify the vehicle stability, the first laboratory conducted tests at the start, in the middle, and at the end of the exercises. No statistically significant differences were found.

PEMS
Only one laboratory (from the five that were common in the two ILCEs) used the Horiba (Kyoto, Japan) OBS-ONE, which measures CO 2 and carbon monoxide CO with heated NDIR (non-dispersive infrared detection), NO x with heated CLD (chemiluminescence detection), and particle number with CPC (condensation particle counter) after a hot catalytic stripper [16].
The other laboratories used the AVL (Graz, Austria) MOVE, which measures CO 2 and CO with NDIR, NO x with NDUV (non-dispersive ultraviolet) [12], and particle number with DC (diffusion charger) after a hot catalytic stripper [10]. Exhaust flow meters of 2 or 2.5 inches were fitted. It should be mentioned that the laboratory particle number systems have only evaporation tubes (i.e., no catalytic stripper), but for the size range of this study (>23 nm) no differences are expected [17].

Golden Vehicle and Fuel
The Golden vehicle was a Euro 6d-temp 1.6 L diesel vehicle (max power approximately 90 kW) with diesel oxidation catalyst (DOC), diesel particulate filter (DPF), and selective catalytic reduction (SCR) for NO x . No maintenance or repair took place during and between the two ILCEs. For each ILCE, all laboratories used reference fuel from the same batch, fulfilling EN 590:2013 specifications, which is the European Standard specifying requirements and test methods for automotive diesel fuel. The analysis of the fuels of the two ILCEs showed minor differences ( Table 2).
The B5 (with 5% biodiesel) and B7 (with 7% biodiesel) fuel specifications of the regulations are also given for comparison. The 2019 fuel was also compliant with the B7 reference fuel specifications (Regulation 2017/1151, Annex IX). The density at 15 • C, the PAH (polycyclic aromatic hydrocarbons), and the FAME (fatty acid methyl esters) content of the 2018 fuel were slightly lower than the limits for B5 reference fuel (Regulation 692/2008, Annex IX), and the distillation 95% point was slightly higher ( Table 2).

Calculations
From the data, only the PEMS PN results of one lab were excluded, because it was found that there was condensation in the sampling tube of the instrument. For each lab the arithmetic mean (µ) of the three repetitions was calculated for each pollutant for the bags and the PEMS ( Figure A1 in the Appendix A, Bags x i , PEMS y i ). Then, the mean of the means with the maximum and minimum values were used for graphical purposes for all laboratories or the common five laboratories for both exercises, in 2018 and 2019. Furthermore, for each pollutant and test, the difference between PEMS and bags was calculated (z i ). Then, for each lab, the mean difference of the three tests was calculated. Finally, the mean of the laboratory PEMS-bags differences (along with the minimum and maximum values) were plotted and compared with the permissible tolerances set in the regulation for the "validation" checks. The repeatability and reproducibility following ISO 5725 were also calculated [18]. Table 2. Fuel characteristics at the two inter-laboratory comparison exercises (ILCEs) in 2018 and 2019 and comparison with reference fuels B5 and B7 and the European standard EN 590:2013. FAME: fatty acid methyl esters; PAH: polycyclic aromatic hydrocarbons. The terms "% m/m" and "% v/" are used to represent the mass fraction and the volume fraction, respectively.  The CO 2 emissions ( Figure 1a) were around 140 g/km with a variability of approximately ±3 g/km up to ±7 g/km, with the bag results typically on the lower variability range (the exception was 2019, with 6 labs). The PEMS had a low variability (±2 g/km) only in 2019 within 5 labs. No limit is shown in the figure because the EU monitoring CO 2 approach is fundamentally different from the not-to-exceed approach for pollutants. The topic will be analyzed in the Section 4.

Absolute Emission Levels
The NO x emissions ( Figure 1b) were approximately half of the applicable limit for this vehicle (40 mg/km). In the 2018 ILCE they were slightly less (5 mg/km) compared to the 2019 ILCE. The variability was ±5 mg/km to ±8 mg/km without a particular trend for the ILCE year or the measurement method.
The PN emissions (4 × 10 11 p/km) were close to the applicable limit for this car (6 × 10 11 p/km). The variability was large (1-3 × 10 11 p/km), with PEMS at the larger end of the variability. The 2019 ILCE had a lower variability for both the laboratory and PEMS, typically 1 × 10 11 p/km (25% of the emission level).
The CO emissions were very low (50 mg/km), 10 times below the limit. The variability was similar between bags and PEMS, typically 25 mg/km (50% of the emissions).
In general, the variability of PEMS was similar to the variability of the laboratory equipment. The variability presented includes the variability of the instruments, but also of the vehicle and the laboratories' facilities and procedures.

PEMS Differences
The high variability of the emission levels may mask any bias between PEMS and bags. For this reason, the differences of the two methods for each test and pollutant were calculated. The average differences are plotted in Figure 2.
The differences between PEMS and bags were within 5 g/km for CO2 (Figure 2a), with practically equal means. The differences marginally decreased in the 2019 ILCE (±4 g/km). The differences were well within the permissible tolerance of 14 mg/km for CO2 (±10 g/km or 10%, whatever is larger) allowed in the regulation.
The differences for NOx were less than ±10 mg/km in the 2019 ILCE and ±5 mg/km in 2018 (Figure 2b), with almost equal means. The higher variability in 2019 had to do with the high difference at one laboratory (PEMS almost 10 mg/km lower than bags). The reason is not clear because there were no real time NOx data from that laboratory. In any case, all differences were well within the permissible tolerance of 15 mg/km for NOx (±15 g/km or 15%, whichever is larger).
The differences for PN were around ±1 × 10 11 p/km (Figure 2c), but the mean differences were almost equal. The higher variability in 2018 had to do with the high difference

PEMS Differences
The high variability of the emission levels may mask any bias between PEMS and bags. For this reason, the differences of the two methods for each test and pollutant were calculated. The average differences are plotted in Figure 2.
The differences between PEMS and bags were within 5 g/km for CO 2 (Figure 2a), with practically equal means. The differences marginally decreased in the 2019 ILCE (±4 g/km). The differences were well within the permissible tolerance of 14 mg/km for CO 2 (±10 g/km or 10%, whatever is larger) allowed in the regulation.
The differences for NO x were less than ±10 mg/km in the 2019 ILCE and ±5 mg/km in 2018 (Figure 2b), with almost equal means. The higher variability in 2019 had to do with the high difference at one laboratory (PEMS almost 10 mg/km lower than bags). The reason is not clear because there were no real time NO x data from that laboratory. In any case, all differences were well within the permissible tolerance of 15 mg/km for NO x (±15 g/km or 15%, whichever is larger).
The differences for PN were around ±1 × 10 11 p/km (Figure 2c), but the mean differences were almost equal. The higher variability in 2018 had to do with the high difference at one laboratory (1.2 × 10 11 p/km) (note that one laboratory was excluded from the analysis due to condensation issues). The reason is not clear, but it is possible that there were higher particle losses in the connecting tube from the vehicle to the dilution tunnel. Still, the differences were within the permissible tolerance of 2.2 × 10 11 p/km for PN (±1 × 10 11 p/km or 50%, whichever is larger).
The differences for CO were less than ±40 mg/km in the 2018 ILCE and ±25 mg/km in 2019 (Figure 2d). The PEMS means were 5-20 mg/km higher than the bags. The differences were well within the ±150 mg/km permissible tolerance prescribed in the regulation.
Sustainability 2021, 13, x FOR PEER REVIEW 6 of 13 at one laboratory (1.2 × 10 11 p/km) (note that one laboratory was excluded from the analysis due to condensation issues). The reason is not clear, but it is possible that there were higher particle losses in the connecting tube from the vehicle to the dilution tunnel. Still, the differences were within the permissible tolerance of 2.2 × 10 11 p/km for PN (±1 × 10 11 p/km or 50%, whichever is larger). The differences for CO were less than ±40 mg/km in the 2018 ILCE and ±25 mg/km in 2019 (Figure 2d). The PEMS means were 5-20 mg/km higher than the bags. The differences were well within the ±150 mg/km permissible tolerance prescribed in the regulation.

Discussion
This study presented the emissions of a vehicle during two ILCEs, determined with bags from the full dilution tunnel or PEMS measuring from the tailpipe. The mean measured CO2 value was around 140 g/km. Each vehicle registered in the EU has its own (vehicle-specific) OEM-declared CO2 value stated in its Certificate of Conformity (CoC). As this vehicle was not type-approved, but only calibrated to the Euro 6d-temp standards, the official CO2 value was not available. The type-approval documentation of this vehicle model (interpolation family) declared that in the best case (officially called vehicle low) the

Discussion
This study presented the emissions of a vehicle during two ILCEs, determined with bags from the full dilution tunnel or PEMS measuring from the tailpipe. The mean measured CO 2 value was around 140 g/km. Each vehicle registered in the EU has its own (vehicle-specific) OEM-declared CO 2 value stated in its Certificate of Conformity (CoC). As this vehicle was not type-approved, but only calibrated to the Euro 6d-temp standards, the official CO 2 value was not available. The type-approval documentation of this vehicle model (interpolation family) declared that in the best case (officially called vehicle low) the CO 2 emissions were 138 g/km and in the worst case (officially called vehicle high) 152 g/km. Our tests were conducted with values (mass, road load coefficients) similar to the low case. Hence, the mean value of 140 g/km measured in this campaign is in agreement with the expected value of 138 g/km.
To put the results into perspective, the measured value is higher than the target value of 130 g/km for the fleet of each vehicle manufacturer in the 2015-2019 period (Regulation (EC) 443/2009). It should be highlighted, though, that the targets were applicable to another test cycle: the NEDC (new European driving cycle). Furthermore, other differences, for example the road load determination, and the weight of the vehicle can further influence the CO 2 emissions in the order of 11-25% [19][20][21]. In 2019, for the third consecutive year, the average CO 2 emissions from light-duty vehicles in EU increased, reaching 122.4 g/km, making the 2021 target of 95 g/km challenging [22]. This means that the share of hybrids and electric vehicles has to be increased. The recent announcement of the European Commission requires a 55% average CO 2 reduction of new cars from 2030 and 100% from 2035 compared to 2021 levels. As a result, all new cars registered as of 2035 will be zero-emission [23]. Considering that the average age of vehicles in EU is 11.5 years, the end-of-life vehicle recycling policies become very important [24][25][26][27].
Regarding the other pollutants, the results follow a not-to-exceed approach. The CO emissions were ten times below the respective limit, while the NO x emissions were half. The results are in good agreement with other studies which showed that the emissions of Euro 6d-temp (or, since September, 2020 Euro 6d) vehicles are much lower than the limits for typical environmental and driving conditions [6]. The market share of these vehicles is still low (around 11% of diesel vehicles), as they entered into force in 2017 [28]. Nevertheless, the future Euro 7 limits are expected to be very low, as they will be referring to near-zero-emission vehicles.
Comparing the emissions of the 2018 and 2019 ILCEs, the mean emissions of PN and CO were the same. The mean emissions of the vehicle were slightly lower in the 2019 ILCE for CO 2 (around −1 g/km), and slightly higher for NO x (+5 mg/km). The differences between the two ILCEs were not statistically significant (two independent group tests, T-student test). The stability tests of the first laboratory at the start, middle, and at the end of the exercise did not show any trend for the CO 2 (+0.5 g/km), while there was a small increase for NO x (not statistically significant) (+3 mg/km). Nevertheless, we cannot exclude a possible effect of the higher FAME content and the higher density in the 2019 ILCE on NO x and CO 2 [29][30][31]. Measurable differences have been reported between no, 5%, or 10% biodiesel content in the fuel [32], or even between 2% and 4% for NO x [33].
In order to exclude the vehicle variability from the PEMS and bags comparisons, for each test and pollutant the PEMS and bags difference was calculated, and then the mean and the standard deviation of the differences were evaluated. This study clearly showed that the PEMS results are equivalent with the bags results, under laboratory conditions (30 min test duration, constant temperature and pressure, no vibrations). This conclusion is based on the similar emission levels measured for all pollutants, but also on test-bytest differences, which did not reveal any bias. What is important is that the differences between PEMS and bags were much lower than the permissible tolerances prescribed in the regulation, indicating that the permissible tolerances could be reduced. For example, for NO x the 15 mg/km tolerance could be reduced to 10 mg/km. Similarly, the CO tolerance of 150 mg/km could be reduced to at least 100 mg/km. A reduction of the 50% permissible tolerance of PN to 40% also seems feasible. These results are in agreement with our older study with a gasoline car, where similar differences were found, except for PN (where the differences were close to the tolerance) [15]. They are also in agreement with our recent review that summarized tests from different laboratories [34]. Other researchers found ±8% CO differences for a gasoline vehicle and 15% on average for a 6.7 L diesel vehicle [35].
For NO x the differences were ±15% (absolute emission levels were not reported). For PN, recent studies also found differences <25%, on average, with maximum differences <40% [36][37][38]. The main message from all these results is that the permissible tolerances in the regulation regarding differences between PEMS and laboratory equipment should be tighter. On the one hand, the PEMS perform well; on the other hand, as the emission limits will be lower toward near-zero-emission vehicles, the error margin of the PEMS needs to be lower. CO 2 is the gas with the smallest variability. Table 3 summarizes the results from the two ILCEs with the vehicle of this study considering a different number of laboratories. In 2018 the bags mean was between 139.1 g/km (16 labs) and 140 g/km (7 labs), and in 2019 between 137.8 g/km (5 labs) and 139.8 g/km (6 labs). With PEMS the means varied between 139.6 g/km (5 labs) and 140.8 g/km (7 labs) in 2018 and 138.4 g/km (5 labs) and 139.7 g/km (6 labs) in 2019. The variability (reproducibility in these cases) was 2.1% to 3.8% for bags and 2.1% to 4.7% for PEMS, indicating no increase with PEMS in 2019. The larger PEMS CO 2 variability in 2018 was due to two laboratories. One of them used a more suitable (smaller) flow meter in 2019, and the variability decreased. The variability of CO 2 is an indirect measure of the exhaust flow meter (EFM) variability and uncertainty, because typically CO 2 measurements agree within 2%. The measured variability of ±5 g/km translates into 3.5% for the 140 g/km CO 2 emission of this vehicle. Other researchers also found the agreement between PEMS and bags to be better than 5% [35,39]. This uncertainty is much lower than the 7.5-10% currently assumed for PEMS in the JRC assessment of the measurement uncertainty of PEMS [13,34]. This result indicates that the EFM uncertainty should be re-assessed with more and newer data. Table 4 summarizes the reproducibility results of inter-laboratory exercises with dieselfueled vehicles and our 2018 and 2019 exercise (our detailed results are given in Table A1 of Appendix A). The CO 2 reproducibility was typically 1.5-2.5%, 10-40% for CO, 10% for NO x , and >25% for PN. Some exceptions (e.g., 58% for PN and 29% for NO x ) have to do with the particularities of the specific vehicles, the vehicle pre-conditioning, and the low emission levels. The reproducibility levels of this study are in line with the previous values. Even though we are moving toward sustainable technologies, such as electric vehicles, the internal combustion engine will remain at least for another 15 years. It is necessary that the emissions reach near-zero levels. The question that arises is how accurately the instruments can measure at such low levels. This study assessed levels ten times lower than the limit for CO, half for NO x , but close to the limit for PN. Especially for NO x and PN, further studies are necessary at much lower levels. The CO 2 (and fuel consumption), which is an important greenhouse gas, can still be measured very accurately. This study clearly showed that PEMS measure accurately at levels below the current limits, and the values of this study can be used as an input for researchers and regulators for the future assessment of the limits.

Conclusions
In this study two inter-laboratory comparison exercises (in 2018 and 2019) were conducted with a 1.6 L diesel Euro 6d-temp vehicle. The test cycle was the WLTC (worldwide harmonized light vehicles test cycle). Seven labs in 2018 and six in 2019, in addition to the regulated procedures with bags, used their own PEMS (portable emissions measurement systems) to measure the emissions.
The results of this study showed that PEMS can accurately measure the emissions with a variability quite similar to that of the laboratory grade equipment, at least under controlled laboratory ambient conditions. Furthermore, they support a reduction of the permissible tolerances in the regulation for the differences between PEMS and laboratory grade equipment.   The CO2 emissions followed the speed trace. The NOx emissions were relatively high at the beginning of the cycle, but gradually decreased to very low levels, especially when The CO2 emissions followed the speed trace. The NOx emissions were relatively high at the beginning of the cycle, but gradually decreased to very low levels, especially when The CO 2 emissions followed the speed trace. The NO x emissions were relatively high at the beginning of the cycle, but gradually decreased to very low levels, especially when the SCR (selective catalytic reduction for NO x ) was at the appropriate temperature. Some spikes stand out during accelerations around 600-900 s because the SCR was not fully warmed up. The PN emissions were relatively high, close to the PN limit, which is not so typical for DPF-equipped vehicles. For this reason, the PN emissions followed the speed signal. The CO emissions were low, with some emissions at the cold start and during accelerations and incomplete combustion. Table A1. Mean values, within-lab variability (repeatability) (s r ), between-labs variability (s L ), and reproducibility (s R ) for the laboratory equipment (bags) and the PEMS. The values are given separately for the two ILCEs in 2018 (7 labs) and 2019 (6 labs) for various pollutants.