Air Quality Sensor Networks for Evidence-Based Policy Making: Best Practices for Actionable Insights

Abstract

(1) Background: This work evaluated the usability of commercial “low-cost” air quality sensor systems to substantiate evidence-based policy making. (2) Methods: Two commercially available sensor systems (Airly, Kunak) were benchmarked at a regulatory air quality monitoring station (AQMS) and subsequently deployed in Kampenhout and Sint-Niklaas (Belgium) to address real-world policy concerns: (a) what is the pollution contribution from road traffic near a school and at a central city square and (b) do local traffic interventions result in quantifiable air quality impacts? (3) Results: The considered sensor systems performed well in terms of data capture, correlation and intra-sensor uncertainty. Their accuracy was improved via local re-calibration, up to data quality levels for indicative measurements as set in the Air Quality Directive (U_exp < 50% for PM and <25% for NO₂). A methodological setup was proposed using local background and source locations, allowing for quantification of the (3.1) maximum potential impact of local policy interventions and (3.2) air quality impacts from different traffic interventions with local contribution reductions of up to 89% for NO₂ and 60% for NO throughout the considered 3 month monitoring period; (4) Conclusions: Our results indicate that commercial air quality sensor systems are able to accurately quantify air quality impacts from (even short-lived) local traffic measures and contribute to evidence-based policy making under the condition of a proper methodological setup (background normalization) and data quality (recurrent calibration) procedure. The applied methodology and learnings were distilled in a blueprint for air quality sensor networks for replication actions in other cities.

1. Introduction

Despite overall improvements in air quality, air pollution still presents major health concerns [1]. This is evidenced by the recent tightening of the health-based air quality guidelines set by the World Health Organization (WHO) [2]. In the European Union, it is estimated that 96% of the urban population is exposed to levels of fine particulate matter above these new WHO guidelines. Especially in urban environments, pollution hotspots are generated and tend to vary greatly in both time and space, which requires dedicated policy measures in certain targeted areas. As road traffic represents an important source of, e.g., NO_x, PM, UFP and BC [3,4,5,6,7,8,9,10,11,12,13,14,15], many air quality policies (e.g., low emission zones, speed reductions, school streets) are targeted at reducing local traffic density to improve the local air quality, safety and overall wellbeing of city dwellers [16,17,18,19,20,21,22]. Quantifying the resulting air quality impacts from local policy interventions is often hard, as there are few (or none) regulatory air quality monitoring stations (AQMS) available, and dedicated monitoring campaigns are difficult and costly to set up using traditional monitoring equipment. Moreover, fixed air quality monitoring stations (AQMS) are mostly not installed at representative locations to assess the impact of traffic interventions.

Recent advances in sensor and Internet of Things (IoT) technologies have resulted in a wide range of commercially available “lower-cost” sensor systems that allow for quantification of relevant urban pollutants, e.g., particulate matter (PM_x), nitrogen dioxide (NO₂) and ozone (O₃), at a much higher spatiotemporal resolution [23,24,25]. These tools allow regional or local authorities to set up dedicated monitoring campaigns more easily. However, these lower-cost sensor technologies typically suffer from a lower accuracy when compared to the regulatory equivalent or reference methods. Moreover, they can be sensitive towards environmental conditions and other pollutants and experience sensor drift over time [25,26,27,28,29]. In the literature, a wealth of studies is available on the lab- or field-based evaluation of sensors [30,31,32,33,34,35,36,37,38,39,40,41], co-location or network calibration approaches [25,26,28,42,43,44,45,46,47,48,49], personal exposure [3,50,51,52] and mapping [53,54,55,56,57,58] proof of concept applications. These studies often conclude that air quality sensors have the potential for policy support, but fall short in addressing real-life policy concerns or in indicating how such a monitoring campaign could be set up methodologically or controlled in terms of data quality. This work presents a collaborative effort of a research institute (VITO), local authorities from three municipalities and two cities and the regional environmental agency (VMM) to evaluate whether contemporary sensor solutions can be used to quantify air quality impacts from real-world traffic interventions. Air quality impacts from traffic management scenarios have been shown before, using expensive high-end equipment [22,59,60,61] or modeling [16,62,63], both of which require substantial user experience. We wondered to what extent cities or municipalities can apply more affordable sensor solutions themselves as tools for evidence-based policy making. In a prior co-creational research trajectory, an inventory of available air quality sensor systems was created following a literature and market search. Contemporary air quality concerns and network requirements were identified in a series of workshops and questionaries with involved municipalities and cities, and finally two pilot sensor networks were deployed in Kampenhout and Sint-Niklaas. This paper explores the usefulness of air quality sensors for evidence-based policy making by (1) evaluating sensor and intra-sensor performance and stability over time and (2) testing two real-life use cases to quantify the local pollution contribution from road traffic and potential air quality impacts from different traffic management scenarios.

Within the following paragraphs, the selection of the considered sensor systems is described in Section 2, followed by the setup of the co-location campaigns to evaluate the data quality of the sensor systems (Section 2.1.1 and Section 2.1.2) and subsequent local calibration (Section 2.1.3). The aim and setup of the pilot deployments in Kampenhout and Sint-Niklaas is described in Section 2.2 and Section 2.3, respectively. Results of the co-location campaigns (Section 3.1) and subsequent Kampenhout (Section 3.2) and Sint-Niklaas (Section 3.3) pilots are provided in Section 3 and discussed in Section 4.

2. Materials and Methods

This study considers a field benchmarking study of two commercially available sensor systems and subsequent application in two pilot deployments. In Kampenhout (BE), the municipality was interested in the air quality impact from road traffic restrictions (“school street”) near an elementary school. In Sint-Niklaas (BE), the city wanted to gain insight in the air pollution contribution from road traffic at the central city square (Grote Markt), as a baseline assessment before implementing a new traffic management plan. Requirements of both sensor networks, therefore, focused on traffic-related pollutants (PM, NO₂) in an outdoor environment. A set of minimal functional requirements for the sensor networks is provided in

Table 1. Functional requirements of the air quality sensor network.

Pollutants	NO2, PM1, PM2.5, PM10
Time period	3 months/pilot
Additional metrics	Temperature (°C), Relative humidity (%)
Number of sensors/locations	Minimum 3 (preferably 5)
Housing	Outdoor, weather resistant
Power	via solar panel
Communication	No WiFi available → via SIM/GPRS
Data quality	High quality needed: Good comparability (precision) between sensor boxes

.

After conducting a literature search and market analysis on commercially available sensor systems [64] and comparing sensor system specifications to the defined functional requirements (Supplementary Table S1), we decided to proceed with Kunak Air A10 (Kunak Technologies, Navarra, Spain; ~€7000/unit; purchase) and Airly PM + Gas sensor (Airly, Kraków, Poland; ~€300/unit; rental). The Kunak Air A10 holds Alphasense (Essex, UK) electrochemical sensors for NO₂, NO and O₃, holds an OPC-N3 (Alphasense, Essex, UK) for PM₁, PM_2.5 and PM₁₀, and provides meteorological data on external temperature, relative humidity and pressure. The Airly PM + Gas sensor holds Alphasense (Essex, UK) electrochemical sensors for NO₂ and O₃, holds a PMS5003 (Plantower, Beijing, China) for PM₁, PM_2.5 and PM₁₀, and provides meteorological data on external temperature and relative humidity. Further sensor specifications are provided in Supplementary Table S2. Both sensor systems fulfilled our minimal requirements but were significantly different in terms of pricing (commonly observed in the air quality sensor market [64]) and target users (more scientific targeted versus community groups). In consultation with the municipalities, we decided to select one sensor system at each side of the (cost) spectrum to evaluate both qualitative (user-friendliness, configuration options, transparency, etc.) and quantitative sensor performance (accuracy and precision).

The study consisted of an initial co-location campaign of 2 weeks, co-locating all 6 sensor systems (3 of each brand/type) at an urban background air quality monitoring station (AQMS) of the Flemish Environmental Agency. After this co-location, the first pilot was conducted in Kampenhout (12 February 2021–17 May 2021). This pilot was immediately followed by a second 2-week co-location campaign (co-location 2) at the same AQMS and the second pilot study in Sint-Niklaas (10 June 2021–15August 2021). After finishing both pilot studies, we performed a final 2-week co-location campaign (co-location 3) at the AQMS (Figure 1). The recurrent co-location campaigns at a regulatory AQMS allowed for (1) performance (accuracy and precision) evaluation over time, (2) application of linear correction functions to improve sensor performance before each pilot and (3) evaluation of sensor drift and sensitivity to meteorological conditions over time.

2.1. Co-Location Campaigns

During each co-location campaign, the 6 sensor units (3 Kunaks, 3 Airlys) were co-located for two weeks on the roof of a regulatory AQMS in Antwerp, Belgium (51°12′34.82″ N, 4°25′54.51″ E), near the inlets of the reference monitor for NO and NO₂ (42C NO-NO₂-NO_x analyzer, Thermo) and equivalent (automatic) monitor for PM (FIDAS 200, Palas). In total, we considered three co-location campaigns in order to be able to compare the sensor and intra-sensor performance in different seasons and under varying environmental conditions. Moreover, potential drift or aging were considered by evaluating the accuracy over the considered time period (11 months in total).

2.1.1. Sensor Performance

We evaluated the comparability between the sensor systems and the reference (NO_x) or equivalent (PM) method by calculating the following performance metrics on an hourly and daily averaging time:

• Coefficient of determination (R²): value between 0 and 1, where 1 is the best possible value. When the R² is equal to 1, the measurements are on a perfect line, and this means that all variance in the senor data can be explained by the measured reference concentrations.
• Pearson correlation (COR): value between −1 and 1 that represents the degree of correlation between sensor and reference measurements.
• Root Mean Squared Error (RMSE): Root of the mean squared error of the measurement error between sensor and reference data; it is a frequently applied accuracy statistic and is very dependent on peaks/outliers.
• Mean Absolute Error (MAE): The mean absolute error between sensor and reference data; can be interpreted as the mean deviation between sensor and reference.
• Mean Bias Error (MBE): Mean error between sensor and reference data. This metric can be both positive and negative and represents respectively the degree of over- or underestimation of the sensor.
• Expanded Uncertainty: Measure for the uncertainty (%) around the limit or target value, as defined by the EU [65]. We calculate the non-parametric approach (Uexp), proposed by the Flanders Environmental Agency, to calculate the uncertainty near 50 µg m⁻³ for PM₁₀, 30 µg m⁻³ for PM_2.5 and 40 µg m⁻³ for NO₂. Although both approaches aim at quantifying the sensor uncertainty near the limit value, the EU approach used in the Demonstration of Equivalence (DOE), the relative expanded uncertainty for the candidate method (Wcm) is derived from the logistic regression between sensor and reference data (model derivation), while the non-parametric approach (Uexp) is quantified experimentally (95 percentile MAE of measured concentrations within 10% range of the regulatory limit/target concentration).

In addition to the calculated performance metrics, we explored the temporal variability of the sensor and reference data using TimeVariation graphs (openair package in R [66,67]) and the sensor sensitivity to environmental conditions (temperature, relative humidity and ozone), as these variables are known to impact the performance of low-cost optical PM and electrochemical NO₂ sensors [24,25,27,29,32,33,68,69,70] and need proper compensation in order to provide reliable results [25,26,28,42,45,71].

2.1.2. Intra-Sensor Performance

As the comparability between the sensor units is key to compare concentrations from multiple sensor locations against each other, the precision was evaluated by calculating the following:

• Min–Max correlation between sensor units of the same brand (Kunak, Airly);
• Min–Max MAE between sensor units of the same brand (Kunak, Airly).

2.1.3. Local Sensor Calibration

Besides evaluating the sensor performance, we explored whether a local re-calibration increased the sensor and intra-sensor performance. As the sensor systems already included calibration algorithms, we didn’t consider complex multivariate calibration models (as applied in Hofman et al. [25]), but relied on linear rescaling factors, derived from the slope of the linear regression function (intercept = 0) between the reference and sensor data, as shown in Equations (1) and (2):

$$C_{SENSOR\_RAW}=a\ast C_{REF}$$

(1)

$$C_{SENSOR\_CAL}=\frac{C_{SENSOR\_RAW}}{a}$$

(2)

where $C_{SENSOR\_RAW}$ and $C_{REF}$ represent the measured concentrations of the sensor and reference equipment (AQMS), respectively, and $a$ is the derived slope from the linear regression function between the raw sensor and reference data (Equation (1)). The calibrated sensor data ($C_{SENSOR\_CAL}$) can subsequently be calculated by dividing the raw sensor data by the derived slope (a) from Equation (1) (Equation (2)). Former studies already showed improved performance after local re-calibration [45,46,72,73,74], as the sensor measurements are adjusted for local aerosol properties (size distribution, composition and reflectance) and environmental conditions.

2.2. Kampenhout Pilot

In Kampenhout, three monitoring locations were considered (Figure 2):

• School: Roadside location in front of the school (50°56′43.90″ N, 4°35′11.80″ E) where air quality impacts from implemented traffic measures are expected.
• Environment: Roadside location on adjacent street (50°56′41.34″ N, 4°34′51.32″ E) with similar traffic as school location and not impacted by implemented traffic measures.
• Background: Quiet location behind the school (50°56′46.16″ N, 4°35′10.46″ E) where no road traffic was present and, therefore, regarded as representative for background pollution and local sources other than road traffic.

At each monitoring location, both a Kunak and Airly unit were installed on 12 February 2021. PM, NO and NO₂ concentrations were monitored for three months, until 17 May 2021. Different traffic scenarios (baseline, knip, schoolstreet) were implemented during school opening (8 h 10–8 h 40) and closing (15 h 00–15 h 45 and 11 h 00–11 h 45 on Wednesday) hours:

• Baseline scenario: No traffic restrictions, resulting in unaltered traffic flows.
• Knip scenario: one-way traffic cut in the street where the school was located (green in Figure 2), alternately in the western driving direction during school start times and the eastern direction during school end times.
• Schoolstreet scenario: 2-way traffic restriction in the street of the school and the perpendicular street (blue in Figure 2).

Each traffic scenario was implemented for about 3 weeks, while the monitoring period also included holidays (3 weeks school holidays + 1 additional week of school closure due to the COVID-19 pandemic). The timing of the implemented traffic scenarios is detailed in Supplementary Table S3. Operational performance of the sensor network (QA/QC) during the pilot was evaluated by checking the connectivity and potential operational alerts of the sensors twice a week in the online Kunak and Airly cloud dashboards.

By comparing the measured pollutant concentrations at the school location against the background location for each of the tested traffic scenarios, we were able to quantify the resulting air quality impact and identify the most optimal scenario resulting in lowest pollutant levels at the school entrance.

2.3. Sint-Niklaas Pilot

In Sint-Niklaas, three monitoring locations were considered, with one Kunak and Airly sensor unit deployed at each location (Figure 3):

• Background: Quiet location near the main city square (51°10′1.27″ N, 4° 8′21.70″ E) in a car-free street (Paul Snoekstraat), therefore, regarded as representative for background pollution and local sources other than road traffic. Nearest (quiet) traffic was at 50 m (Collegestraat) and 40 m (Boonhemstraat).
• Grote Markt 1: Roadside location at the eastern side (Apostelstraat) of the central city square (51° 9′50.49″ N, 4° 8′28.33″ E).
• Grote Markt 2: Roadside location at the western side (Nieuwstraat) of the central city square (51° 9′50.41″ N, 4° 8′22.46″ E).

At each monitoring location, both a Kunak and Airly unit were deployed on 10 June 2021. PM, NO and NO₂ concentrations were monitored for three months, until 15 August 2021. From the resulting difference between the roadside locations at the central city square (Grote Markt 1 and 2) and the background location, the local traffic-related pollution contribution was quantified for each pollutant.

Operational performance of the sensor network (QA/QC) during the pilot was evaluated by checking the connectivity and potential operational alerts of the sensors twice a week in the online Kunak and Airly cloud dashboards. By comparing the measured concentrations at the central city square locations against the background location, we were able to quantify the road traffic contribution at the central city square and potential air quality impact from future traffic restrictions.

3. Results

3.1. Co-Location Campaigns

3.1.1. Sensor Performance

Co-location measurements (before rescaling) are visualized in timeseries graphs for the main traffic-related pollutants NO₂, NO and PM₁ for each of the Kunak (Figure 4) and Airly (Supplementary Table S3) sensors. From these time series, it becomes clear that the sensor measurements follow the reference measurements reasonably well, resulting in good correlations between sensor and reference data for NO₂ (r = 0.96–0.97) and NO (r = 0.94–0.99) from Kunak and for PM₁ (r = 0.91–0.94), PM_2.5 (r = 0.89–0.92) and PM₁₀ (r = 0.72–0.75) from Airly (

Table 2. Data quality metrics (R², COR, RMSE, MAE, MBE, Uexp) calculated on the raw and locally re-calibrated (_cal) hourly-aggregated NO₂, NO, PM₁, PM_2.5 and PM₁₀ data for each of the sensor units (3 × Kunak and 3 × Airly). Local re-calibration is based on slope correction (sensor = a ∗ REF; Equation (1)), except for the NO₂ of Airly (NO₂_cal*) where a multilinear model was applied (sensor = a ∗ REF + b ∗ Temp + c). Uexp was only calculated for pollutants with existing limit values (NO₂, PM_2.5 and PM₁₀) and, therefore, not for PM₁ and NO (NA).

	NO2			NO2_cal			NO2			NO2_cal*
	Kunak_1	Kunak_2	Kunak_3	Kunak_1	Kunak_2	Kunak_3	Airly_8463	Airly_12744	Airly_12798	Airly_8463	Airly_12744	Airly_12798
R2	0.92	0.94	0.95	0.92	0.94	0.95	0.00	0.69	0.02		0.84
COR	0.96	0.97	0.97	0.96	0.97	0.97	NA	0.83	0.14		0.92
RMSE	10.30	7.81	10.26	5.42	4.37	5.89	37.17	123.73	37.16		7.08
MAE	9.04	6.42	9.50	4.01	3.38	4.67	33.44	123.30	33.44		5.29
MBE	−9.04	−6.38	−9.50	−0.74	1.04	−1.61	−33.44	123.30	−33.44		0.00
Uexp	42.95	33.00	39.66	26.20	25.11	26.77	110.07	364.54	110.07		36.82
	NO			NO_cal
	Kunak_1	Kunak_2	Kunak_3	Kunak_1	Kunak_2	Kunak_3
R2	0.98	0.90	0.88	0.98	0.90	0.88
COR	0.99	0.95	0.94	0.99	0.95	0.94
RMSE	3.14	3.13	3.39	1.53	3.15	3.52
MAE	2.21	2.52	2.66	1.20	2.54	2.77
MBE	−2.16	0.58	0.64	−0.91	0.88	1.12
Uexp	27.76	15.71	15.71	8.87	12.27	12.27
	PM1			PM1_cal			PM1			PM1_cal
	Kunak_1	Kunak_2	Kunak_3	Kunak_1	Kunak_2	Kunak_3	Airly_8463	Airly_12744	Airly_12798	Airly_8463	Airly_12744	Airly_12798
R2	0.73	0.73	0.75	0.73	0.73	0.75	0.85	0.87	0.82	0.85	0.87	0.82
COR	0.86	0.85	0.87	0.86	0.85	0.87	0.92	0.94	0.91	0.92	0.94	0.91
RMSE	10.67	10.83	11.01	4.75	4.87	4.83	4.85	3.14	5.31	3.01	2.75	3.35
MAE	9.05	9.23	9.47	3.59	3.71	3.65	3.91	2.50	3.95	2.33	2.16	2.30
MBE	−9.05	−9.23	−9.47	0.24	0.03	−0.17	3.30	1.39	3.35	0.49	0.44	0.40
Uexp	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
	PM2.5			PM2.5_cal			PM2.5			PM2.5_cal
	Kunak_1	Kunak_2	Kunak_3	Kunak_1	Kunak_2	Kunak_3	Airly_8463	Airly_12744	Airly_12798	Airly_8463	Airly_12744	Airly_12798
R2	0.28	0.22	0.25	0.28	0.22	0.25	0.81	0.84	0.79	0.81	0.92	0.79
COR	0.53	0.47	0.50	0.53	0.47	0.50	0.90	0.92	0.89	0.90	0.96	0.89
RMSE	9.17	9.62	9.42	9.94	10.84	10.14	16.37	11.72	16.45	3.69	1.58	3.93
MAE	7.51	8.04	7.79	7.73	8.24	7.91	13.58	9.63	13.45	3.00	1.11	3.10
MBE	−5.65	−5.61	−5.84	1.14	1.23	1.06	13.06	8.63	12.92	−0.01	0.43	−0.02
Uexp	68.31	70.53	69.85	91.33	93.94	84.63	97.75	71.08	96.67	18.67	NA	16.54
	PM10			PM10_cal			PM10			PM10_cal
	Kunak_1	Kunak_2	Kunak_3	Kunak_1	Kunak_2	Kunak_3	Airly_8463	Airly_12744	Airly_12798	Airly_8463	Airly_12744	Airly_12798
R2	0.28	0.22	0.25	0.15	0.11	0.12	0.55	0.56	0.52	0.55	0.56	0.52
COR	0.53	0.47	0.50	0.38	0.33	0.34	0.74	0.75	0.72	0.74	0.75	0.72
RMSE	9.17	9.62	9.42	13.02	14.11	13.63	20.37	13.71	19.99	8.26	8.94	8.71
MAE	7.51	8.04	7.79	11.47	12.52	12.10	16.28	11.28	15.85	6.83	7.36	7.20
MBE	−5.65	−5.61	−5.84	−6.11	−7.10	−6.65	13.55	5.81	12.70	−0.31	−0.62	−0.31
Uexp	68.31	70.53	69.85	39.91	47.14	43.75	47.00	47.00	47.00	24.24	24.24	24.24

), with adequate compensation for potential temperature and relative humidity effects (tested but not shown). Nevertheless, considerable under- or overestimation can be observed for some pollutants/sensors, resulting in low accuracies, e.g., for PM₁ from Kunak (MAE ~9 µg/m³).

An overview of associated data quality metrics (R², COR, RMSE, MAE, MBE, Uexp) for co-location campaign 1 based on hourly values is presented for NO₂, NO, PM₁, PM_2.5 and PM₁₀ in Table 2. The same statistics were also calculated on daily-aggregated data and are provided in Supplementary Table S4. Table 2 shows data quality metrics (R², COR, RMSE, MAE, MBE, Uexp) calculated on the raw and locally re-calibrated (_cal) hourly-aggregated NO₂, NO, PM₁, PM_2.5 and PM₁₀ data for each of the sensor units (3 × Kunak and 3 × Airly). Local re-calibration is based on slope correction (sensor = a ∗ REF; Equation (1)), except for the NO₂ of Airly (NO₂_cal*), where a multilinear model was applied (sensor = a ∗ REF + b ∗ Temp + c). Uexp was only calculated for pollutants with existing limit values (NO₂, PM_2.5 and PM₁₀) and therefore not for PM₁ and NO (NA).

As both the Airly and Kunak unit hold the same electrochemical NO₂ sensor (Alphasense NO2-B43F), it was quite surprising to see the difference in sensor performance (similar over the multiple co-location campaigns), probably related to differences in the applied property algorithms to calibrate the sensors and compensate for environmental impacts. The better PM performance observed for the Airly units can be explained by either the difference in PM sensors (PPMS5003 (Plantower, Beijing, China) vs. OPC-N3 (Alphasense, Essex, UK)) or the associated calibration and compensation algorithms.

3.1.2. Intra-Sensor Performance

From Figure 4, low intra-sensorvariability (precision of the sensor units) can be observed, which is very important for the intended purpose of our pilots, since we want to compare different sensor units (locations) against each other. Good intra-sensor performance was confirmed when calculating the Min–Max correlation and MAE between the considered sensor units (

Table 3. Minimum and Maximum Pearson correlations (COR) and Mean Absolute Errors (MAEs) between the considered sensor units for NO₂, NO, PM₁, PM_2.5 and PM₁₀ before (RAW) and after (CAL) local re-calibration.* During the final co-location, only two Airly sensors were available.

			Kunak				Airly
			NO2		NO		PM1		PM2.5		PM10
			RAW	CAL	RAW	CAL	RAW	CAL	RAW	CAL	RAW	CAL
CO-LOCATION 1 22 January 2021–10 February 2021	COR	MIN	0.978	0.978	0.963	0.963	0.964	0.964	0.986	0.986	0.984	0.984
		MAX	0.997	0.997	0.983	0.983	0.984	0.984	0.989	0.989	0.987	0.987
	MAE	MIN	2.597	2.067	0.795	0.758	1.336	0.850	1.424	0.950	2.029	1.860
		MAX	3.631	4.244	2.953	2.491	2.121	2.996	4.445	13.167	7.742	13.327
			NO2		NO		PM1		PM2.5		PM10
			RAW	CAL	RAW	CAL	RAW	CAL	RAW	CAL	RAW	CAL
CO-LOCATION 2 18 May 2021–8 June 2021	COR	MIN	0.986	0.986	0.837	0.837	0.837	0.837	0.921	0.921	0.920	0.920
		MAX	0.993	0.993	0.978	0.978	0.915	0.915	0.959	0.959	0.959	0.959
	MAE	MIN	1.931	2.227	0.741	0.597	0.741	1.479	1.004	1.456	1.387	2.639
		MAX	3.542	2.606	2.197	1.823	1.013	4.923	1.364	3.377	2.162	8.207
			NO2		NO		PM1 *		PM2.5 *		PM10 *
			RAW	CAL	RAW	CAL	RAW	CAL	RAW	CAL	RAW	CAL
CO-LOCATION 3 21 September 2021–14 November 2021	COR	MIN	0.983	0.983	0.982	0.982	0.814	0.814	0.911	0.911	0.926	0.926
		MAX	0.994	0.994	0.990	0.990
	MAE	MIN	4.753	2.365	0.629	0.859	1.452	2.242	1.690	1.924	2.767	3.152
		MAX	10.536	4.634	2.244	2.422

), showing that the agreement between the sensor units is better than the agreement between the sensor and reference measurements. Moreover, correlations and MAEs were also similar between the three co-location campaigns (Table 3), meaning that intra-sensor performance does not deteriorate over the considered time period (11 months).

3.1.3. Local Re-Calibration

As the linear correlations between the sensor and reference data were generally good (mostly r > 0.9) and did not exhibit temperature and/or relative humidity effects (not shown) for NO₂ and NO (Kunak) and PM₁, PM_2.5 and PM₁₀ (Airly), and a significant amount of residual variation can be explained by the slope, slope calibration factors were implemented as explained in Section 2.1.3 and shown in Figure 5. We derived slope calibration factors for each pollutant and sensor and calculated the sensor performance again based on the re-calibrated sensor data (“_cal” in Table 2). This exercise resulted in an improved accuracy (RMSE, MAE, MBE, Uexp) for all pollutants and sensors and similar correlations (R² and COR), as can be seen from Table 2 and Supplementary Figure S2. While the expanded uncertainty of the original sensor data did not reach the data quality objectives (DQOs) for indicative measurements (50% for PM, 25% for NO₂) as defined at the EU level [65], the DQOs were met after slope re-calibration for Airly PM_2.5 and PM₁₀ and Kunak PM₁₀ (<50%) and approximated for Kunak NO₂ (~25%), as can be seen from Table 2.

This re-calibration approach based on slope factors improved the sensor performance for all pollutants/sensors, except for the Airly NO₂, where an additional sensitivity towards atmospheric temperature (°C) was observed (Supplementary Figure S3). By considering a multilinear re-calibration model compensating for both slope and temperature (covariates), we were able to explain the residual variability and improve the sensor performance (Supplementary Figure S3). This was, however, out of scope for this study, as we wanted to evaluate out-of-the-box performance and added value of easily-derived calibration factors for the local user communities.

In conclusion, the considered sensor systems generally performed quite well and followed the pollutant dynamics of the reference measurements. Nevertheless, a local calibration (rescaling) is advisable to improve the sensor accuracy. While Airly performs best for PM, Kunak gives the best results for NO₂ and NO. Note that the evaluation is based on sensor system versions available at the time of the study, and improvements of newer versions can be expected. After implementing the local re-calibration factors, we reached overall good performance for Kunak NO₂ (R² = 0.92–0.95, MAE = 3.38–4.67 µg m⁻³) and NO (R² = 0.88–0.98, MAE = 1.2–2.77 µg m⁻³) and Airly PM₁ (R² = 0.82–0.87, MAE = 2.16–2.33 µg m⁻³) and PM_2.5 (R² = 0.79–0.92, MAE = 1.11–3.1 µg m⁻³). Taking into account the observed data quality and the implemented policy measures (traffic restrictions) in Kampenhout and Sint-Niklaas, further data analysis of the pilots will focus on the best performing sensor systems for traffic-related pollutants NO (Kunak), NO₂ (Kunak) and PM₁ (Airly).

When comparing the sensor performance (Supplementary Table S6) and derived slope factors (Supplementary Table S5) between the recurrent co-location campaigns, it becomes clear that both the raw sensor performance and derived slopes tend to vary throughout the year. The raw NO₂ sensor performance reduces in the summer period (second co-location), possibly due to impacts from temperature and ozone, while PM performance decreases during the winter first and third co-location), probably related to a higher relative humidity. While the raw PM sensor readings underestimated PM concentrations during the summer (co-location 2), they overestimated PM concentrations during the winter (co-locations 1 and 3), as can be seen from Figure 6.

This seasonal variation of environmental parameters, therefore, tends to impact the sensor performance. However, after applying the slope re-calibration, similar sensor performances are achieved to the first co-location period (Supplementary Table S6), indicating that (1) recurrent co-location calibrations can cope with changing environmental impacts and (2) no significant sensor aging or drift is observed in the considered study period (11 months).

3.2. Kampenhout Pilot

3.2.1. Descriptive Statistics and Temporal Variability

As described in Section 2.2, PM, NO and NO₂ concentrations were monitored at three locations (background, school and environment) between 12 February 2021 and 17 May 2021. Slope re-calibration factors from the preceding co-location campaign (co-location 1 in Supplementary Table S5) were applied to the sensor data, and descriptive statistics of the exhibited pollutant concentrations and meteorological conditions (temperature, relative humidity and pressure) are provided in Supplementary Table S7. Data coverage during the pilot was good (>95%), and meteorological conditions were very similar between the considered locations. This is not surprising, as all locations are located within a 400 m radius.

Time series graphs in Supplementary Figure S4 show realistic concentration dynamics throughout the monitoring campaign, and diurnal pollutant variability with distinct morning and evening rush hour peaks were observed for NO₂, NO and PM₁ (Figure 7). From the diurnal pattern of the monitoring locations and the implemented traffic scenarios, it becomes clear that similar pollutant dynamics are observed at all monitoring locations, with highest concentrations at the environment location (roadside, not impacted), followed by the school location (roadside, impacted) and the background location (no traffic). During school start times, concentration differences between the background and school location are highest for NO (16 µg m⁻³), followed by NO₂ (4 µg m⁻³) and are only moderate for PM₁ (<1 µg m⁻³). This can be explained by the fact that NO is emitted by traffic and rapidly oxidizes to NO₂. NO can, therefore, be regarded as a short-lived pollutant, resulting in steep concentration gradients near roads (difference in background vs. school/environment location in Figure 7). The NO₂ concentrations result from primary emissions and converted NO. NO₂ can be transported over longer distances, and the ratio of NO/NO₂ is the result of a complex photochemistry, resulting in a less direct relation of NO₂ to traffic compared to NO. PM is predominantly influenced by background pollution and can be generated by a wide range of direct and indirect emission sources.

During school start (8–9 h) and end (15–16 h) times, increased NO and NO₂ pollutant concentrations at the school location are observed in Figure 7, most clearly during the baseline (no traffic restriction), to a lesser extent during the knip scenario (one-way traffic restriction) and to a negligible extent during the schoolstreet scenario and holiday periods. For PM₁, it is hard to observe any difference in concentrations between the scenarios based on the diurnal graphs (Figure 7). Lowest overall NO₂, NO and PM₁ concentrations are observed during the school holiday periods. Although these observations already suggest an air quality impact from the implemented traffic scenarios, actual scenario differences are calculated in Section 3.2.2.

3.2.2. Traffic Scenario Differences

As the traffic scenarios were only implemented on weekdays (n = 1776), during school start and end times (n = 161), we extracted this data from the overall hourly-aggregated dataset. When comparing the observed pollutant variability of these data subsets between the traffic scenarios (baseline, knip, schoolstreet) and holiday periods at the school location using boxplots and Wilcoxon Rank significance tests (Supplementary Figure S5), significant concentration differences were only observed for NO. However, scenario differences were mimicked due to the varying background concentration. Higher background concentrations were observed for NO₂ (+28%), NO (+44–66%) and PM₁ (+123–185%) during the implemented traffic scenarios, when compared to the baseline scenario (Supplementary Figure S6). In order to take into account background concentration variability, we normalized the school location concentrations using the concentrations observed at the background location, both absolute (µg m⁻³; Equation (3)) and relative (%; Equation (4)), for comparison:

$$C_{norm}=C_{school}-C_{background}$$

(3)

$$C_{norm}\left(\%\right)=\frac{\left(C_{school}-C_{background}\right)}{C_{background}}\times 100$$

(4)

where $C_{norm}$ is the normalized concentration (µg m⁻³), $C_{school}$ is the hourly-averaged pollutant concentration measured at the school location (µg m⁻³) and $C_{background}$ is the hourly-aggregated concentration simultaneously measured at the background location (µg m⁻³). $C_{norm}$ can, therefore, be regarded as the local pollution contribution at the school location.

Based on the resulting boxplots and descriptive statistics of the normalized pollutant concentrations ($C_{norm}$) in Figure 8, we can state that the average local NO₂ contribution (during the scenarios; 8–9 h and 12–13 h/15–16 h) decreases from 4.33 µg m⁻³ (baseline) to 2.9 µg m⁻³ (−33%) during the knip scenario (one-way traffic restriction) and to ~0.5 µg m⁻³ (−89%) for the schoolstreet scenario (two-way traffic restriction). For NO, the air quality impact is even more significant, from 16.83 µg m⁻³ (baseline) to 10.91 µg m⁻³ (−35%) for the knip scenario and 6.72 µg m⁻³ (−60%) for the schoolstreet scenario. Finally, for PM₁, we see no effects on the scenarios, with a negligible local PM₁ contribution at the school location (0.06–0.31 µg m⁻³). The observed concentration differences between the traffic scenarios exceed the intra-sensor uncertainties quantified in the co-location campaigns (Table 3), and can therefore be explained by the implemented traffic scenarios.

3.3. Sint-Niklaas Pilot

As described in Section 2.3, PM, NO and NO₂ concentrations were monitored at three locations (background, Grote Markt 1 and 2) between 10 June 2021 and 15 August 2021. Slope re-calibration factors derived from the preceding co-location campaign (co-location 2 in Supplementary Table S5) are applied to the sensor data. The resulting average pollutant concentrations for NO₂, NO, PM₁, PM_2.5 and PM₁₀ are provided in

Table 4. Average NO₂, NO, PM₁, PM_2.5 and PM₁₀ concentrations exhibited during the pilot campaign in Sint-Niklaas.

	Background	Grote Markt 1	Grote Markt 2
	µg m−3	µg m−3	µg m−3
NO2	12.8	16.4	19.0
NO	2.2	7.6	10.2
PM1	15.2	16.9	15.2
PM2.5	17.8	19.1	17.8
PM10	31.7	34.2	31.7

. Data coverage during the pilot was again very good (>95%), as can be seen from the time series graphs in Supplementary Figure S7.

The average NO and NO₂ concentrations are clearly higher at the central city square locations (Grote Markt 1 and 2), when compared to the background location (up to 6 µg/m³ higher for NO₂ (+50%) and up to 8 µg/m³ higher for NO (+360%)). For PM₁, a modest concentration increase of 2.5 µg m⁻³ (+8%) compared to the background location is only observed at one of the Grote Markt locations (Grote Markt 1). The location differences for NO and NO₂ are larger than the observed uncertainty between the sensors (Table 3) and should, therefore, be explained by effective location differences. Considering traffic as an important source of NO and NO₂ (confirmed in the Kampenhout pilot) and the car-free background location, our results indicate that increased traffic emissions at the Grote Markt locations give rise to elevated atmospheric NO and NO₂ concentrations.

When evaluating the hourly pollutant variability at the central square (Figure 9), higher morning and evening rush hour peaks are observed for NO₂. From midday onwards, NO₂ reacts with O₂ (+UV and heat), forming NO and O₃ and decreasing NO₂ concentrations. This photochemical balance is inversed in the late evening, when an excess of NO from the evening rush hour peak (in absence of UV from solar radiation) reacts with O₃, forming NO₂ again. From 23 h onwards, NO₂ concentrations are reduced due to the absence of direct emissions. As direct NO emissions from traffic are very short-lived (rapid oxidation towards NO₂) and NO is only formed under sunny conditions, nighttime NO concentrations are very low. The diurnal pattern of NO₂ and NO is, therefore, impacted by both direct emissions and photochemical reactions between NOx and ozone. Hourly PM variability at the central square is very low, as can be seen from Figure 9. Hourly-averaged NO₂ and NO concentrations at each monitoring location are provided in Supplementary Figure S8 and show the most distinct location differences for NO, followed by NO₂.

To focus on the actual traffic contributions at the central square (Grote Markt), we calculated normalized pollutant contributions during the morning rush hour (6–9 h), following Equation (3). Local pollutant contributions (difference between background and central square locations) during the morning rush hour amounted to ~8.4 µg m⁻³ for NO₂ (+30%), ~11.7 µg m⁻³ (+180%) for NO and 1–2 µg m⁻³ (+5%) for PM₁. This effect was pronounced for NO₂ (+43%) and NO (+216%) when only considering working days and dropped to ~0% during the weekends, which confirms our traffic hypothesis. Observed air quality impacts again exceeded between-sensor uncertainties for NO and NO₂, which confirms that observed air quality impacts are due to location differences.

4. Discussion

4.1. Data Quality

The considered sensor systems generally performed quite well in terms of data capture (>95%), correlation and intra-sensor uncertainty. However, good out-of-the-box accuracy was not guaranteed, and a local calibration (rescaling) seems advisable to improve the sensor accuracy to supplementary levels (<50% for PM and <25% for NO₂). Performance was related to both hardware setup (applied sensors) and applied compensation and calibration property algorithms, with Airly performing best for PM and Kunak giving the best results for NO₂ and NO. After implementing the local re-calibration factors, we reached overall good performance for Kunak NO₂ (R² = 0.92–0.95, MAE = 3.38–4.67 µg m⁻³) and NO (R² = 0.88–0.98, MAE = 1.2–2.77 µg m⁻³) and Airly PM₁ (R² = 0.82–0.87, MAE = 2.16–2.33 µg m⁻³) and PM_2.5 (R² = 0.79–0.92, MAE = 1.11–3.1 µg m⁻³). While the expanded uncertainty of the original sensor data did not reach the data quality objectives (DQOs) for supplementary measurements (50% for PM, 25% for NO₂), as defined at the EU level [65], the DQOs were met after slope re-calibration for Airly PM_2.5 and PM₁₀ and Kunak PM₁₀ (<50%) and were approximated for Kunak NO₂ (~25%). High intra-sensor precision was observed for both Kunak and Airly. This is important, as many sensor use cases (like the Kampenhout and Sint-Niklaas pilots) rely on comparisons of multiple monitoring locations (between sensor units).

Although the sensor price range was not necessarily indicative of the data quality (Airly PM performed better than Kunak PM), the more costly Kunak sensors offered much more flexibility, local calibration and configuration options, more data monitoring tools with dedicated warnings (e.g., flow PM sensor), extended analytics based on the statistical openair package [67] and availability of governmental air quality monitoring stations in their cloud dashboard. In contrast, the Airly sensors and interface are more basic, but the price of these sensors is ~10 times lower.

Field co-location campaigns seem vital to (1) evaluate and improve the performance of contemporary sensor systems against actual reference equipment and (2) quantify intra-sensor uncertainty for proper interpretation of the sensor data during their implementation (locations differences > instrument noise). In addition, we noticed that environmental impacts changed throughout the year, resulting in varying re-calibration factors for the three co-location campaigns. Ideal calibration should, therefore, be repeated, e.g., by conducting recurrent co-locations (e.g., every 3 months, at least at locations with large seasonality) [26], as applied in our study. Other (re-)calibration approaches that are proposed in the literature include (1) continuous scaling of co-location-derived calibration factors to a deployed sensor network [44,74,75], (2) distant calibration of sensors based on the existing reference network [25,54,76] or (3) a combination of field and mobile calibration approaches [42].

4.2. Pilots

When considering policy measures to improve local air quality (e.g., via traffic measures), it is important to think about the general build-up of air pollution in cities. Urban air pollution consists of background contributions from cross-boundary, regional and urban sources and local contributions from nearby sources, i.e., traffic (Figure 10). The maximum achievable air quality improvement from local measures is therefore the local contribution, which, at the roadside, showed to be the highest for NO, followed by NO₂ and PM. The use of background monitoring locations proved to be very valuable, as we experienced background concentration fluctuations mimicking the impacts from the implemented traffic measures in one of the pilots (Kampenhout). By considering both a local source and background monitoring location simultaneously, measured concentrations at the source location can be normalized continuously for fluctuating background concentrations. This allows for quantification of (1) the local source contribution (= maximum potential air quality impact of local interventions), and (2) resulting air quality impacts from short-lived traffic interventions (e.g., one-way cut and school street).

Although the sensor locations were fairly close to each other (<400 m), quantifiable location differences were observed in both pilots, exceeding the intra-sensor uncertainty quantified in the co-location campaigns. We can therefore state that the considered sensor systems were able to detect air quality impacts from local traffic in front of a school in Kampenhout and at a central city square in Sint-Niklaas. The local traffic-related pollution contribution (potentially impacted by local traffic measures) in Kampenhout was the highest for NO (90%), followed by NO₂ (24%), and was only negligible for PM₁ (~7%) (Figure 11). In Sint-Niklaas, similar findings were observed with the local rush hour contributions of NO₂ and NO (Supplementary Figure S9). This is important information, as it shows the maximal achievable air quality impacts from local policy measures. In Kampenhout, implemented traffic measures (knip, school street) resulted in a reduction of this local pollution contribution of up to 89% for NO₂ and 60% for NO (Figure 11). Both pilots thus support the use of local traffic measures to improve local air quality in cities.

As we only considered a limited monitoring period in our pilots, 3 months (~3 weeks/traffic scenario) in Kampenhout and 2 months in Sint-Niklaas, and seasonal dynamics in air quality and modal split likely influenced the observed traffic scenario impacts, generalization of the observed results is difficult.

Nevertheless, our results seem to confirm the few existing studies reporting air quality impacts from traffic management scenarios near school environments and urban environments in general. A recent intervention study on the impact from traffic restrictions (school streets) at three schools (high-end instruments) in Belgium reported decreased morning rush hour peaks for NO₂, BC and UFP by comparing concentrations during the traffic intervention (school street) with concentrations from the hour before the intervention [61]. Results showed that the implemented traffic restrictions resulted in an average (morning drop-off) concentration decrease for NOx (from +32 to +77% (pre-) to −23% to +22% (post-intervention)), BC (from +18 to +58% (pre-) to −27% to +38% (post-intervention)) and UFP (from +15 to +19% (pre-) to -9% to +8% (post-intervention)) [61]. Another study with 30 AQMesh sensors at 16 schools in London (UK), comparing schools with traffic restrictions (school street) against schools without traffic restrictions [77], reported average reductions in nitric oxide (NO) concentrations of up to 8 µg/m³ (34%) during the morning intervention period, which equates to a reduction in daily average (school day) concentration of approximately 5%. The resultant reduction in nitrogen dioxide (NO₂) during the school drop-off period was estimated as being up to 6 µg/m³ (23%). The morning intervention alone was thus expected to reduce daily average NO₂ by up to 0.4 µg/m³, or 2%. Moreover, 81% of parents and carers supported the measures at their children’s school, and 18% of parents reported driving to school less as a result of school streets [77]. In terms of the local pollution contribution from road traffic, a recent study with mobile sensors on buses in Hong Kong, determining local and background contributions for NO, NO₂, CO and PM_2.5, showed that NO and NO₂ are locally dominated air pollutants, constituting 72%–84% and 58%–71%, respectively, whereas PM_2.5 and CO largely arise from background sources, which contribute 55%–65% and 73%–79%, respectively [78]. Another recent study showed the distinct traffic-related impacts from the COVID-19 lockdown in the United Kingdom on resulting NO₂ and NO concentrations, with observed concentration reductions of 32 to 50% at urban traffic stations and 26 to 46% at urban background stations, while no concentration reductions (or even increases) were observed for O₃ and PM [59].

Although these studies are all different in terms of implemented traffic interventions (traffic restriction, cut, lockdown, etc.), applied instrumentation (sensors vs. monitors, fixed vs. mobile deployments) and data analysis (normalized based on previous hour [61], normalized based on simultaneous background concentration (our study), normalized based on locations without traffic interventions [77]) and consider different time periods and/or pollutants, there seems to be common evidence that road traffic results in local pollution contributions (NO>NO₂>PM), and local air quality improvements can be expected from when implementing traffic interventions.

4.3. Blueprint for Urban Air Quality Sensor Networks

A total of 79 Belgian municipalities and cities and the Flanders Environmental Agency (VMM) contributed in the research trajectory towards the usability of commercial air quality sensor systems for evidence-based policy making. This trajectory included a literature search and market analysis [64], workshops and survey on the needs and requirements, problem statement, functional properties, and visualization and communication aspects of air quality sensors. After a preparatory phase, the above pilot sensor networks were deployed to address the real-world policy concerns of local authorities. Based on the feedback from the involved cities, sensor data were deemed useful as an evidence base for planned policy measures and were communicated to citizens. The involved municipalities and cities indicated that organizing air quality sensor networks at the local municipal level is still challenging, with sensor selection, data quality evaluation and data analysis as the most challenging processes, requiring support from external experts or agencies.

To this end, we developed a hands-on tool (blueprint) for local governments aiming to roll out air quality sensor networks in their municipality or city. The blueprint includes scoping advice and practical tools (e.g., sensor selection tool) with regard to the entire design process, including defining a research question, experimental setup, roll out, data analysis and communication (Figure 12). This blueprint is available online [79].

5. Conclusions

This work describes a collaborative effort of a research institute (VITO), local authorities and the regional environmental agency (VMM) in substantiating local air quality policies with actionable sensor data from commercially available air quality sensor systems (Airly, Kunak). The aim was to evaluate the usability of commercial “low-cost” air quality sensor systems to substantiate evidence-based policy making by quantifying the local pollution contribution from road traffic at a central city square and potential impacts from different traffic measures in a school environment.

Both pilots demonstrate that contemporary air quality sensors are able to accurately quantify air quality impacts from (even short-lived) local traffic measures and contribute to evidence-based policy making, under the condition of a proper methodological setup (background normalization) and data quality (recurrent calibration) procedure. When considering commercially available sensor solutions, co-location campaigns remain vital to benchmark the data quality of the sensors, evaluate the validity of observed effects (sensor precision) and improve sensor performance through calibration. Including a local background location in the sensor network has proven methodologically valuable to normalize for background pollutant dynamics and to quantify local pollutant contributions (i.e., potential impact of local policy measures). Based on the feedback from the involved cities, sensor data were deemed useful and can empower local authorities to implement new air quality management plans. To this end, a blueprint was developed to help local authorities in setting up local air quality senor networks.