Concept Drift Mitigation in Low-Cost Air Quality Monitoring Networks ^†

Abstract

Future air quality monitoring networks will include fleets of low-cost gas and particulate matter sensors calibrated using machine learning techniques. Unfortunately, it is well known that concept drift is one of the primary causes of losses in data quality in operational scenarios. This work focuses on addressing a low-cost NO₂ sensor calibration model update triggered via a concept drift detector. This study defines which data are most appropriate for use in the model updating process in order to maintain compliance with the relative expanded uncertainty (REU) limits established by the European Directive, as well as evaluate the potential of general and importance-weighted calibration models in the mitigation of concept drift effects.

1. Introduction

The data quality of low-cost gas sensors calibrated by means of machine learning techniques is still the crucial factor limiting their spread. Recently, the introduction of a concept drift detector in the chain value of data formation, able to provide an alert for the possible retraining or update of the calibration model, has been proposed [1]. At this point, what to do after a re-calibration request has arrived is an open question. The first question is about which data to select for the update and whether the reference data are available or not. An alternative model that we will consider for tackling this issue is the general calibration model [2]. This approach has been introduced as an attempt to reduce calibration costs. It consists of identifying and applying a general calibration model to all co-located nodes, thus avoiding the need for additional ad hoc calibrations, since the inherent variability of each individual sensor is incorporated into a single model. Also, an importance-weighted calibration model has been considered. Its procedure is to “weigh” the samples of the test set in order to “match” the distribution used during the training phase [3].

2. Materials and Methods

The winter 2020 co-location campaign dataset created in Portici (Italy), during which three MONICA devices (AQ6, AQ11, AQ12) were co-located with a reference mobile laboratory for two months and characterized by the presence of concept drift, is used to address concept drift handling in two steps: (i) selecting the appropriate data to use for the calibration model’s update and (ii) exploring the general and importance-weighted calibration models’ performances as alternative models. The dataset is divided into eight one-week time slots and, after that, the concept drift in T4 is characterized by the worst-quality NO2 estimations for the subsequent time slots [4]. Relative expanded uncertainty (REU) is the metric adopted. Three options are investigated for data selection: the data preceding the concept drift alert (called “Last”), the data following the concept drift alert (“Next”), or parts of both (“Mixed”) [5]. The main idea is to try to mitigate the effects of concept drift by exploiting the information content of the co-location data as much as possible, so the following two models will be explored.

General calibration model: if n sensors are placed in co-location, then the set of the medians of all the single quantities involved in the calibration model’s creation constitutes the training set which the general calibration model is trained on.

Importance-weighted calibration model: The importance of a sample (the “weight”) is calculated as the ratio between the probability density functions of the test and training sets. Once the weights are obtained, these will be applied in the training process, obtaining a new calibration model.

3. Results and Discussion

The REU charts show that the “Next” approach is to be preferred over the others, but it has a drawback: the node keeps releasing poor-quality data. This amount of invalid data is reduced by applying the “Mixed” approach. However, the data contained in the “Last” batch are not usable since they are not representative of the “Next” operational scenario. The general calibration model works well for the AQ6, suggesting that it is efficient at the mitigation of the concept drift’s consequences (the REU plot drops below 25% at 45 µg/m3), while for AQ12 devices it matches ad hoc model performance. The AQ11’s intrinsic variability makes this instrument too different from the others. The application of the importance-weighted calibration model for AQ6 and AQ12 does not improve their performances compared to the ad hoc model, whilst it works in the T5 time slot for the AQ11 device.

4. Conclusions

The preliminary results of this work make clear the effectiveness of both the proposed methodological approach and the alternative calibration models used, as well as extending the validity of their calibrations. A further investigation is ongoing, aimed at further improving the obtained results through the use of a stacking ensemble which embeds the general calibration model and importance-weighted calibration model as base learners.