1. Introduction
Air pollution represents one of the main concerns for public health in almost every country around the globe. It is proven that this issue is the cause of premature death for seven million people annually [
1]. The European Commission has also shown that European citizens spend around the 90% of their time in indoor environments, mainly at home or in workplaces [
2]. For this reason, the indoor environment can drastically affect people’s health, positively or negatively. Another study conducted by the US Environmental Protection Agency (EPA) has shown that indoor environments can be two to five times more toxic than outdoor locations [
3]. Therefore, it is clear that air quality monitoring in indoor environments is needed for the personal exposure risk assessment of air pollutants. Currently, air quality monitoring is performed by government authorities through the use of chemical analyzers, which are bulky, heavy, and require routine maintenance. In addition, they are also very expensive (their prices ranges between EUR5000 and EUR30,000) [
4,
5], thus, they hardly can be afforded by households and common citizens.
In recent years, the use of air quality monitors (AQMs) based on low-cost gas sensors (LCSs) has become more and more popular, due to their affordability, portability, and ease of use [
5,
6,
7]. The working principles of LCSs, on which AQMs are based, are comprised by different technologies that provide several advantages, such as low power consumption, device compactness, and little need for maintenance. The use of LCSs was tested, not only for air quality assessment, but also for other real-world applications, such as malodor detection [
8,
9].
A key factor determining the performance of LCS sensors and AQMs based on them is represented by the algorithm, or the calibration model, used for their calibration [
5]. The linear regression (LR) and the multivariate linear regression (MLR) methods, along with several machine learning models, are the most frequently used.
Concerning the particulate matter (PM) concentration measurement, LCSs based on laser light scattering are currently employed worldwide, and their performance has been investigated in several studies. For example, Si and others [
10] tested four different calibration models to evaluate the performance of the PMS5003 sensor. They compared the results given by the LR, MLR, and other machine learning algorithms, such as XGBoost, and the artificial feedforward neural network (ANN). The authors of this study found that the most promising method to obtain good quality data from the PMS5003 sensor in an outdoor environment is represented by the feedforward ANN. Other examples of machine learning calibration models employed for LCS calibration can be found in [
11].
Regarding LCSs for measuring pollutant gases, if we consider their advantages, they look to be the ideal substitute for chemical analyzers, but unfortunately, they cannot offer the same accuracy level, and the data produced by such devices have not always proved reliable [
4,
5,
6,
7]. In fact, LCSs, and therefore AQMs, are sensitive to temperature and humidity changes; moreover, they suffer from drift phenomena. These factors are the origin of inaccuracies in measurements and may lead to the deterioration of AQM performance. In addition to these elements, it has to be considered that in general, LCSs may not be selective; this means that measurements performed by sensors designed for carbon monoxide, for example, can be affected by the presence of other gases (called interfering gases) which can alter the correct measurement of carbon monoxide concentrations. This effect is also known as sensor cross-sensitivity.
To account for these adverse factors, researchers have found that the adoption of advanced calibration techniques can significantly improve AQM performance. They have also found that the most effective way to maximize their performance in real-world applications is to calibrate them in co-location with reference instruments placed in the final deployment environment.
The most common calibration algorithms for LCSs used in the recent studies are based on multivariate linear regression (MLR), support vector regression (SVR), random forest regression (RF), and artificial neural networks (ANN).
In any case, the current research on LCS calibration does not provide clear indications about which calibration approach achieves the best AQM performance, but previous studies seem to indicate that the reliability of data produced by AQMs also depends on the environmental variables [
4]. More specifically, the LCS calibration process can be affected by the combination of concentration levels of target gases, the variability range of temperature and humidity, and the concentration levels of interfering gases [
4].
In this respect, a consistent number of studies have been conducted to explore the performance of calibration algorithms in outdoor environments featuring different conditions; on the contrary, very few investigations (to the best of our knowledge) have been performed to assess the effectiveness of such techniques in an indoor, real-world scenario using the co-location of reference instruments with the AQM in question.
Castell et al. [
4] highlighted that a good performance of AQMs in the laboratory is not indicative of good performance in real-world scenarios. In this study, CO, NO, O
3, and NO
2 LCSs were calibrated by using linear regression (LR) models in outdoor environments. However, it must be noted that, concerning the state-of-the-art of calibration models, the most widely used method to calibrate LCSs is the multivariate linear regression (MLR) [
5]. Other approaches for AQM calibrations are the random forest (RF) algorithm, the support vector machine (SVM), and artificial neural networks (ANN) [
5].
Several works tested these calibration models in an outdoor environment, reporting various results in terms of performance quality. In this study, we focused our investigation on the electrochemical LCSs, neglecting of the scope other types of air pollutant sensors. In
Table 1, data from previous studies are summarized to provide an indication of the results achieved in outdoor environment using the previously mentioned models to calibrate AQMs or LCSs.
Concerning the use of AQMs based on LCSs in indoor environments, some studies were conducted to explore their potentialities and capabilities. In one of these studies, Pitarma and others [
19] proposed a wireless sensor network system for monitoring CO and CO
2 concentrations, along with other environmental parameters. In the work of Zhang [
20], a wider range of air pollutants was monitored through a custom-built monitoring system capable of detecting TVOCs (total volatile organic compounds), CO, CO
2, NO
2, SO
2, O
3, PM
10, PM
2.5, and PM
1. An IoT (Internet of Things) based air quality monitoring platform called “Smart-Air” was presented by Byung Wan Jo [
21] for measuring VOC, CO, and CO
2. This system was implemented in the Hanyang University of Korea to demonstrate its feasibility. However, all these studies were conducted without comparing the performance of AQMs in co-location with reference instrumentation to provide an idea of the effective quality of the data produced by them. Moreover, no information was provided about the LCS calibration algorithms used in the AQMs. Few studies involving LCSs in co-location with reference instruments to test AQM data reliability in an indoor environment were conducted. Most studies are focused on investigating the use of AQMs for PM, TVOC, and CO
2 concentration measurements [
22,
23,
24]. To the best of our knowledge, only Tryner and Volckens et al. [
25] measured indoor CO, NO
2, and O
3 concentrations, testing these in a kitchen of an occupied home in co-location with reference monitors. The calibration algorithm used in this study was the MLR, while the total duration of the experiment was 168 h.
The previous studies involving AQMs based on LCSs have proven that AQM data quality significantly depends on the calibration algorithm used, on the concentration levels of the target gases, and on the variability of the interfering parameters, such as temperature, humidity, and the interfering air pollutants. All these factors, in conjunction with the lack of studies comparing CO, NO
2, and O
3 sensor performance with reference monitors in an indoor environment, have induced us to explore the variation of the performance of these LCSs by using different calibration algorithms in an indoor experiment. Therefore, to accomplish this objective, an AQM designed and developed in our laboratories has been placed in co-location with reference monitors in an occupied home for measuring NO
2, CO, and O
3 concentrations during everyday life. The experiment enabled us to compare the performance of electrochemical CO, NO
2, and O
3 sensors by using different calibration algorithms such as MLR, RF, SVM, and ANN (see
Section 2.3 for more details).
3. Results
The experiment lasted 186 h and 54 min, producing a dataset featured by a total number of 5607 records, taken at a sampling rate of 2 min. The calibration dataset includes data recorded from 13 December 2021 to 17 December 2021, providing 2575 records; the validation dataset includes readings from 17 December 2021 to 21 December 2021, resulting in 3032 records.
CO, NO
2, and O
3 concentration statistics, as monitored by the reference during the calibration and validation period, are summarized in
Figure 4a, while in
Figure 4b, the temperature and relative humidity ranges are shown. Data related to these last two parameters were obtained from the sensors mounted inside the AQM and placed very close (less than 10 cm) to the LCSs. For this reason, they were able to measure the values actually experienced by the sensors rather than the values related to the room where the AQM was placed.
Model evaluations based on the metrics selected for their assessment are revealed in
Table 5. By examining this table, we see that, if we consider the coefficient of determination (
R2) and the validation dataset, the best performance related to CO, NO
2, and O
3 is respectively achieved by the CO(2) predictor set calibrated through ANN, by the NO2(1) calibrated through MLR, and the O3(1) calibrated by MLR.
Time series and scatter-plots referring to the previously mentioned three cases are shown in
Figure 5,
Figure 6,
Figure 7,
Figure 8 and
Figure 9. In particular,
Figure 5 shows the time series related to the calibration dataset, while in
Figure 6, it is possible to see the data concerning the validation dataset. In these figures, some events originating the pollutant gases are reported for an indicative idea about the possible sources of domestic pollution. During the experiment, a systematic logging of every event related to the monitored pollutant gases was not carried out, because the aim of this study is mainly focused on assessing the AQM performance calibrated by different models.
Figure 5 reports the predictions during the calibration period described for CO (
Figure 5a), NO
2 (
Figure 5b), and O
3 (
Figure 5c). These are featured by a coefficient of determination of 0.975, 0.734, and 0.432, respectively. Similarly to
Figure 5, data reported in
Figure 6 are related to the validation period for CO (
Figure 6a), NO
2 (
Figure 6b), and O
3 (
Figure 6c). The coefficient of determination in these three cases is respectively 0.924, 0.890, and 0.137.
To better understand the differences in prediction capability of the AQM by using the calibration models under investigation, it is useful to analyze
Table 6 and
Table 7. In these tables, the median values of
R2 and
nRMSE are reported for each model by separately considering the three pollutants. In particular, by taking into account the median values and not the best ones,
Table 7 shows that, in terms of
R2, ANN achieves the best performance in predicting the CO concentrations, while for NO
2 and O
3, the MLR model offers the best results. As expected, values related to the calibration dataset shown in
Table 6 are in general better than the respective values shown in
Table 7, where calculations performed by the validation dataset are shown.
In this experiment, we decided to investigate the effects of different inputs or predictors for MLR, RF, SVM, and ANN calibration models (see
Table 4). These inputs can be distinguished into two subsets. The first subset is defined by taking as a predictor the difference between the “working” and “auxiliary” electrodes of each sensor selected for predicting the gas concentrations. The second subset includes the sets where a predictor variable is represented by every single electrode of the sensor (see
Table 4). The results obtained by comparing the performance achievable through the two subsets and considering all the calibration models related to CO, NO
2, and O
3 are reported in
Figure 10. The indicators used for this analysis are represented by
R2 and
nRMSE, while the results originated by the first subset are marked with the “net” suffix, and the results from the second subset are denoted by the “single” suffix.
4. Discussion
By analyzing
Table 5 and
Table 7, it can be seen that the best performance related to the validation dataset was achieved for CO and NO
2 predictions, while all the indicators show that the poorest performance was for the O
3 measurements. This result is further highlighted by the scatter plots shown in
Figure 9, where the spread of point clouds indicates the low correlation between the predicted ozone concentrations and those measured by the reference. One possible explanation can be provided by the high cross-sensitivity of ozone sensors to NO
2 concentrations. If we carefully inspect
Figure 5 and
Figure 6, it can be noted that NO
2 emissions are almost always originated in concurrence with ozone production. This element, in conjunction with a lower range of ozone concentrations compared with the NO
2 levels, shown in
Figure 4, could explain this relevant difference between the ozone concentration prediction and the other results. Moreover, by inspecting
Figure 4b, it can be seen that the temperature and relative humidity levels experienced by the sensors under test mostly fall into a relatively restrained range. This factor has contributed to limit the interfering effects of these two parameters, ending up benefiting the overall LCS performance.
If we want to try a comparison with similar previous works, we must distinguish between studies carried out in outdoor environments and indoor investigations performed in an occupied home. While the first category of works is characterized by a relevant number of studies (see the review written by Karagulian et al. [
5]), to the best of our knowledge, the only study performing AQM data quality assessment by comparison with reference monitors for CO, NO
2, and O
3 is provided in Tryner’s work [
25], although this study was carried out through a test methodology different from the approach followed here. In any case, some similarities can be found by comparing the two experiments: in general, CO, and NO
2 predictions are more accurate than the ozone predictions, and in particular, CO measurements exhibit more correlation with the reference data. The indicators used for both the studies were
R2,
MAE, and
RMSE, allowing us to build
Table 8, where the median values of the indicators summarize the differences found between that study and our current research.
Concerning the investigations carried out in outdoor environments, we can refer to the work of Karagulian [
5], which also provides data related to the calibration model used in the various studies. The comparison between this work and the previously mentioned studies is exposed in
Table 9, where it can be noted that data related to CO and NO
2 are quite comparable, except for in the SVM case. On the contrary, results concerning the O
3 pollutant significantly differ from the data found in this experiment. A possible explanation could be that LCS performance is significantly sensitive to both the gas concentration levels and the magnitude of interfering gas concentrations. These two factors can be remarkably different from one environment, or location, to another, causing discrepancies in AQM performance.
Another aspect concerning AQM performance assessment is represented by the model selected for LCS calibration. In this study, we have investigated four calibration models (MLR, RF, SVM, and ANN) utilizing
R2,
MAE,
RMSE, and
nRMSE indicators. By examining
Table 5,
Table 6 and
Table 7, it can be noted that, in general, the best performance is achieved by MLR, RF, and ANN models, but also that the data does not clearly indicate the presence of a specific outperforming algorithm. Rather, our analysis suggests that in most cases, the SVM approach provides less accurate predictions for each monitored pollutant. This last element is possibly due to the intrinsic difficulty in finding the optimal model hyperparameter set. The computing time required for determining the optimal hyperparameter combination was significantly larger compared to the that one for other models, and perhaps more extensive efforts and trials would be needed to find them. Finally, it must be noted that fast and optimal hyperparameter tuning is an active research area within the scientific community [
14]. However, by assessing those results from a practical point of view, MLR models are much easier to implement using electronic microprocessor boards of AQMs, requiring fewer computational resources. On the contrary, RF and ANN models require more computational power, memory, and dedicated software libraries.
Finally, another aspect investigated through this experiment is represented by the selection of predictors for the calibration models.
Figure 10 clearly shows that, taken separately, the “working” and the “auxiliary” signals in the predictor sets generally lead to better performance. This conclusion can be drawn by noting that the median values of R
2 and nRMSE shown in
Figure 10 are always better in this case, as compared to the “net” predictor choice.
5. Conclusions
The experiment carried out in this work clearly points out that CO, NO2, and O3 pollution can be an issue equally, if not more, concerning than the outdoor emissions of these gases. Monitoring this pollutant by chemical analyzers is not feasible for households, due to their high costs and logistic issues. Moreover, the permanent buzzing sound they usually generate during their operation could be a problem for their use in homes and apartments. An option for addressing these issues could be represented by AQMs based on LCSs, which are significantly less expensive and not noisy.
An experiment was conducted in an occupied home to demonstrate the effectiveness of the AQM for monitoring CO, NO2, and O3 pollutants by using different calibration models. We found that CO and NO2 pollutant concentration measurements are in good agreement with the reference instruments data, if calibrated through MLR, RF, and ANN models. In particular, by considering the validation period, the best performance in terms of R2 for CO concentration measurements was achieved through the ANN model (R2 = 0.924), while the best MAE and RMSE values were achieved by MLR calibration (MAE = 0.099 ppm; RMSE = 0.140 ppm). Moreover, in the case of the NO2, we found that in the validation period, the best performance was given by the MLR model (R2 = 0.924; MAE = 8.381 ppb; RMSE = 10.618 ppb).
Moreover, we proved that model prediction capabilities could be further optimized by separately using sensor electrode signals as inputs.