Insights into Air Quality Index (AQI) Variability with Explainable Machine Learning Techniques ^†

Abstract

In this study, a combined approach joining the machine learning model Extreme Gradient Boosting (XGBoost) with Shapley Additive Explanation (SHAP) is adopted to simulate the temporal pattern of the air quality index (AQI) and subsequently explore the key factors affecting AQI variability. Based on the analysis of air pollutants and meteorological data acquired from two air quality monitoring stations in Rome (Italy), over the 2018–2022 period, the results demonstrate the effectiveness of the proposed methodological approach in elucidating the role of the main factors driving AQI evolution, and their interaction effects.

1. Introduction

The air quality index (AQI) is a dimensionless indicator used to provide real time air quality and health information [1,2]. Although member states in the European Union are subject to the same air quality legislation [3], a common AQI has not been adopted, and each country applies its own [4]. In Italy, a decentralized network of 21 regional environmental protection agencies (ARPAs) monitors and manages air quality, with the national-level agency (National System for the Environmental Protection, SNPA) coordinating these efforts. While the calculation of the AQI may vary slightly between different Italian regions, the basic principle remains the same: the AQI is developed using criteria pollutant criteria, the short-term average concentrations of which are compared to the short-term limit values set by the current legislation [5]. The pollutant with the highest value relative to its limit value determines the AQI value. Finally, the AQI values are grouped into five classes with a uniform interval width of 50, indicating good, acceptable, moderate, poor, and very poor levels of air quality.

From a health perspective, measuring the AQI based on the dominant pollutant may not accurately reflect our current understanding of the adverse health effects of ambient air pollution, especially in circumstances involving the combined effect of multiple pollutants [6] or low-level exposure [7]. Therefore, a deeper characterization of the factors affecting AQI variability would be beneficial to provide more reliable data and information for air pollution management and more effective health protection strategies.

Stochastic models such as ARIMA, SARIMA, ARIMAX, SARIMAX have been traditionally used for air pollutant time series modeling [8]. Recently, the superiority of machine learning models has been shown in comparative studies investigating the abilities of various existing techniques when applied for time series forecasting [9,10].

Therefore, a combined approach integrating a tree-based machine learning (ML) algorithm (XGBoost) [11] with a game theory-based explanation model (SHAP) [12] has been implemented in this study. The objective is to simulate the temporal pattern of AQI and then explore the influence of various driving factors and their interactions on AQI variability.

Despite their remarkable predictive capabilities, ML models are regarded as “black boxes”, that is, it is difficult for the ML algorithm to provide a suitable explanation for how it arrived at an answer [13,14]. The lack of interpretability and explainability is a huge challenge when there is a need to explain the rationale behind a model’s output. To overcome this issue, explainable artificial intelligence strategies (XAI) have been developed to understand how an ML model makes its predictions [15] and to assess the impact of individual factors and their interrelationships on the model output.

The proposed methodological framework has been applied to a dataset comprising hourly data for air pollutant concentrations and meteorological parameters acquired, over the 2018–2022 period, in Rome (central Italy). The results obtained from this study demonstrate the efficacy of both XGBoost and SHAP algorithms in predicting and explaining AQI variability. The innovative character of the proposed work consists of building a machine learning model that simulates AQI as a function of parameters that are not present in its calculation formula. Moreover, through the SHAP analysis, the work carried out also proposes an examination of the complex interconnections between air pollutants and meteorological factors and AQI.

2. Materials and Methods

2.1. Study Area

The area of interest in this study is Rome (Lat. 41.91 N, Lon. 12.48 E), the capital of Italy, located about 27 km from the coast of the Tyrrhenian Sea. The urban area is approximately 1300 km², including highly urbanized zones alternating with parks and green areas. The territory is mainly flat, with an altitude between 13 and 140 m above sea level, while the rural areas surrounding the city have a more complex hilly orography. According to the Köppen–Geiger climate classification [16], Rome belongs to the Mediterranean climate class (Csa), i.e., a temperate climate, with dry and hot summers [17]. In the absence of significant industrial plants, the air quality is strongly influenced by local emissions (vehicular traffic and domestic heating) and by advective phenomena [18]. A network of 18 air quality monitoring stations and 4 weather stations are managed by the Agency for Environmental Protection of the Lazio Region (ARPAL); acquired data are made publicly available after a validation process. According to the reports on environmental data drafted by ARPAL [19] over the 2013–2022 period, in the urban area PM₁₀, NO₂ and O₃ exceeded target values.

2.2. Observational Dataset

To build the working dataset, hourly data for the concentrations of five air pollutants, namely O₃, nitrogen dioxide (NO₂), nitrogen oxide (NO), and particulate matter (PM₁₀ and PM_2.5), as well as three meteorological parameters, temperature (T), relative humidity (RH) and wind speed (ws), have been downloaded from the official ARPAL website (https://www.arpalazio.net/main/aria/sci/basedati/chimici/chimici.php (accessed on 7 March 2025), https://www.arpalazio.it/rete-micro-meteorologica, (accessed on 10 March 2025)). The data needed for our study were acquired from two air quality monitoring stations selected according to their location, one in downtown Rome on the edge of a large green area, Villa Ada (VA), and the other one in the rural surrounding areas situated in a protected natural area, Castel di Guido (CG), classified as urban and rural background stations, so that they are representative of the pollution conditions in the urban and rural environment, respectively. For meteorological parameters, the two nearest weather stations have been selected, i.e., Boncompagni e Castel di Guido, respectively. Figure 1a,b show a map of the studied area and the AQI temporal pattern at the CG and VA sites.

The daily AQI has been calculated according to the equation:

$$AQi = max\left(I_i\right) = 100{\displaystyle \frac{C_i}{S_i}}$$

(1)

where i indicates the pollutant; C_i is the hourly concentration for NO₂, the 8 h carried mobile average for O₃ and the daily average for PM₁₀; and S_i is the corresponding threshold level for the protection of human health established by the legislation.

As this is a daily index, the reference pollutants are those for which parameters with a short-term average are established by current legislation [5] and whose effect could pose a potential risk to the population. The reference values are as follows: the hourly maximum average for NO₂ is 200 µg/m³, the 8 h average for O₃ is 120 µg/m³, and the daily average for PM₁₀ is 50 µg/m³. Overall, a dataset consisting of 3652 daily observations, covering the 2018–2022 period, was set up for the development of the XGBoost model.

2.3. Methodological Approach

Data loading, processing, statistical analysis, and modeling were accomplished in the R software environment and its associated packages (R version 4.3.1)

2.3.1. XGBoost

The (XGBoost) algorithm is a supervised machine learning technique based on the ensemble of decision trees, designed to solve classification, and regression problems. It has been chosen to predict the time-varying AQI pattern due to its scalability, speed, performance, and interpretability compared to other algorithms [20].

Assuming hourly AQI concentrations as the model output (or target), the model can be expressed in terms of 8 features (or explanatory variables), as follows:

$$AQI ~\mathrm{xgboost}\left({\mathrm{O}}_3, {\mathrm{NO}}_2, \mathrm{NO}, {\mathrm{PM}}_{10}, {\mathrm{PM}}_{2.5}, \mathrm{T}, \mathrm{RH},\mathrm{ws}\right)$$

(2)

where $xgboost$ is the function implementing the boosted regression tree technique in the R software environment (xgboost R package 1.7.8.1), and the chosen features are available in all the sites considered. To build the model, the observational dataset has been randomly partitioned into a training dataset for the model development (75% of the observations) and a test dataset used to check the model performance (25% of the observations). The hyperparameter tuning, as defined at https://xgboost.readthedocs.io/en/latest/parameter.html (accessed on 7 March 2025), was conducted by means of Bayesian optimization (ParBayesianOptimization R package 1.2.6). The XGBoost models’ skill has been evaluated by comparing predicted and observed AQI values using the following statistical indicators for the training and test datasets: the coefficient of determination (R²), the mean bias error (MBE), and the root mean square error (RMSE).

2.3.2. SHAP

The SHAP method is a technique that allows the output of any machine learning model to be explained. Based on the coalitional game theory, this approach enables an understanding of how the model reaches its predictions by assessing each feature’s contribution to model output and addressing complex feature interactions [21]. For theoretical insights readers can refer to [12]. In this study, once obtained, SHAP (shapViz R package 0.9.7) was applied to the outcomes of the XGBoost models to elucidate the extent and the nature (positive or negative) of the contribution of each feature to the AQI predictions. The SHAP explanations are supported by several SHAP visualization tools enabling a more intuitive comprehension of the model output.

3. Results and Discussion

3.1. Explorative AQI Analysis

An explorative AQI data analysis was carried out before the XGBoost model was applied. The results show (i) a mostly acceptable level of AQI at both examined sites, i.e., an AQI smaller than or equal to 100, Figure 1b, (ii) the pollutants with the largest individual AQI, i.e., in the order, O₃, PM₁₀ and NO₂, (iii) the seasonal shift in AQI, which is higher during spring and summer indicating poorer air quality in these seasons, and (iv) a similar spatial distribution of AQI values among sites. Interestingly, during the study period there was only one day when the AQI exceeded 150 at the CG site.

3.2. XGBoost Model Performances

The best hyperparameters of the XGBoost model obtained by Bayesian optimization are reported in

Table 1. Best XGBoost model hyperparameters.

Eta.	Max_Depth	Min_Child_Weight	Subsample	Colsample_Bytree	Gamma
0.2	8	8	0.5	0.9	1.73

.

As can be seen from

Table 2. AQI XGBoost model performance on training and testing data during the period 2018–2022.

Statistical Indicator	Castel di Guido	Villa Ada
R2	0.88	0.88
MBE	−0.13	0.30
RMSE	5.96	6.93

and the scatter plots in Figure 2a,b, a good degree of concordance is found between the observed and predicted AQI values over the 2018–2022 period.

Overall, the performances of the models are satisfactory for daily predictions and consistent with other studies [22].

3.3. SHAP Analysis

For each site, Figure 3 summarizes, in descending order, the relative importance as well as the positive or negative contribution of each feature to the target prediction. The number on the left is the global importance of the features based on the mean absolute SHAP value. O₃, PM₁₀, PM_2.5, T, and ws turn out to be the most important features. The order of the most essential features is slightly different between the two sites; however, the overall contribution of O₃ and PM₁₀ to AQI is predominant at each site. Moreover, the AQI is more sensitive to air pollutants than meteorological factors, which is consistent with the fact that the AQI is calculated according to the concentrations of three air pollutants.

It is intriguing to emphasize the role of PM_2.5, which is not directly included in the AQI calculation yet but has been integrated into the model as a predictor due to its known strong interconnection with O₃ [23]. NO₂ and NO provide a negligible contribution, although, according to [19], NO₂ still represents a criticality in the Rome area together with O₃. Among the meteorological features considered, ws and T are the most important features, while the results show that the RH’s contribution is much smaller.

At both sites, elevated concentrations of O₃ and particulate matter (PM₁₀ and PM_2.5) are found to have a positive SHAP value, thereby contributing to a better prediction of AQI values. As expected, this finding indicates a correlation between elevated concentrations of pollutants and high AQI values.

The role of meteorological parameters in the AQI is more intricate in comparison to that of pollutant factors. Elevated T values are found to have a positive SHAP value, indicating a positive correlation between AQI and T. This phenomenon can be attributed to the fact that this parameter enhances O₃ formation [24], which is the primary contributor to the AQI. Conversely, higher ws values have a negative SHAP value, thus contributing to a lower prediction of Air Quality Index (AQI) values which translates into a better air quality likely attributable to pollutant dispersion [25].

A seasonal analysis confirmed O₃ as the most significant feature at both locations and in each season except in winter when the contribution of PM₁₀ becomes more significant. Moreover, it identifies a relevant role of T in summer and winter at the CG and VA sites, as well as a relevant role of ws in spring, summer, and fall at the VA site.

Insights into the shape of the relationship between the feature values and their impact on the prediction can be obtained with the SHAP dependence plots. In Figure 4a the relationship between O₃, the most significant features for the VA site, and its SHAP values is illustrated. The analysis of this relationship enables the identification of the turning points, i.e., points where the feature contribution to the AQI prediction undergoes an inversion in behavior, from positive to negative or vice versa, with respect to the base value. Figure 4a shows that O₃ has a positive SHAP value above 50 µg/m³.

The vertical spread of SHAP values at a fixed value of the feature, which can be seen in Figure 4a, is a sign of interaction effects with all the other features in the model. To explain the effect of the interactions, the dependence plot showing the SHAP values of O₃ at the VA site has been split into two parts, as shown in Figure 4b,c. The first represents the impact of O₃ on the AQI after all interaction effects have been removed; the second represents the interaction between O₃ and PM₁₀ which is the strongest interaction determined by SHAP theory. Following the red arrow, we can see that below the turning point, increasing the PM₁₀ values at a fixed O₃ level increases the SHAP interaction values, which in turn increases the AQI and reduces air quality.

Finally, a short-term and site-specific analysis has been carried out to reveal some peculiarities in the variability of AQI during a selected pollution event, identified on the 24 of April 2019. In Figure 5 the time trend of distinctive features contributing to an AQI value higher than 150 is illustrated, showing that on the peak day all features push the prediction above the base value. Moreover, PM₁₀ has a dominant role that is not apparent in the seasonal analysis. According to the analysis of the available metadata, this event can be attributed to a massive dust storm pushing Saharan dust over the Mediterranean Sea in late April 2019 that has been well documented at national and European levels [26].

4. Conclusions

The XGBoost model accurately predicts the AQI variability, showing a correlation of 0.88 for the testing datasets. The remaining discrepancies between the predicted and observed AQI levels may be due to the lack of available additional predictor factors, such as the height of the boundary layer or the contribution of other pollutants, such as volatile organic compounds (VOCs). The SHAP analysis makes it possible to identify the main determinants of AQI variability, both over the study period and in relation to a specific event, as well as the effect of the main interactions between features. It is also worth noting that our analysis suggests a relevant role of factors that are sensitive to climate change, such as O₃ and T, in determining the variability of the AQI.

It is important to specify that caution should be exercised when deriving insights from SHAP analyses. SHAP only tells you what the model is doing within the context of the data on which it has been trained. However, even if it does not necessarily reveal the true relationship between variables and outcomes in the real world, it could help generate hypothesis to unveil implicit knowledge or even new knowledge.

Overall, the results obtained highlight the effectiveness of an integrated approach combining ML and SHAP algorithms in elucidating the main factors driving AQI evolution. This knowledge is expected to be useful for optimizing AQI prevention and mitigation measures for the protection of public health in the examined area.