Features Exploration from Datasets Vision in Air Quality Prediction Domain

Abstract

Air pollution and its consequences are negatively impacting on the world population and the environment, which converts the monitoring and forecasting air quality techniques as essential tools to combat this problem. To predict air quality with maximum accuracy, along with the implemented models and the quantity of the data, it is crucial also to consider the dataset types. This study selected a set of research works in the field of air quality prediction and is concentrated on the exploration of the datasets utilised in them. The most significant findings of this research work are: (1) meteorological datasets were used in 94.6% of the papers leaving behind the rest of the datasets with a big difference, which is complemented with others, such as temporal data, spatial data, and so on; (2) the usage of various datasets combinations has been commenced since 2009; and (3) the utilisation of open data have been started since 2012, 32.3% of the studies used open data, and 63.4% of the studies did not provide the data.

1. Introduction

According to the United Nations (UN) in 2018, more than 55% of the world’s population lives in urban areas. The trend shows that by 2050 urban population will increase until 68%; particularly compared to other regions, the urban population will grow faster in Asia and Africa, considering that these regions have more rural population [1]. Among the positive effects, such as better employment and education opportunities, enhanced healthcare system, greater access to social services, urbanisation also has negative consequences being a cause of air pollution or the increased demands on resources, among others. According to the World Health Organisation (WHO), every year, more than seven million persons die because of this problem or related to that [2].

It is very important to understand which pollutants are considered when determining air quality, and how to calculate and represent air quality indicators. Regarding the pollutants, they form from natural and anthropogenic sources. The WHO identifies the following pollutants as having serious impacts: particulate matter with diameter less than 2.5 micrometers (PM_2.5), particulate matter with diameter less than 10 micrometers (PM₁₀), nitrogen oxide (NO_x), ground-level ozone (O₃) and sulfur dioxide (SO₂) [3]. Depending on the region and the presence of predominant pollutants, it is proposed to use different indices for calculating air quality, for example, the United States Environmental Protection Agency (EPA) Air Quality Index (AQI), the Canada Air Quality Health Index (AQHI), Common Air Quality Index (CAQI) or Daily Air Quality Index (DAQI), among others.

Information about air quality prediction can prompt authorities and decision-makers to apply protective measures in order to reduce air pollution, and this knowledge helps citizens to organise their daily activities by escaping high polluted areas [4,5]. In order to predict air quality more accurately, it is important to consider external factors that influence air quality and include them as input to run models. As an example of those external factors are precipitation, wind direction, traffic intensity or population density, among others [6,7,8,9]. It should also be emphasised the effect to publish this kind of data as open data, which existence is beneficial both for government and for citizens. The availability of open data have an impact in many areas, such as an increase of transparency, improvement of efficiency and effectiveness of government services, empowerment of citizens, engagement and participation of citizens in governance [10,11]. At the same time, these data can be used by researchers as real inputs to run their models in research works.

Taking the aspects mentioned above into account, the main goal of this manuscript is to analyse and synthesise studies related to air quality prediction using Machine Learning (ML) technologies, and find out: (1) What types of datasets are used to improve air quality predictions? (2) What characteristics of the dataset are important for efficient and effective air quality forecasting? and (3) Which features are the most used to define ML models? We believe that this work can be useful for other new works in the field of air quality prediction. Furthermore, considering the scale of the scope in which the topic may be addressed, it should be noted that the perspective of this work is based on data science, and how the obtained results can be used to start a new study to predict air quality in a certain area.

The rest of the paper is organised as follows. Section 2 explains the methodology. Section 3 presents the obtained results and introduces the discussion predicated on the acquired outcomes. Eventually, in Section 4 the conclusions are included.

2. Methods

To achieve the central goals for which this study is targeted, we used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [12] in order to select relevant papers. Those papers were queried in Association for Computing Machinery (ACM), IEEE Xplore, Scopus and Web of Science (WoS) databases using the following query: (“machine learning”) AND (“prediction” OR “forecast”) AND (“air quality” OR “air pollution”), which was being applied to title, abstract and keywords. At the first stage, it was selected all papers published until 28 September 2020 (search date) and the result was 1214 papers in total. Then duplicated and non-empirical manuscripts were removed. Afterwards, based on the inclusion/exclusion criteria listed in

Table 1. Inclusion and exclusion criteria.

Inclusion Criteria	Exclusion Criteria
Papers written in English	Non-English written papers
Publications in scientific journals	Non-reviewed papers, editorials, presentations
Publications until 28 September 2020	Publications after 28 September 2020
Publications focused on outdoor air pollution	Publications focused on indoor air pollution
Extra dataset together with air quality data	Using only air quality data
Analysis with implementation of ML techniques	Analysis without implementation of ML techniques
Models applied for forecasting purpose	Works without forecasting models

, screening of title, abstract and keywords, and full-text assessment were implemented. Later, the manuscripts set was filtered by focusing on several aspects. Mainly the main emphasis was to select papers concentrated on forecasting models of outdoor air pollution, which analyses were performed applying ML technologies. Another essential point was to consider the type of datasets, which assumed that in addition to air quality data, the studies should also include different datasets, such as meteorological, spatial or traffic, among others. It also should be mentioned that only journal papers were included in the final set, which has 93 items. After reviewing those papers, the key features were extracted, which are presented in detail in the next section. The described workflow of the selection procedure of the relevant studies is illustrated in Figure 1.

3. Results and Discussion

After analysing the manuscripts set the main objective results, the exploration and observation based on those obtained results are introduced at this stage. The following essential components of the selected studies were extracted, and the result is summarised in

Table A1. Features of the selected papers. $N/S$: Not Specified. Published in Zenodo with this following doi: https://doi.org/10.5281/zenodo.4302469 [49]. (accessed on 27 February 2021).

Work	Year	Case Study	Prediction Target	Dataset Type	Data Rate	Period (Days)	Open Data	Algorithm	Time Granularity	Evaluation Metric
[36]	2020	USA	PM2.5	Spatial, Temporal, AOD, PBL Height	Daily	5779	No	Hybrid	24 h	RMSE, SD, R2
[50]	2020	Canada	UFP	MET, Traffic, Land Use, BEV	N/S	120	No	Ensemble		RMSE, R2
[51]	2020	Taiwan	PM2.5, PM10	MET	N/S	2192	No	Hybrid	8 h	RMSE, MAE
[39]	2020	China	PM2.5, NOx	MET, Traffic	Hourly	731	No	Regression, Ensemble	1 h	RMSE, ME, NRMSE, NME, POD, POF, R2
[21]	2020	USA	PM2.5	MET, Temporal	Hourly	730	No	NN		RMSE, MAE, MAPE
[42]	2020	India	PM2.5	MET	Hourly	1230	No	NN		RMSE, R2
[52]	2020	USA	AQI	MET	Hourly	851	Yes	Regression	1 h	RMSE, MAE, NRMSE, R
[53]	2020	Turkey	PM10	Spatial, Land Use	N/S	3652	No	Regression, Ensemble, NN		RMSE, MAE, R2
[54]	2020	China	PM2.5	MET	Hourly	31	Yes	NN	1 h	RMSE, R
[55]	2020	China	AQHI, IAQL	MET, Temporal	Hourly	730/1826	Yes	Ensemble	12 h	Acc, MSE, WP, WR, WF
[56]	2020	China	PM10	MET	Daily	1096	No	NN	24 h	RMSE, ME, R, EOp
[37]	2020	Tunisia, Italy		MET, Temporal	Hourly	1461/366	No	Ensemble	1 week	aRRMSE, aRMSE, R2, aCC, MSE, aRE, RP
[38]	2020	China	PM2.5	MET	N/S	46	Yes	Ensemble	24 h	RMSE, MAE, SMAPE
[41]	2020	China	PM2.5	MET	Hourly	1825	No	NN	1 week	RMSE
[57]	2020	China	PM2.5	MET	N/S	1096	Yes	NN	24 h	RMSE, MAE, MAPE
[46]	2020	China	O3	MET, UV Index	Daily	1491	Yes	Hybrid	1 week	RMSE, MAE, MAPE, IA
[58]	2020	South Korea	PM2.5, PM10	MET	Hourly	1461	Yes	Hybrid	15days	RMSE, MAE
[40]	2020	China	PM2.5, PM10, NO2, NO, CO	MET	Daily	4656	No	NN	24 h	MSE
[59]	2020	Taiwan	PM2.5	MET, Spatial, Temporal	Hourly	365	Yes	Ensemble	24 h	RMSE, NRMSE, R2
[60]	2020	UK	PM2.5	MET, Spatial, Temporal, AOD, Land Use	Daily	3287	Partially	Ensemble	24 h	RMSE, MSE, R2
[61]	2020	Ecuador	PM2.5	MET, Spatial, Temporal, Traffic	5 s	4	No	Other Algorithms		Acc
[62]	2020	China	PM2.5	MET	Hourly	365	No	Ensemble	48 h	MSE, IA, NMGE, R2
[63]	2020	China	PM2.5	MET	Hourly	1461	No	Ensemble	24 h	RMSE, MB, ME, R
[64]	2020	China	AQI	MET	Hourly	2192	No	NN	48 h	RMSE, Acc
[32]	2020	China	AQI	MET	Hourly	730	Yes	NN	24 h	RMSE, MAE, R2, FB
[65]	2020	South Korea	PM2.5, PM10	MET, Temporal, Spatial	Minutely	7	No	Hybrid		RMSE
[66]	2020	China	PM2.5, PM10, O3, NO2, SO2, CO	MET, Social Media	Daily	731	Yes	NN	24 h	RMSE, MAE
[67]	2020	Thailand	PM10	MET	Secondly	59	No	NN	1 h	RMSE, MAE, MAPE, R
[68]	2020	China	AQI	Spatial	Daily	1086	Yes	Hybrid	5 days	RMSE, MAE, MAPE, R
[31]	2020	Germany	CO2, NH3, NO, NO2, NOx, O3, PM1, PM2.5, PM10, PN10	MET, Temporal, Traffic, SP	Hourly	62	No	NN	1 h	RMSE, R, NMB, NMSD, RS, SD, SD${}^{\prime }$
[43]	2020	Mongolia	PM2.5	MET, Temporal, Land Use, PD	Hourly	2922	No	Regression, Ensemble	24 h	RMSE, R2
[44]	2020	Taiwan	PM2.5	MET, Temporal, Spatial	Hourly	2192	No	NN	8 h	RMSE, MAE, MAPE
[69]	2020	Turkey	PM10	MET	Daily	766	No	Regression, NN		RMSE, MAE, R2
[70]	2020	Jordan	O3	MET, Temporal	Daily	1496	No	NN, Regression, Ensemble	24 h	RMSE, MAE, R2
[71]	2019	South Korea	PM10, PM2.5	MET, Spatial, Human Movements	Hourly	115	No	NN, Regression	1 h	RMSE, R2
[72]	2019	China/Taiwan	PM2.5	MET	Hourly	3693	No	NN, Other Algorithms	5 days	RMSE
[73]	2019	South Korea	O3	MET	Hourly	1096	No	Ensemble	24 h	IA
[74]	2019	USA	NO2, NOx	MET, Spatial, Traffic	biweekly	8023	No	Ensemble		RMSE, R2, RMSEIQR
[6]	2019	Europe	NO2, PM2.5	AOD, Traffic, Land Use, Altitude	N/S	365	Yes	Regression, Ensemble, NN		RMSE, R2, MSE-R2
[75]	2019	China	PM2.5	MET, AOD	Hourly	1096	Yes	Hybrid	24 h	RMSE, R2
[76]	2019	China	SO2	MET, Temporal, Land Use, OMI-SO2, PPS, TS	Daily	365	Partially	Hybrid	24 h	RMSE, R2, RPE
[77]	2019	China	PM2.5	MET	Hourly	731	No	NN	3 h	RMSE
[78]	2019	China	PM2.5	MET, WFD, Spatial	N/S	61	No	Ensemble	24 h	MAE, SMAPE, MSE
[79]	2019	China	PM2.5	MET	Hourly	1826	Yes	NN	2 h	RMSE, MAE, SMAPE
[80]	2019	China	PM2.5	MET	N/S	2191	Yes	Ensemble	1 week	RMSE, MAE
[81]	2019	Italy	CO(GT), NO2(GT)	MET	Hourly	183	Yes	NN	1 h	RMSE, MAE, MAPE
[82]	2019	China	PM2.5	Spatial	Hourly	365	No	NN	1 week	RMSE, MAE, MAPE
[7]	2019	China	AQI	MET, WFD, Traffic, POI Distribution, FAPE, RND	Hourly	366	Yes	NN	48 h	MAE, MAP
[83]	2019	Taiwan	PM2.5	MET	Hourly	2557	No	Hybrid	4 h	RMSE, Gbench
[84]	2019	Iran	PM2.5	MET	Hourly	1826	No	Ensemble, NN, Hybrid	48 h	RMSE, MAE, R2
[85]	2019	Poland	NO2	MET, Temporal, Traffic	Hourly	731	No	Ensemble		MAPE, MADE, BIC, R2
[86]	2019	India	O3, PM2.5, NOx, CO	MET, Traffic	Hourly	730	No	NN		RMSE, NSE, PBIAS, R
[87]	2019	China	PM2.5	MET	Hourly	1826	No	NN	72 h	RMSE, IA, MAE, R
[47]	2019	China	PM2.5	MET	Hourly	366	No	NN	10 h	RMSE, NRMSE, MAE, SMAPE, R
[88]	2019	China	PM2.5	MET, AOD	N/S	730	Yes	NN		RMSE, MAE, MSE, R2
[89]	2019	Iran	PM2.5	MET, Temporal, Spatial, AOD, Altitude	Daily	1460	Yes	Ensemble, NN		RMSE, MAE, R2
[90]	2019	India	O3	MET	Hourly	92	No	Ensemble		IoAd, R2, PEP
[91]	2019	China	O3	MET	Hourly	365	No	Ensemble, NN		RMSE, R, NMB, NME, MFB, MFE
[92]	2019	UK	SO2	MET	Hourly	120	Yes	Ensemble		RMSE, MAE, R2, RAE
[93]	2019	Taiwan	AQI	MET, Temporal	Hourly	851	No	Regression, NN	6 h	RMSE, MAE, R2
[94]	2019	Iran	PM10, PM2.5	MET, Temporal, Spatial	Daily	3652	Yes	Regression, NN	1 week	RMSE, R2
[95]	2018	China	PM2.5	MET, Temporal, AOD	Hourly	731	Partially	NN	72 h	RMSE, MAE, MSE, IA, TPR, FPR, SI
[96]	2018	Slovenia	PM10, O3	MET, Temporal	Hourly	1461	No	Other Algorithms	24 h	MAE, RPS
[8]	2018	China	O3	MET, Land Use, Elevation, AEI, NDVI, RND, PD	Hourly	365	Yes	Ensemble		RMSE, R2, RPE
[9]	2018	China	PM2.5	MET, AOD, Elevation, PD, RND, NDVI	Daily	1095	Yes	Ensemble	1 month	RMSE, R2, RPE
[97]	2018	China	PM2.5	MET, Spatial	Hourly	61	No	Regression	24 h	total accuracy index (pt), a total absolute error index (et)
[98]	2018	UK	AQI	MET	Hourly	605	Yes	NN		RMSE, MAPE, band Acc
[99]	2018	Kuwait	O3	MET	Hourly	669	No	NN	72 h	RMSE, MAE
[100]	2018	Spain	O3	MET	Hourly	730	Yes	Ensemble	24 h	RMSE, MAE, R2
[101]	2018	Egypt	PM10	MET, Temporal	Hourly	276	No	Regression	1 h	RMSE, R, t-Value
[102]	2018	China	PM2.5	MET	Hourly	1826	No	NN	1 h	RMSE, MAE, IA, R
[103]	2018	USA	O3, PM2.5, SO2	MET	Hourly	3652	Yes	Other Algorithms	24 h	RMSE
[104]	2017	USA	BC	MET, Spatial, Temporal	Daily	4383	Yes	Regression	24 h	R2
[22]	2017	Canada	O3, PM2.5, NO2	MET, Temporal	Hourly	1826	No	NN	48 h	MAE, R, ME, SS
[105]	2017	China	PM2.5	MET, Social Media	Hourly	365	No	NN	24 h	RMSE
[106]	2017	Ecuador	PM2.5	MET	Daily	1827	No	Ensemble, Regression, NN		MSE, MAPE
[107]	2017	China	PM2.5	MET, Temporal, Spatial, AOD	Daily	365	Yes	Ensemble		RMSE, R2
[108]	2017	Kuwait	PNCs	MET	5min	30	No	NN		RMSE, NRMSE, IA, R2
[109]	2017	Egypt	PM10	MET, Temporal	Hourly	368	No	Regression	1 h	RMSE, R, z’, t-value
[110]	2017	China	NO2, NOx, O3, PM2.5, SO2	MET, Temporal	Daily	2191	No	NN	24 h	RMSE, MAE, IA, R2
[111]	2017	China	AQI	MET	Daily	851	No	Regression		RMSE, MAE, MAPE, MSE
[112]	2016	Qatar	O3, NO2, SO2	MET, Temporal	15min	92	No	Regression	24 h	RMSE, NRMSE, PTA
[113]	2016	France	O3, NO2, PM10	MET	Hourly	1733	No	Hybrid	24 h	RMSE, MAE, NRMSE, MBE, IA, R
[114]	2014	Saudi Arabia	PM10	MET	Hourly	366	No	Regression	1 h	RMSE, MAE, MBE, FACT2, R, IA
[115]	2014	France	O3, NO2, PM10	MET	Hourly	731	Yes	Ensemble	72 h	RMSE
[16]	2013	China	PM1.0, UFP	MET, Traffic, Temporal	Minutely	3	No	Regression, Ensemble, NN		AUC, R, R2, Precision, Recall, f measure, weighted f-measure
[116]	2013	Greece	O3	MET	Hourly	7305	No	NN	6 h	RMSE, R2, R
[117]	2013	India	AQI	MET	Daily	1825	Partially	Ensemble		RMSE, MAE, R
[118]	2012	China	SPM, SO2, NO2, O3	MET	Daily	1095	Yes	Regression	24 h	RMSE, MAE, CWIA, RE
[119]	2012	Iran	CO	MET	Hourly	1492	No	Hybrid	24 h	RMSE, RME, MARE, R2
[120]	2012	Saudi Arabia	O3	MET, Temporal	Minutely	183	No	NN, Ensemble	1 h	MAE, MAPE, SD, MD, R
[121]	2009	Europe	O3	MET, Land Data, Chemical, Emission	Hourly	120	No	Ensemble	24 h	RMSE
[122]	2008	China	RSP(PM10), NOx, SO2	MET	Hourly	61	No	Regression	1 week	RMSE, MAE, WIA

in Appendix A: Year, Case Study, Prediction Target, Dataset Type, Data Rate, Period (Days), Open Data, Algorithm, Time Granularity and Evaluation Metric.

Dataset Type: after examining the selected papers, the following dataset types were extracted (Figure 2): ‘MET’: meteorological data, ‘Spatial’: topographical characteristics, the locations of the stations, ‘Temporal’: includes the day of the month, day of the week, the hour of the day, ‘AOD’: aerosol optical depth, ‘Social Media’: microblog data, ‘Traffic’, ‘PBL Height’: planetary boundary layer height, ‘Land Use’, ‘BEV’: built environment variables, ‘UV Index’: ultraviolet index, ‘SP’: sound pressure, ‘PD’: population density, ‘Human Movements’: floating population and estimated traffic volume, ‘Altitude’, ‘OMI-SO₂’: satellite-retrieved SO₂ from Ozone Monitoring Instrument-SO₂, ‘PPS’: pollution point source, ‘TS’: transportation source, ‘WFD’: weather forecast data, ‘POI Distribution’: point of interest distribution, ‘FAPE’: factory air pollution emission, ‘RND’: road network distribution, ‘Elevation’, ‘AEI’: anthropogenic emission inventory, ‘NDVI’: normalized difference vegetation index, ‘Chemical’: chemical component forecast data (organic carbon, black carbon, sea salt, etc.), ‘Emission’.

From Figure 2 it can be seen that among the 26 dataset types meteorological data is the most used dataset, appearing in eighty-eight publications. The next relatively more frequent dataset types are ‘Temporal’, ‘Spatial’, ‘Traffic’, ‘AOD’ and ‘Land Use’ datasets.

Figure 2 shows the number of publications for each dataset type; however, it is also very important to see the number of publications for dataset combinations. From the dataset types mentioned above, thirty combinations were formed and used in the publications.

Table 2. The number of publications of dataset combinations.

Dataset Combinations	Publications Numbers
MET	45
MET, Temporal	11
MET, Spatial, Temporal	5
Spatial	3
MET, AOD	2
MET, Traffic	2
MET, Social Media	2
Others	23

shows the number of publications for each dataset combinations. The most detected combination is meteorological data jointly only with air quality data, appearing in forty-five papers. It should be noted that there are twenty-three datasets combinations, each of them appears only in one publication, so they are combined as Others for the convenience of further analysis.

To find out dataset features used in each research work, each component of Table A1 in Appendix A was observed in terms of dataset types, and the results of the observation are displayed below.

Year: includes years of publications. Figure 3 demonstrates the distribution of the used dataset combinations over the years, mentioning the number of publications of each published year, and it could help to identify the progress throughout the period.

It can be observed that intensive dataset combinations have been applied since 2016, particularly during 2019 and 2020. Only meteorological data were dominant throughout the whole period. The increase in the number of manuscripts can be attributed to the open data movement promoted by the governments [13]. This aspect will be analysed later.

Case Study: are the countries which were served as a case study in the papers. In the majority of the papers (forty) China was a case study. Here is a list of the rest of the countries with the number of publications: USA-six; Taiwan-six; India-four; Iran-four; South Korea-four; UK-three; Canada-two; Ecuador-two; Egypt-two; Europe-two; France-two; Italy-two; Kuwait-two; Saudi Arabia-two; Turkey-two; Germany-one; Jordan-one; Mongolia-one; Poland-one; Qatar-one; Slovenia-one; Spain-one; Thailand-one; and Tunisia-one. Apart from this examination, it will be helpful also to know dataset combinations for each case study. Figure 4 illustrates the distribution of dataset combinations in terms of the case study. As may be noted, China was a case study in the papers with the majority dataset combinations (China with ‘MET’ is the dominant combination (twenty-one papers)), exclusive of ‘MET, Spatial, Temporal’.

Prediction Target: is the dominant pollutant in a certain area for which prediction different techniques have been performed. In general, seventeen prediction targets were utilised: PM_2.5, O₃, NO_x, PM₁₀, air quality index (AQI), SO₂, carbon monoxide (CO), ultrafine particle (UFP or PM_0.1), particulate matters less than 0.1 micrometers in diameter, air quality health index (AQHI), individual air quality index (IAQI), Ammonia (NH₃), particle number concentrations (PNCs (particle number concentration is the total number of particles per unit volume of air [14])), particles less than 10 nanometers (PN₁₀), black carbon (BC), suspended particulate matter (SPM) and carbon dioxide (CO₂).

As we mentioned in the introduction, there are several indices that help to facilitate the interpretation of air pollution. Figure 5 presents the distribution of dataset combinations in terms of prediction target, and it can be seen, that prediction target can be an individual pollutant, as well as an air quality index. However, the prevailed targets are individual pollutants, particularly, PM_2.5, O₃, NO_x, and PM₁₀, which can be explained with the importance of those pollutants. Moreover, according to the United States Environmental Protection Agency (USEPA), air quality in a certain area is defined by the above-mentioned pollutants [15]. It can be viewed, that PM_2.5 being the most used prediction target (forty-eight papers), was applied in the publications with all the combinations, specially with ‘MET’ it was the most used combination by researchers (twenty-one papers). It is noteworthy, that development of technology gives an opportunity to observe finer particles (PM_0.1, PN₁₀ [16,17]), which have higher toxicity and are easily inhaled.

Data Rate: is the timespan during which the sensors provided data. Figure 6 shows the distribution of dataset combinations in terms of data rate. Overall, biweekly, daily, hourly, minutely, secondly, 15 min, 5 min, 5 s data rates were used in the studies, and nine studies did not provide information about data rate. It can be seen, that hourly data rate being the most used (fifty-six papers) is utilised in the publications with all combinations, particularly with ‘MET’ it was the most used combination by researchers (thirty-two papers).

Period (Days): is the period (the number of days) of the data collection. The summary statistics of these days reveals a mean of days of 1300.63 days (Std.Dev: 1484.68) and a median of 731 days (Min: 3 and Max: 8023).

The most used periods are 365 days in nine papers. The result shows that connecting this feature with the data rate, it can give an idea about the volume of the data used for the analysis. Obviously, it cannot provide any guarantee about the quality of data, and it can include noisy data; however, we assume that the final utilised data were not reduced significantly after the data cleaning process.

Open Data: contains information about data availability. Figure 7 illustrates the distribution of dataset combinations in terms of data availability. There are three categories: Yes, No, Partially. The first two, basically, show if the authors provide or do not provide data used in the studies, the papers with Partially refer to the studies where the authors provided only the part of data. It is interesting to know about data accessibility throughout the period. From Figure 8 it is detectable that since 2012 the authors had started to use open data in their research, which, interestingly, corresponds to the period when the idea of open data portals [18,19] and smart cities [20] has appeared. Figure 9 displays the data availability per case study. It can be observed that China includes all three categories.

It would be also interesting to observe the relation between the authors’ affiliation and the case study of certain research. The results show that in the majority of the papers (55), the affiliations of all the co-authors are located in the corresponding case studies. In eleven papers the author’s affiliations are located in the countries different from case studies. For example, in the following paper [21], the author’s affiliations are located in China and the case study is USA. In twenty-seven papers, the co-authors’ affiliation partially correspond to the case study. For instance, in this paper [22] the case study is Canada and the author’s affiliations belong to China and Canada.

Algorithm: are the ML algorithms on which the applied methods are based. Figure 10 shows the distribution of dataset combinations in terms of ML algorithms. The ML algorithms used in the studies are Neural Network (NN), Regression, Ensemble, Hybrid Model and Other Algorithms. Here are the main methods used in each category: NN—Long Short Term Memory (LSTM), Multilayer Perceptron (MLP), Gated recurrent unit (GRU); Regression—Support Vector Machine (SVM); Ensemble—Random Forest (RF), Extreme Gradient Boosting (XGBoost), Light Gradient Boosted Machine (LightGBM); Hybrid Model—the majority of the methods of this category are based on SVM, for example Partial Least Square-SVM, Multi-output SVM and Multi-Task Learning (MM-SVM); Other Algorithms—includes the works applied Decision Tree Algorithm (C4.8), Reinforcement Learning, Bayesian Model, Regularization and Optimization. It should be pointed out, that in contrast other dataset combinations, ‘MET’ and ‘Others’ include all categories of the algorithms.

Having different prediction targets and methods, it would be valuable to see if there is any relation between targets and applied methods in order to figure out which methods are used to predict a particular target. According to the results of the study, the following connection was detected (main prediction targets and corresponding methods): PM- LSTM, SVM, RF; O₃- MLP, RNN; NO_x-SVM, RF, RNN; SO₂-SVM; CO-LSTM; AQI-SVM.

Time Granularity: is the time interval, for which period the prediction was applied. Figure 11 shows the distribution of dataset combinations in terms of time resolution. The used time resolutions are 1 h, 2 h, 3 h, 4 h, 6 h, 8 h, 10 h, 12 h, 24 h, 48 h, 72 h, five days, one week, 15 days and one month. It must be mentioned that these extracted intervals are the maximum intervals applied in each article. It is detectable that 24 h is the most used time resolution regarding the number of publications and different dataset combinations. Furthermore, it can be seen, that the most extended prediction time resolution, one month, is applied in publication with ‘Others’ combination, and considering that the longer resolution decreases the accuracy, it can be seen that there is only one paper implemented prediction for one month.

Evaluation Metric: are the measures which were used to evaluate the applied method. Overall, sixty-nine metrics were used to evaluate the methods, from which the most used metrics are Root Mean Square Error (RMSE) in seventy-seven papers, Mean Absolute Error (MAE) in forty-two papers. Figure 12 demonstrates the distribution of dataset combinations in terms of evaluation metric (each database combination is marked with a different color). It can be shown, that compared to other dataset types ‘MET’, ‘MET, Temporal’ and ‘Others’ were combined with more metrics, particularly, RMSE with ‘MET’ (forty-one papers) and MAE with ‘MET’ (twenty-four papers) are the most used combinations. Additionally, taking into consideration the most used prediction target (PM_2.5) and the most used time resolution (24 h), the results show that PM_2.5 was a prediction target in eighteen papers with the combination of RMSE and ‘MET’, and in ten papers with the combination of MAE with ‘MET’, and 24 h was a predicted time resolution in ten papers with RMSE and ‘MET’ combination and in six papers with MAE and ‘MET’ combination. Furthermore, the metrics that have been used in more than six publications with corresponding equations and descriptions are extracted and displayed in

Table 3. The most used metrics (more than six publications) with corresponding equations and definitions (where N is the number of predict days, O_i and P_i are the observed and predict values, respectively, and $\overline{O_{\mathrm{i}}}$ is the average of observed data).

Metrics	Equations	Description
RMSE	$\sqrt{\frac{1}{N}{\sum }_{i=1}^N{(O_{\mathrm{i}}-P_{\mathrm{i}})}^2}$	It measures the geometric difference between observed and predict data.
MAE	$\frac{1}{N}{\sum }_{i=1}^N\left|O_{\mathrm{i}}-P_{\mathrm{i}}\right|$	It measures the average magnitude of the errors in a set of predictions, without considering their direction.
R2	$\frac{{\left({\sum }_{i=1}^N(P_{\mathrm{i}}-\overline{P_{\mathrm{i}}})(O_{\mathrm{i}}-\overline{O_{\mathrm{i}}})\right)}^2}{{\sum }_{i=1}^N{(P_{\mathrm{i}}-\overline{P_{\mathrm{i}}})}^2{\sum }_{i=1}^N{(O_{\mathrm{i}}-\overline{O_{\mathrm{i}}})}^2}$	It shows how differences in one variable can be explained by a difference in a second variable.
R	$\frac{{\sum }_{i=1}^N(P_{\mathrm{i}}-\overline{P_{\mathrm{i}}})(O_{\mathrm{i}}-\overline{O_{\mathrm{i}}})}{\sqrt{{\sum }_{i=1}^N{(P_{\mathrm{i}}-\overline{P_{\mathrm{i}}})}^2{\sum }_{i=1}^N{(O_{\mathrm{i}}-\overline{O_{\mathrm{i}}})}^2}}$	It measures the strength and the direction of a linear relationship between two variables.
MAPE	$\frac{{\sum }_{i=1}^N\left|O_{\mathrm{i}}-P_{\mathrm{i}}\right|}{N}\times 100\%$	It measures the size of the error in percentage terms.
IA	$1-\frac{{\sum }_{i=1}^N{(O_{\mathrm{i}}-P_{\mathrm{i}})}^2}{{\sum }_{i=1}^N{(\left|O_{\mathrm{i}}-\overline{O_{\mathrm{i}}}\right|+\left|P_{\mathrm{i}}-\overline{O_{\mathrm{i}}}\right|)}^2}$	It is the ratio of the mean square error and the potential error.
MSE	$\frac{1}{N}{\sum }_{i=1}^N{(O_{\mathrm{i}}-P_{\mathrm{i}})}^2$	It measures the average squared difference between the observed and the predict values
NRMSE	$\frac{1}{O\min -O\max }\sqrt{\frac{1}{N}{\sum }_{i=1}^N{(O_{\mathrm{i}}-P_{\mathrm{i}})}^2}$	It is the normalised version of RMSE, which makes easier to compare different models with different scales.

. The metrics are RMSE, MAE, Coefficient of Determination (R²), Correlation Coefficient (R), Mean Absolute Percentage Error (MAPE), Index of Agreement (IA), Mean Square Error (MSE), Normalised Root Mean Square Error (NRMSE) [23,24,25,26,27,28,29,30].

Another point to which attention should be paid is understanding in the world of evaluation metrics how to choose the best and the most acceptable model. To select the best model, the majority of the authors selected different benchmark models and, applying the same validation metrics to all models, chose the outperformed model. Only a few authors, such as Goulier et al. [31] and Zhang et al. [32] have focused on the importance to test whether the model performs well enough, acceptable or not. It is important to follow up on evaluation studies to ensure that the evaluation procedure is correct. For example, the articles by Kadiyala and Kumar [33], Alexander et al. [34], Janssen et al. [35] can serve as a guide for researchers.

It is worth mentioning the limitations noted by the authors in their works. The accuracy of model performance depends on many factors, such as ML algorithms, spatial characteristics, prediction targets, temporal resolution, etc. Several authors have mentioned the structural limitations of algorithms, such as the tendency to overfit, complexity, difficulty with interpretation, and time-consuming [36,37,38]. Regarding the prediction target, depending on which pollutant is the prediction target the accuracy may vary since the chemical structure of the pollutants is different. For example, Li et al. in their study [39] found out that the proposed model predicts better PM_2.5 than NO_x, as NO_x is highly reactive and has larger temporal variability. Therefore, many studies mentioned the implementation of the proposed model for predicting other pollutants as future work [21,40]. Another limitation is the lack of data in spatiotemporal resolution [41,42]. Missing values can also be included in this scope, depending on their quantity, the performance can be significantly reduced [43,44]. An important factor is the presence of sudden changes. One solution might be to collect more data, as the training dataset will include more sudden changes, which in turn will lead to better performance in case of sudden changes [42]. Including other datasets such as aerosol optical depth data and meteorological data can help to overcome this issue [45]. It might also be useful to apply techniques for handling imbalanced datasets [40]. Another limitation that we have already mentioned is a prediction with the long temporal resolution since due to the accumulated error, the accuracy decreases as the temporal resolution increases [46,47].

4. Conclusions

Predicting air quality with higher accuracy is gaining in importance and necessity day by day. Therefore, it is very essential to explore the state-of-the-art of the field. Of the numerous aspects that exist in the field of research, this article, through reviewing studies, focuses on datasets in order to examine which datasets are used by researchers and to identify additional variables that they have taken into account in their analysis to predict air quality. A set of the most relevant papers in this field have been selected using ACM, IEEE Xplore, Scopus and WoS databases. Overall, ninety-three papers were selected, reviewed and, afterwards, the essential dataset features were extracted and synthesised (Year, Case Study, Prediction Target, Dataset Type, Data Rate, Period (Days), Open Data, Algorithm and Time Granularity). The results show that twenty-six datasets are used to supplement data collected by air quality sensors, including ‘MET’, ‘Temporal’, ‘Spatial’ and ‘Social Media’, among others. The results show a significant difference on the use of ‘MET’, which is the dominant dataset used in 94.6% of the studies, and 48.4% of the studies combined with only air quality data.

Regarding data availability, it was shown that since 2012 a new stage has begun, associated with the use of open data portals [48], which is crucial for science and contributes to the improvement and development of various research fields and encourages the emergence of new exciting results, which, in turn, has also led to an increase in the number of publications.

A very important finding is to explore and understand which methods are most commonly used and dominant in the field to predict a specific target, for example, to predict particulate matter, LSTM, SVM and RF were found to be the most commonly used methods.

In general, it may be inferred that extra datasets can have significant importance, and involving them in the analysis could improve air quality prediction and obtain more accurate results. However, it is difficult to indicate which datasets are more valuable and it should also be noted that it is not always advisable to include many datasets, as having a huge dataset can be a problem as it requires more training time and may contain redundant data.

Therefore, future work can be addressed to the establishment of a framework based on the same conditions (model, prediction target, evaluation metric, time resolution) with the objective to validate and compare the improvement of each dataset type.