Improving the Estimation of PM_2.5 Concentration in the North China Area by Introducing an Attention Mechanism into Random Forest

Abstract

Fine particulate matter with an aerodynamic diameter less than 2.5 µm ($\mathrm{P}{\mathrm{M}}_{2.5}$) profoundly affects environmental systems, human health and economic structures. Multi-source data and advanced machine or deep-learning methods have provided a new chance for estimating the $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations at a high spatiotemporal resolution. In this paper, the Random Forest (RF) algorithm was applied to estimate hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ of the North China area (Beijing–Tianjin–Hebei, BTH) based on the next-generation geostationary meteorological satellite Himawari-8/AHI (Advanced Himawari Imager) aerosol optical depth (AOD) products. To improve the estimation of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration across large areas, we construct a method for co-weighting the environmental similarity and the geographical distances by using an attention mechanism so that it can efficiently characterize the influence of spatial–temporal information hidden in adjacent ground monitoring sites. In experiment results, the hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ estimates are well correlated with ground measurements in BTH, with a coefficient of determination (${\mathrm{R}}^2$) of 0.887, a root-mean-square error (RMSE) of 18.31 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and a mean absolute error (MAE) of 11.17 µg/${\mathrm{m}}^3$, indicating good model performance. In addition, this paper makes a comprehensive analysis of the effectiveness of multi-source data in the estimation process, in this way, to simplify the model structure and improve the estimation efficiency of the model while ensuring its accuracy.

1. Introduction

China’s economy has developed rapidly in the past few decades, but it also faced serious air pollution problems, especially in the Beijing–Tianjin–Hebei (BTH) region [1]. Fine particulate matter with aerodynamic diameters less than 2.5 $\mathsf{\mu }$m ($\mathrm{P}{\mathrm{M}}_{2.5}$) has been a major issue of public concern [2], associated with the risk to public health [3] and the impact on climate change [4] and economic structures [5,6]. Three common methods, such as ground station measurements, model simulations and satellite estimations can all provide $\mathrm{P}{\mathrm{M}}_{2.5}$ data. However, differences in data quality make it difficult for a single data source to meet high-precision data requirements.

Currently, numerous studies have estimated the concentration of fine particulate matter by employing satellite-based aerosol optical depth (AOD) products, and the data quality is improved by integrating other multi-source data [7,8,9]. Moderate Resolution Imaging Spectroradiometer (MODIS) AOD products are the most widely available data for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation [10,11,12]. In the early studies, the Dark Target (DT) and Deep Blue (DB) aerosol retrieval algorithms in MODIS provided global daily AOD products only at 10-km. The application of these products in studies regarding $\mathrm{P}{\mathrm{M}}_{2.5}$ is limited by their coarse spatial resolution, such as the first global mapping about $\mathrm{P}{\mathrm{M}}_{2.5}$ spatia-l-temporal distribution [13]. With the improvement of the AOD retrieval algorithm, the quality of AOD and $\mathrm{P}{\mathrm{M}}_{2.5}$ data has also been improved [14,15,16,17], especially the daily AOD at 1 km produced by the multi-angle implementation of atmospheric correction (MAIAC) algorithms [18,19,20]. In addition, some $\mathrm{P}{\mathrm{M}}_{2.5}$ products using geostationary satellite AOD data can reach hourly levels, such as Himawari-8/AHI (Advanced Himawari Imager) [21,22] and Fengyun-4A/AGRI (the Advanced Geosynchronous Radiation Imager) [23,24], but the spatial resolution is still limited.

In most AOD-$\mathrm{P}{\mathrm{M}}_{2.5}$ estimation models, the minimum data unit is a vector of the site $\mathrm{P}{\mathrm{M}}_{2.5}$ value and the corresponding one-dimensional correlation factor (e.g., AOD, meteorological). Therefore, both linear and nonlinear models are fitted to a class of tabular feature data [25]. Some statistical regression models are simple and easy to implement, such as linear regression (LR) [26], linear mixed-effects models (LME) [27], geographically weighted regression (GWR) [14], geographically and temporally weighted regression (GTWR) [28], and some hybrid models [29,30,31]. Essentially, these liner models with local variation have difficulty capturing complex spatiotemporal relationships and have limited accuracy. In recent years, some deep learning and data-driven methods achieved promising results in remote sensing [32,33,34,35]. Machine learning methods, with their powerful ability to establish complex nonlinear relationships between various interacting predictor variables, are emerging as the dominant estimation method, such as artificial neural networks [36] and geo-intelligent deep belief networks [37].

In particular, some decision-tree-based models are more advantageous in fitting data with tabular features, such as the random forest (RF) [11,19,21], the Light Gradient Boosting Machine (LightGBM) [22], and the eXtreme Gradient Boosting (XGBoost) [38]. They are based on the idea of bootstrap aggregating and the random subspace method [39]. It aggregates a set of weak learners to form a strong one, and these weak learners, i.e., basic decision trees, are trained by randomly bootstrapped samples from the training set. Moreover, the node splitting features are also randomly selected during the training process of the decision tree. Thus, it can nonparametrically model the indefinable and complex non-linear relationship among input features. However, these decision-tree-based models are difficult to capture spatio-temporal correlation features of neighboring sites. Therefore, it is an important way to extract the spatio-temporal correlation via other different methods, as well as to construct a hybrid model for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. In addition, in contrast to some other deep learning methods in the study of intelligent processing of remote sensing information [40,41], the full exploitation of multi-source data as well as spatio-temporal information plays a key role in the efficiency of the model [42,43].

There are two main hybrid model ways to achieve better $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration estimation [8]: one is to fuse multiple sources of data to improve spatio-temporal resolution, and the other is to assemble other methods for capturing spatio-temporal correlation features of neighboring sites. Typically, these data fusion methods simply complement the spatio-temporal missing data, and it is difficult to verify the effectiveness of the data. The most essential characteristic of $\mathrm{P}{\mathrm{M}}_{2.5}$ is its spatial and temporal heterogeneity. Many researchers have tried to solve this problem by using geographical association information, which is constructed by weighting spatio-temporal distance from ground-based $\mathrm{P}{\mathrm{M}}_{2.5}$ measurements [19,20,37]. However, ground-based $\mathrm{P}{\mathrm{M}}_{2.5}$ monitoring sites are usually sparsely distributed in urban areas. Common weight methods with spatio-temporal distance is prone to over fitting in training. Moreover, its verifiability and effectiveness will be dramatically reduced when actually applied to large scale $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration estimation.

In this paper, we improve the estimation of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration across large areas using multi-source data by aggregating random forest and the attention mechanism. The attention mechanism-based method is proposed to characterize the influence of spatial–temporal information hidden in neighboring $\mathrm{P}{\mathrm{M}}_{2.5}$ monitoring sites. Then, the extracted features are connected with a random forest model for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. In addition, we verify the effect of AOD data quality on the final estimation accuracy of the model and analysis of the feature importance ranking results to form some explanatory notes on the estimation model. Finally, $\mathrm{P}{\mathrm{M}}_{2.5}$ estimations were formed by fusing satellite and model data, and we make a comparative analysis of spatial–temporal information at different scales to further demonstrates the rationality of the estimated results.

2. Materials and Methods

2.1. Datasets and Preprocessing

The experimental data in this paper include the hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ observations from monitoring stations, 5 km resolution AOD products of Himawari-8, and other auxiliary data.

Table 1. Summary of the dataset.

Type	Abbreviation	Content	Spatial Resolution	Temporal Resolution	Source
$\mathrm{P}{\mathrm{M}}_{2.5}$	$\mathrm{P}{\mathrm{M}}_{2.5}$	$\mathrm{P}{\mathrm{M}}_{2.5}$	site	hourly	CNEMC
AOD	H-AOD	Himawari AOD	5 km	hourly	JAXA
	M-AOD	Himawari model AOD	5 km	hourly	JAXA
Meteorological	TEM	2 m air temperature	0.1° × 0.1°	hourly	ECMWF
	UW	10 m u-component of wind	0.1° × 0.1°	hourly	ECMWF
	VW	10 m v-component of wind	0.1° × 0.1°	hourly	ECMWF
	PRE	Total precipitation	0.1° × 0.1°	hourly	ECMWF
	SP	Surface pressure	0.1° × 0.1°	hourly	ECMWF
	BLH	Boundary layer height	0.25° × 0.25°	hourly	ECMWF
	RH	Relative humidity	0.25° × 0.25°	hourly	ECMWF
Land-related	NDVI	NDVI	1 km	16-day	MYD13A2
	DEM	DEM	90 m	-	SRTM
	LULC	LULC	500 m	annually	MCD12Q1

lists all the information about the data used in this study. The study area in this paper is the Beijing–Tianjin–Hebei urban agglomeration, with the geographical location of 113.45° E 119.83° E and 36.08° N 42.67° N. Figure 1 shows the geographical location of the region and the distribution of the state-controlled environmental monitoring sites located within the region. The elevation map in the figure shows that the topographic distribution of the region has its own distinctive characteristics. The Beijing–Tianjin–Hebei urban agglomeration has a high population concentration and is one of the core economic regions in China. In addition, with the air pollution control in recent years, the spatial pattern of some major steel, coal, power, and light industries in the Beijing-ring region has changed dramatically. As one of the heavy industrial regions in China, most of the industrial production has shifted to the southern part of Hebei province, which further leads to the clustering of heavy industry in this areas. In terms of topography and climatic environment, air pollution tends to accumulate in these regions. In terms of pollution sources, the periphery of the region is a heavy industrial agglomeration, which has a certain impact on the air quality of Beijing in the central region of the urban agglomeration.

2.1.1. Ground-Level $\mathrm{P}{\mathrm{M}}_{2.5}$ Measurements

Hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ observations from monitoring stations in BTH of 2017 are collected for model fitting and validation. They are publicly available on the website of the China Environmental Monitoring Centre (CEMC, http://www.cnemc.cn/, accessed on 7 February 2024). As shown in Figure 1, there are about 80 monitoring sites in total, and most of them are distributed in the city areas of BTH.

2.1.2. AOD Data

Himawari-8 is a Japanese geostationary satellite that launched on 7 October 2014. It carries an Advanced Himawari Imager (AHI) and produces hourly AOD at a 5 km resolution during the daytime. In recent years, Himawari-8 AOD has been evaluated and used increasingly in the study of China air pollution [21,22,44]. Here, we obtained official Himawari-8 AOD (Level 3 Version 3.0) at 500 nm in 2017 and used AOD-merged data, which are with broader spatial and temporal coverage and slightly better quality [45]. Meanwhile, model-assimilation AOD products of Himawari-8 were selected for achieving full-coverage $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. This product is the forecast (every hour) of aerosol properties by the MRI/JMA global aerosol model called the Model of Aerosol Species in the Global Atmosphere (MASINGAR). This product is assimilated by Himawari L3 aerosol optical depth at 00, 03, 06, and 09 UTC. There are quality differences between these AOD data, which could help to verify the effect of AOD data quality on the final $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation accuracy of the model.

2.1.3. Auxiliary Data

Auxiliary data consist of meteorological and land-related data. They are helpful for capturing the complex AOD-$\mathrm{P}{\mathrm{M}}_{2.5}$ relationships associated with spatiotemporal variations. The meteorological data are extracted from the European Centre for Medium-Range Weather Forecasts (ECMWF, https://www.ecmwf.int, accessed on 7 February 2024) and ERA5-Land reanalysis products, including temperature (TEM; unit: K), surface pressure (SP; unit: hPa), relative humidity (RH; unit: %), precipitation (PRE; unit: mm), and 10 m u-component and v-component of wind (UW/VW; unit: m/s). The boundary layer height (BLH; unit: m) is obtained from the ERA5 hourly data on single levels. In addition, this study adopted three land-related datasets, including the normalized difference vegetation index (NDVI) products derived from the MODIS MYD13A2, the digital elevation model (DEM) products inferred from the shuttle radar topography mission, and the land use/cover (LULC) products derived from the MODIS MCD12Q1.

2.1.4. Processing of Data

Although the spatial and temporal resolution of current satellites has been greatly improved, it is difficult for a single data source to meet the high-precision requirement of all applications. Moreover, due to the influence of cloud coverage, complex surface conditions, and other factors, satellite AOD data often also have a large amount of spatial information missing. In order to obtain full-coverage of $\mathrm{P}{\mathrm{M}}_{2.5}$ spatial and temporal distribution data, the research on the reconstruction of missing information in satellite remote sensing estimated $\mathrm{P}{\mathrm{M}}_{2.5}$ can be summarized into two main aspects [8,17,31]: one is to prioritize the reconstruction of AOD missing data, so as to reduce or eliminate the spatial missingness of estimated $\mathrm{P}{\mathrm{M}}_{2.5}$ data; the others is to directly use single-source satellite AOD data for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation and then seamlessly reconstruct it by fusing multiple sources of $\mathrm{P}{\mathrm{M}}_{2.5}$.

In order to achieve $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation at 1 km, we improved the official Himawari-8 hourly AOD at a 5 km resolution to 1 km by using the AeroCGAN model [46]. Meanwhile, estimation experiments were conducted separately using three types of AOD data: official Himawari-8 AOD (H-AOD), official Himawari-8 model-AOD (M-AOD), and mixed-AOD (Mix-AOD) of the above. Then, according to different AODs with ground-based monitoring $\mathrm{P}{\mathrm{M}}_{2.5}$, we make a temporal and spatial matching processing for AODs and relevant auxiliary data, and then form a control variable experiment dataset. In this way, we will verify the effect of AOD data quality on the final estimation accuracy of the model and achieve full-coverage $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation.

In addition, since the auxiliary data are diverse in terms of spatial resolutions, they are resampled to a 1 km resolution with bilinear interpolation to ensure data consistency. Bilinear interpolation can produce smooth interpolation results with the distance-weighted average of the four nearest pixels.

After data processing, based on the data in the study area in 2017, there are 115,029 valid samples matched for satellite observation AOD data, and 222,547 valid samples data matched for both model-assimilation AOD data and mixed AOD data. These data will be used for model training and validation.

2.2. Methods

Supported by abundant observational data and machine learning methods, the main current $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation methods are essentially optimization processes that are searching for the relationship between $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations and observational data. More formally, the commonly used model can be expressed as $\mathrm{P}{\mathrm{M}}_{2.5}$ = f (AOD, meteorological, land-related, geographic factors).

Typically, the AOD is essential, basic data. Meteorological factors and land-related data are listed in Table 1; they are helpful for capturing the complex AOD-$\mathrm{P}{\mathrm{M}}_{2.5}$ relationships associated with spatiotemporal variations. In addition, geographic factors, such as the neighboring ground-based $\mathrm{P}{\mathrm{M}}_{2.5}$ measurements, are valuable prior information for spatial correlations. Previous studies measured this correlation by weighting spatio-temporal distance [19,37]. However, as shown in Figure 2, ground-based $\mathrm{P}{\mathrm{M}}_{2.5}$ monitoring sites are usually sparsely distributed in urban areas. Moreover, such correlations are usually dynamic and highly susceptible to changes in time and local environmental conditions. When the correlations are constructed on the basis of spatial distance [19,37], in most studies, the weight of such correlations is not consistent with the spatial and temporal heterogeneity of $\mathrm{P}{\mathrm{M}}_{2.5}$. For example, when several neighboring stations with similar spatial distances are in the upstream and downstream channels of pollution dispersion, it is obvious that the upwind stations are less affected, but the downwind stations are relatively more affected. It is difficult to distinguish their differences with the correlation only based on distance.

According to the above analysis, common weight methods with spatio-temporal distance show strong effectiveness in spatio-temporal correlation information when training the model with training data (like ${\mathrm{P}}_1$), but they provide weak effectiveness when actually applied to large-area data (like ${\mathrm{P}}_2$), as in Figure 2.

2.2.1. Spatial Features

In order to more efficiently utilize the spatially correlated prior information provided by the observations of neighboring sites, we construct a co-weighting similarity of spatial distance and environment factors using the attention mechanism. Attention mechanism-based neural network models have been widely used in computer vision, natural language processing, and spatio-temporal prediction tasks [47,48]. Essentially, the attention mechanism is used to calculate the similarity relationship of the vectors and learn the features of the data accordingly. Inspired by this mechanism, we designed the new spatial feature extraction method.

For the $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration of a target pixel, its neighboring $\mathrm{P}{\mathrm{M}}_{2.5}$ sites usually show a certain correlation to it. Just like in the first law of geography, all things are related, but nearby things are more related than distant things [49]. This law can interpret previous studies that constructed this correlation by weighted spatio-temporal distances [19,37]. However, the impact of weights are highly dynamic, when changing over time and with environmental factors. Therefore, this study further considers the third law of geography: the more similar the geographic configurations of two points (areas), the more similar the values (processes) of the target variable are at these two points (areas) [50]. With this law, we construct co-weights of spatial distance (the first law) and environmental similarity (the third law) by using an attention mechanism. Moreover, since there might be many irrelevant series, it results in very high computational cost and degrades the performance if directly using all kinds of time series as the encoder inputs to capture the correlations between different observation data. Therefore, we further synergize the environmental similarity and spatial distance to capture the dynamic correlations by using a designed attention mechanism.

Let $s_{target}$ be the site of our estimated target, and its neighboring sites are $S=[s_1,\cdots ,s_i,\cdots ,s_N]$, every site $s_i$ has an environment vector ${\mathbf{e}}_i$ and a $\mathrm{P}{\mathrm{M}}_{2.5}$ value $p_i$.

$$\begin{array}{c}\hfill {\mathbf{e}}_i=\left[AOD,TEM,RH,BLH,WS,WD,NDVI\right]\end{array}$$

(1)

where AOD, TEM, RH, BLH, WS, WD, and NDVI are defined in Table 1. Furthermore, $s_{target}$ has an environment vector ${\mathbf{e}}_{target}$ like ${\mathbf{e}}_i$. To reduce the computational complexity of the attention mechanism in extracting spatially correlated features, only these seven variables were selected for the environmental similarity extraction. Then, we calculate the attention weights ${Sim}_i$ (i.e., impacting weight) between neighboring sites as follows:

$$\begin{array}{c}\hfill {Sim}_i=\eta tanh\left({\mathbf{W}}_i\left[{\mathbf{e}}_i;{\mathbf{e}}_{target}\right]+{\mathbf{K}}_i{\mathbf{e}}_{target}+{\mathbf{b}}_i\right)\end{array}$$

(2)

where $\left[{\mathbf{e}}_i;{\mathbf{e}}_{target}\right]$ is a concentration operation between ${\mathbf{e}}_i$ and ${\mathbf{e}}_{target}$. $\eta$, ${\mathbf{W}}_i$ , ${\mathbf{K}}_i$, and ${\mathbf{b}}_i$ are the learnable parameters.

Then, a SoftMax function operation is performed to normalize all attention weights. In this process, we construct the neighboring site attention weights considering both spatial distance and environmental similarity.

$$\begin{array}{c}\hfill a_i=Softmax\left({Sim}_i\right)=\frac{\left.exp\left(\left(1-l_i\right){Sim}_i+l_i\right)\right)}{{\sum }_{j=1}^{N}exp\left(\left(1-l_j\right){Sim}_j+l_j\right)}\end{array}$$

(3)

where $l_i$ is the geospatial distance weight, and the specific value is the reciprocal of the spatial distance $\left(\frac{1}{d_i}\right)$ between the site target site $s_{target}$ and neighboring site $s_i$. Similarity $a_i$ indicates the attention value between the i-th neighboring site and the site to be estimated.

The attention weights of spatial features are jointly determined by the input features and the spatial distance in the encoder. The score calculated by Equation (3) semantically represents the site correlation under the effect of distance and environment. When the spatial distance between two sites is closer, the effect of spatial distance weight is higher than the environmental similarity weight, thus the correlation between them is higher; on the contrary, when the spatial distance between two sites is farther, the effect of the environmental similarity weight is higher than that of the spatial distance weight, thus the environmental similarity effect is higher.

Finally, we weighted the spatial correlation of $\mathrm{P}{\mathrm{M}}_{2.5}$ for all spatially neighboring sites and obtained the neighboring spatial features ${\mathrm{SAtt}}_{s_{target}}$ of the site to be estimated as follows:

$$\begin{array}{c}\hfill {\mathrm{SAtt}}_{s_{target}}=\sum _{i=1}^n{a}_i·p_i\end{array}$$

(4)

where $a_i$ is the spatial distance and environmental similarity co-weighting by Equation (3), and $p_i$ is the $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration at the i-th site.

In addition, considering the numerical differences among the variables, each variable was normalized separately before feature extraction.

2.2.2. Temporal Features

The time stamp could be a temporal feature in some machine learning methods; especially, the minimum data unit of this study is a vector of features. Moreover, numerous studies have verified that the values of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations show strong variation and correlation patterns across the geographical space and the temporal dimension [22,51,52]. The sites in the BTH often observe much higher $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations in winter than summer. The $\mathrm{P}{\mathrm{M}}_{2.5}$-AOD correlation became noticeably higher from 9:00 to 17:00 local time, whereas the $\mathrm{P}{\mathrm{M}}_{2.5}$/AOD ratio notably decreased in the Beijing–Tianjin–Hebei, Yangtze River Delta, and Chengyu regions. Specifically, there is a high correlation between 12:00 and 14:00 LT (Local Time) and 13:00 and 17:00 LT. The ratio in a day has a clear unimodal pattern, with the peak occurring, particularly in the fall and winter, at about 10:00 or 11:00 LT. The $\mathrm{P}{\mathrm{M}}_{2.5}$-AOD association significantly varies over the course of a week in the winter. Additionally, the winter is the period of time when most metropolitan agglomerations have their best association and highest ratio [53]. Therefore, considering that our approach aims to estimate hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations, we selected month and hour as temporal features.

2.2.3. Random Forest

Random forest is based on the idea of bootstrap aggregating and the random subspace method [39]. It aggregates a set of weak learners to form a strong one; these weak learners, i.e., basic decision trees, are trained by randomly bootstrapped samples from the training set. Moreover, the node splitting features are also randomly selected during the training process of the decision tree. Thus, it can nonparametrically model the indefinable and complex non-linear relationship among input features [11,19].

Decision tree-based models, such as RF, XGBoost, and LightGBM, are more advantageous in fitting data with tabular features as well as $\mathrm{P}{\mathrm{M}}_{2.5}$ estimating. Previous studies have demonstrated that there is little difference in performance between these decision tree-based models in $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation [38]. Compared to image or natural language data, tabular data are heterogeneous, resulting in dense numbers and sparse categorical features. In addition, the correlation between features is weaker than the spatial or semantic relationships in image or natural language data. The variables of tabular data can be correlated or independent, and features usually have no positional information. Therefore, it is necessary to discover and exploit correlations relying on spatial information. The spatio-temporal correlation feature extraction is the primary optimization direction of these $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation models.

Especially, $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations show strong variation and correlation patterns across the geographical space and the temporal dimension. It is difficult to capture such features by relying on random forest alone to mine the value of neighboring site data. Therefore, this study selected the classic random forest model and aggregating attention mechanism to achieve $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. With the attention mechanism described above, we obtain new spatio-temporal features that are fed into the random forest together with the auxiliary data.

There are three key parameters that should be tuned in a random forest model, which are the number of trees in the forest (N), the number of input features to consider when splitting data at a decision node (m), and the minimum number of samples required to be at a leaf node (n). Through controlled variable experiments, we finally set each of these three parameters as 500, 13, and 2, respectively. Actually, this is also related to the amount of our data and the number of features.

2.2.4. Model and Evaluation

According to the above analysis, the flowchart of the proposed attention mechanism-based random forest model is shown in Figure 3, including input data, feature processing, RF regression, and output data. Firstly, we made a temporal and spatial matching processing for AODs and relevant auxiliary data to form a training dataset. Then, we used the attention mechanism-based spatial correlation extraction method (Equations (1)–(4)) to obtain the spatial feature. Furthermore, the time stamp is chosen as the temporal feature. These above data are used as input features of the random forest model to finally estimate $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations. Here, the steps of the attention mechanism-based random forest are summarized in Algorithm 1.

Algorithm 1 Attention Mechanism-Based Random Forest Algorithm for $\mathrm{P}{\mathrm{M}}_{2.5}$ Estimation

Input: Dataset $D=\left\{({\mathbf{x}}_1,y_1),\cdots ,({\mathbf{x}}_i,y_i),\cdots ,({\mathbf{x}}_N,y_N)\right\}$, each ${\mathbf{x}}_i$ contains F features (defined in Table 1). Training times is T. The random forest RF has M decision trees (DTree). 1: SAtt ← Equations (1) to (4); attention mechanism-based spatial correlation 2: TF; time stamp as temporal features 3: ${\mathbf{x}}_i^{\prime }\leftarrow$ [${\mathbf{x}}_i$, SAtti, TFi]; features concat 4: $D^{\prime }=\left\{({\mathbf{x}}_1^{\prime },y_1),\cdots ,({\mathbf{x}}_i^{\prime },y_i),\cdots ,({\mathbf{x}}_N^{\prime },y_N)\right\}$; each ${\mathbf{x}}_i^{\prime }$ contains $F^{\prime }$ features (i.e., dimension); 5: fort = 1, 2, …, T do 6:     randomly bootstrapped K samples: 7:     $D_1,\cdots ,D_k,\cdots ,D_K\in D^{\prime }$ 8:     for $D_k\in D^{\prime }$ do 9:           Randomly select f feature subsets from the $F^{\prime }$-dimension features; 10:         training each decision tree: 11:         for DTreei in RF do 12:            select the optimal one from the f features for tree node splitting 13:            DTreei $\leftarrow \{D_k,f\}$ 14:         end for 15:     end for 16:     RFt← {DTree1, $\cdots ,$DTreei, $\cdots ,$DTreeM} 17: end for 18: return RF

In addition, based on previous studies [8,19,28], 10-fold cross-validation [54] is adopted to investigate the effectiveness of the proposed method. The general 10-fold cross-validation randomly partitions all the samples into 10 folds with an equal number of subsamples, which is also referred to as sample-based cross-validation. One subsample is retained for validation, and the remaining 9 subsamples are used for training. This process of subsampling is repeated 10 times with each fold for validation in return. The overall performance of the model is estimated by averaging the 10 evaluation results to reduce the random effect.

During the evaluation stage, three quality metrics are used to measure the performance: determination coefficient (${\mathrm{R}}^2$), root-mean-square error (RMSE), and mean absolute error (MAE). They are denoted as follows:

$$R^2=\frac{\begin{array}{c}{\sum }_{i=1}^n{(P_i-\overline{Q_i})}^2\end{array}}{\begin{array}{c}{\sum }_{i=1}^n{(O_i-\overline{Q_i})}^2\end{array}}$$

(5)

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^n{(P_i-Q_i)}^2}$$

(6)

$$MAE=\frac{1}{n}\sum _{i=1}^n|P_i-Q_i|$$

(7)

where $O_i$ is the obvious value, $\overline{Q_i}$ is the average of obvious values, $P_i$ is the estimate value, and n is number of data points.

3. Results and Discussion

3.1. Model Performance

Table 2. Comparison of performance with other methods.

Methods	Resolution	${\mathrm{R}}^2$	RMSE	MAE	Source AOD	Period	Region	Reference
LME	10 km, daily	0.79	26.74	-	MODIS	2015	China	Ma et al. (2016) [27]
GWR	10 km, daily	0.64	32.98	21.25	MODIS	2013	China	Ma et al. (2014) [14]
GTWR	3 km, daily	0.80	18.00	12.03	MODIS	2015	China	He et al. (2018) [28]
Geo-DBN	10 km, daily	0.88	13.03	8.54	MODIS	2015	China	Li et al. (2017) [37]
DNN	1 km, hourly	0.84	19.90	11.89	Himawari	2017	BTH	Sun et al. (2019)
Two-stage	1 km, daily	0.85	11.02	-	MODIS, Himawari	201804–201902	China	Jiang et al. (2021) [26]
STRF	1 km, daily	0.85	15.57	9.77	MODIS MAIAC	2015–2016	China	Wei et al. (2019) [19]
STET	1 km, daily	0.89	10.35	6.71	MODIS MAIAC	2017–2018	China	Wei et al. (2020)
STLG	5 km, hourly	0.85	13.09	8.11	Himawari	2018	China	Wei et al. (2021)[22]
XGBoost	5 km, hourly	0.84	18.10	11.40	Himawari	2016	Central and Eastern China	Chen et al. (2019) [38]
RF	1 km, hourly	0.81	25.51	15.91	Himawari	2017	BTH	This study
STAttenRF	1 km, hourly	0.89	18.31	11.17	Himawari	2017	BTH	This study

shows the estimation performances of the proposed and some existing methods. With the development of analysis methods, the early geographically weighted regression model (GWR) is gradually replaced by some new methods that are based on machine learning (XGBoost, Two-stage, STRF). Furthermore, the stream trend of the current research is to further improve the performance of the machine learning method by adding spatio-temporal features. In addition, it can be seen that the accuracy for most of the model estimation is within a certain range (${\mathrm{R}}^2$: 0.65 to 0.90; RMSE: 10.0 to 30.0 $\mathsf{\mu }$g/${\mathrm{m}}^3$, approximately). This may be due to the difference in the spatial and temporal resolution of the data or the spatial and temporal range of the research. For example, some early studies for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation [14,37] are mostly based on 10 km resolution AOD data from MODIS satellite observations. Then, to benefit from the improvement in AOD data quality (such as MODIS MAIAC AOD with 1 km spatial resolution, Himawari AOD with 5 km hourly resolution, etc.) and the development of machine learning methods, some researchers have gradually developed more efficient $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation methods [19,22,31], and the study area has changed from highly polluted areas (such as Beijing–Tianjin–Hebei, Yangtze River Delta, and Pearl River Delta) to large regional nationwide areas and long time series.

In this study, we use the attention mechanism to construct spatial proximity feature extraction and then introduce it into the random forest model for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. We also compared it with the performance of the basic random forest model without spatio-temporal features (${\mathrm{R}}^2$ = 0.81, RMSE = 25.51 $\mathsf{\mu }$g/${\mathrm{m}}^3$, MAE = 15.91 $\mathsf{\mu }$g/${\mathrm{m}}^3$). The proposed model which captures the spatio-temporal features, achieves a higher efficiency (${\mathrm{R}}^2$ = 0.89, RMSE = 18.31 $\mathsf{\mu }$g/${\mathrm{m}}^3$, MAE = 11.17 $\mathsf{\mu }$g/${\mathrm{m}}^3$). Although the accuracy index of the proposed method is not the highest among the existing methods, it has definite improvements in spatial and temporal resolution (1 km, hourly). Meanwhile, it is also demonstrated that the spatial proximity features extracted by using the attention mechanism can provide effective supports for the final estimation.

3.2. Feature Correlation and Importance Analysis

For the task of estimating $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations, most machine learning methods are black-box models which do not provide us with much knowledge about the massive data. In other words, they do not tell us how these input variables are connected to the output of the predictions and which factor gains the most attention in the model. In previous studies, they usually used the Pearson correlation coefficient and histograms as descriptive statistics for the correlation analysis [20,53]. Therefore, this paper also conducts a preliminary feature correlation analysis for the problem accordingly.

The frequency distribution histograms of the data with descriptive statistics (minimum, maximum, mean, and standard deviation) are shown in Figure 4. It can be seen that the frequency distribution of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration is more similar to the AOD data, which is in line with the consensus of much research that the AOD data can directly reflect the corresponding spatial and temporal distribution characteristics of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration, and they have a certain degree of similarity in spatial and temporal distribution. In addition, there is a large difference between the Himawari satellite retrieval AOD and the model-assimilation AOD. The value of model-assimilation AOD data is higher than that of the satellite inversion overall, which indicates that the model-assimilation AOD value may be overestimated, but the overall frequency distribution structure has some similarity between these data. Although some previous studies have shown that the relative humidity, air temperature, and wind speed have a strong influence on the spatial and temporal distribution of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations [8], it is difficult to show the intuitive similarity only in their frequency distributions. These results also reflect that the relationship among the spatio-temporal distribution of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations, meteorological factors, and surface environment data is complex and nonlinear. The DEM data are sparsely distributed because the data are obtained by matching the air quality monitoring stations and the DEM values reflecting the geospatial location of the stations to a certain extent.

The frequency distribution of PRE data is very concentrated and near the Y axis; it may have a small effect on the accuracy of $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation in the model and needs to be further confirmed for it to be retained in the subsequent experimental analysis. According to the above analysis, to ensure the efficiency of the $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation model, suitable characteristic variables need to be selected, and the nonlinear relationships between multiple variables need to be adequately fitted and to be experimentally compared to form a definitive model data selection strategy.

The Pearson correlation coefficients between $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration data and other variables are shown in Figure 5. It can be seen that the Pearson coefficient of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration and Himawar satellite AOD (H-AOD) data reaches 0.43, which is the highest among all variables, and this further indicates that the AOD is always the core data for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. In addition, among the variables related to the atmospheric physical properties of $\mathrm{P}{\mathrm{M}}_{2.5}$, the correlation between $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration and boundary layer height and relative humidity is higher (both reach 0.31), while the correlation with wind speed (−0.16, 0.14) and temperature (−0.061) have lower correlations. Usually, $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution is more influenced by wind speed and temperature, etc., during dissipation, and this single intuitive coefficient makes it difficult to reflect the characteristic correlation of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration. Therefore, although the histogram structure of frequency distribution of AOD and $\mathrm{P}{\mathrm{M}}_{2.5}$ data is similar and the Pearson correlation coefficient is high, the accuracy of directly constructing the fitted relationship between AOD and $\mathrm{P}{\mathrm{M}}_{2.5}$ for $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation is limited, and it is necessary to use multivariate synergy to fit the nonlinear relationship.

The random forest model requires a tree node splitting calculation during the construction of the decision tree, such as a splitting function based on the gini coefficient. This approach also reflects the importance of the feature variables on the estimation of the random model. Therefore, this study provides a detailed comparative analysis of the importance of information related to these variables in the random forest, as shown in Figure 6. Overall, among the feature variables, the model considers AOD, BLH, and RH to be the most important for the estimation results, which is also consistent with most existing studies [19,55]. Moreover, this information importance also corresponds in some aspects to Figure 4 and Figure 5. When the model added temporal features, the time stamp (Month, Day, Hour) shows a certain of importance for the estimation results. There is little change in the feature importance ranking, but only the NDVI importance rate is reduced greatly.

3.3. The Impact of AOD data Quality on Model Accuracy

To further compare and analyze the effect of AOD data quality on the estimation of atmospheric $\mathrm{P}{\mathrm{M}}_{2.5}$ by the random forest model, we conducted separate comparison experiments according to the different AOD data sources, as shown in

Table 3. Comparison of experimental results of $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation with different AOD data.

Data	Method	Model Fitting			Model Validation
		${\mathrm{R}}^2$	RMSE	MAE	${\mathrm{R}}^2$	RMSE	MAE
H-AOD	RF	0.972	9.45	5.46	0.812	25.51	15.91
	STAttenRF	0.983	7.56	4.51	0.887	18.31	11.17
M-AOD	RF	0.973	9.94	5.66	0.805	27.86	15.35
	STAttenRF	0.985	7.47	4.34	0.874	20.68	12.87
Mix-AOD	RF	0.968	10.18	5.96	0.793	28.20	17.02
	STAttenRF	0.981	7.63	4.54	0.861	22.43	13.66

. Obviously, the coverage of model assimilation AOD data is higher. In addition, we verified the correlation accuracy (satellite retrieval and model assimilation AOD data vs. the AOD data from ground monitoring sites), as shown in Figure 7. There are 6169 valid data matched between the satellite observed AOD data and AERONET in the study area in 2017, and the correlation between them reached 0.836. There are 6946 valid data matched between the model assimilation AOD data and AERONET, and the correlation between them was only 0.585. It is shown that there is a huge difference in data quality between the two cases. Furthermore, the visual comparison of the scatter plotted in Figure 7 also shows that there is a large overestimation of AOD in the model assimilation AOD. However, when these AOD data are input into the random forest model to estimate the spatial and temporal distributions of $\mathrm{P}{\mathrm{M}}_{2.5}$, Table 3 shows that the difference in estimation results is small, as well as the $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation based on the Himawari satellite retrieval AOD being ${\mathrm{R}}^2$ = 0.812, RMSE = 25.51 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and MAE = 15.91 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and the Himawari model assimilation AOD being ${\mathrm{R}}^2$ = 0.805, RMSE = 27.86 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and MAE = 15.35 $\mathsf{\mu }$g/${\mathrm{m}}^3$. These results are somewhat different from previous studies that pursued the quality of AOD data and used it to obtain high quality $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation. However, by mixing the two types of AOD data and replacing them with model assimilation AOD data when the AOD is missing from the satellite retrievals, the accuracy of the model is reduced to a certain extent. Based on this comparison, this study concluded that there may be errors in the full-coverage $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation by directly using the mixed AOD, so the full-coverage results should be obtained by estimating $\mathrm{P}{\mathrm{M}}_{2.5}$ separately and then performing data fusion. The full-coverage results were obtained by fusing the data after estimating $\mathrm{P}{\mathrm{M}}_{2.5}$ separately. In addition, after using the attention mechanism to extract spatio-temporal features, the model accuracy was significantly improved. As shown in Table 3, the evaluation indicators of $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation by Himawari satellite retrivals AOD were ${\mathrm{R}}^2$ = 0.887, RMSE = 18.31 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and MAE = 11.17 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and the index of $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation by Himawari model assimilation AOD was ${\mathrm{R}}^2$ = 0.874, RMSE = 20.68 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and MAE = 12.87 $\mathsf{\mu }$g/${\mathrm{m}}^3$. These situations further illustrate the importance of spatio-temporal characteristics for the random forest model.

With the comparison of the importance of input variables in the random forest model in Figure 6, we provide a preliminary explanation for this phenomenon. Although the correlation between AOD and $\mathrm{P}{\mathrm{M}}_{2.5}$ is strong, the random forest responds differently to AOD data of different quality when multiple variables are input into it. For example, for the Himawari AOD data in Figure 6, the random forest considered the AOD data to be the most important (more than 20%). While for the model-assimilation AOD data, for its lower data precision (Figure 7), the random forest model adaptively reduced its importance of layer height BLH and relative humidity RH. However, the model is still able to generate better $\mathrm{P}{\mathrm{M}}_{2.5}$ spatial and temporal distribution estimation results in cooperation with several variables in the end. In the estimation results by the mixed AOD data, all three evaluation indicators show some degree of degradation compared with the separate estimation. This may be due to the misleading effect of data quality differences on the same characteristic variables in the random forest model, which is difficult to distinguish in the random forest model.

Based on above analysis, we believe that although the quality of AOD data varies greatly, when multiple variables are input into the random forest model, the random forest will adjust the importance of different-quality AOD data to achieve a certain of robustness (the $\mathrm{P}{\mathrm{M}}_{2.5}$ distribution estimated based on satellite retrievals AOD data or model-assimilation AOD data). The non-linear relationship between the variables can be fitted well, and thus, the spatial and temporal distribution of $\mathrm{P}{\mathrm{M}}_{2.5}$ can be estimated better. In addition, comparing the final results, we can find that the model accuracy is significantly improved after using the attention mechanism to extract spatio-temporal features.

3.4. Spatio-temporal Validation and Analysis

The spatial and temporal distribution of seasonal $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration is calculated from the average of hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ data in the corresponding time interval, as shown in Figure 8. Among them, the point source distribution markers in the figure are the seasonal averages of the ground monitoring stations, which range from March to May each year in spring, June to August each year in summer, September to November each year in autumn, and December to February each year in winter. It can be seen that the seasonal distribution of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations in the Beijing–Tianjin–Hebei region varies significantly. Based on the statistical data of $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution concentrations at ground stations, it can be concluded that the pollution level is higher in winter (∼70.12 ± 85.26 $\mathsf{\mu }$g/${\mathrm{m}}^3$), the average values in spring (∼47.61 ± 46.09 $\mathsf{\mu }$g/${\mathrm{m}}^3$) and autumn (∼46.94 ± 42.92 $\mathsf{\mu }$g/${\mathrm{m}}^3$) are close to each other, and the lowest pollution level is in summer (∼42.15 ± 25.23 $\mathsf{\mu }$g/${\mathrm{m}}^3$). The seasonal $\mathrm{P}{\mathrm{M}}_{2.5}$ spatial distribution data were obtained statistically after model estimation in this section, and the $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution level in the northwest region of Beijing, Tianjin, and Hebei is lower. The $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution level in the southeast region is higher, which shows a trend of gradually severe pollution from the northwest to southeast. This spatial pattern distribution is extremely similar to the digital elevation map of the Beijing–Tianjin–Hebei region shown in Figure 1. With higher terrain in the northwest mountainous region and lower terrain in the southeast open plain, it shows a sloping trend from the northwest to southeast. It can be seen that $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution is mainly distributed in the urban built-up areas, while decreasing spatially toward the suburbs. The main reasons are: the northwest region is located in the mountainous area, the population and urban built-up areas are sparsely distributed, and there are few sources of pollution emissions caused by industrial agriculture and human activities. In contrast, Beijing–Tianjin–Hebei is one of the core economic regions in China and also one of the heavy industrial regions in China. With the air pollution control in recent years, the spatial pattern of some major iron and steel, coal, electricity, and light industries in the Beijing-ring region has changed dramatically. Most of the industrial sites have shifted to the southern part of Hebei province, thus leading to poorer air quality in the southeast, which is the area of heavy industries. Although the pollution sources around Beijing are reduced, topographically, the pollution from the southeast heavy industry region still spreads to the Beijing area and gathers and dissipates in the region. Beijing is still a population gathering area, and it is difficult to reduce the traffic as well as living pollutant emissions, especially the large amount of pollution emissions caused by coal burning and fossil-fuel heating in the winter, etc. The local topography and terrain and even autumn and winter meteorological conditions also contribute to the gathering of pollution. It is not conducive and also exists in the Yangtze River Delta and Sichuan Basin regions [56,57].

Comparing the spatial distribution values with the monitoring point values, the overall $\mathrm{P}{\mathrm{M}}_{2.5}$ spatial distribution accuracy is similar to that of the corresponding ground monitoring points. Among them, the estimated values of Beijing and Tianjin are relatively more accurate, which may be attributed to the relatively large number of air quality monitoring stations and more uniform spatial distribution in these two cities. On the contrary, the ground stations in Hebei province are sparsely distributed in each city and the number of stations is small, so the visual comparison is slightly underestimated. In addition, a significant overestimation of $\mathrm{P}{\mathrm{M}}_{2.5}$ occurs along the Bohai Sea coast, which may be due to the difference in aerosol types along the seashore, model training data obtained from ground monitoring sites that are in the land area, and meteorological conditions such as sea fog and sea wind. This also indicates that the data-driven model relies on the complete data set with high accuracy.

This study cross-validated the hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ estimates with ground monitoring values for each season in 2017, and the relevant scatter plots are shown in Figure 9. It can be seen that although the highest RMSE (11.841 $\mathsf{\mu }$g/${\mathrm{m}}^3$) and MAE (7.909 $\mathsf{\mu }$g/${\mathrm{m}}^3$) indicators for $\mathrm{P}{\mathrm{M}}_{2.5}$ were found in summer, the ${\mathrm{R}}^2$ was only 0.783. On the contrary, the lowest RMSE (24.937 $\mathsf{\mu }$g/${\mathrm{m}}^3$) and MAE (13.611 $\mathsf{\mu }$g/${\mathrm{m}}^3$) indicators were found in winter, but the ${\mathrm{R}}^2$ reached 0.912. The scatter plot also shows the lowest $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration values in summer (about 0–150 $\mathsf{\mu }$g/${\mathrm{m}}^3$) and the highest $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration values in winter (about 0–500 $\mathsf{\mu }$g/${\mathrm{m}}^3$), and therefore, the estimated $\mathrm{P}{\mathrm{M}}_{2.5}$ in winter has difficulty reflecting the advantage in RMSE and MAE indexes, and the measurement of model performance should be compared comprehensively. In addition, the indicators are closer in spring (${\mathrm{R}}^2$ = 0.852, RMSE = 17.976 $\mathsf{\mu }$g/${\mathrm{m}}^3$, MAE = 9.502 $\mathsf{\mu }$g/${\mathrm{m}}^3$) and autumn (${\mathrm{R}}^2$ = 0.879, RMSE = 15.105 $\mathsf{\mu }$g/${\mathrm{m}}^3$, MAE = 9.243 $\mathsf{\mu }$g/${\mathrm{m}}^3$). The differences in model performance for different seasons of $\mathrm{P}{\mathrm{M}}_{2.5}$ may have some correlation with the sample size and seasonal natural conditions. For example, on the one hand, in BTH region, there is more snow and ice cover in winter and cloudy and rainy weather in summer. Those differences have a greater impact on the number of effective samples, thus the samples in winter and summer will be significantly less than that in spring. On the other hand, summer weather conditions are conducive to the rapid dissipation of pollution, making it difficult to form aggregated pollution. The autumn and winter seasons are closely related to the increase in pollution emissions, and the Beijing–Tianjin–Hebei region is more influenced by the northwest dust in spring, which also makes the nonlinear relationship between AOD and $\mathrm{P}{\mathrm{M}}_{2.5}$ more complicated. It is difficult for the model to completely fit such complex relationships without multi-source data and the attention mechanism.

Table 4. Cross-validation of estimated and measured average $\mathrm{P}{\mathrm{M}}_{2.5}$ at different hours.

Time	Samples	${\mathrm{R}}^2$	RMSE	MAE	Slope	Estimated	Measured
08:00	3099	0.817	16.32	10.52	0.72	50.2 ± 27.8	49.3 ± 34.7
09:00	4739	0.820	17.13	11.24	0.76	52.3 ± 32.9	49.8 ± 39.2
10:00	6931	0.855	22.19	12.11	0.78	57.4 ± 50.2	55.6 ± 59.4
11:00	7244	0.881	20.61	11.81	0.83	55.1 ± 51.7	53.2 ± 58.6
12:00	7188	0.884	19.46	10.82	0.85	51.3 ± 51.1	49.8 ± 56.6
13:00	6953	0.902	18.01	9.87	0.86	50.0 ± 51.7	49.7 ± 57.0
14:00	6848	0.891	19.02	10.54	0.85	49.4 ± 51.0	50.6 ± 56.9
15:00	6550	0.903	18.84	11.01	0.84	49.1 ± 52.2	52.2 ± 58.7
16:00	4500	0.878	18.76	10.83	0.80	44.6 ± 44.6	49.3 ± 51.2
17:00	2814	0.745	16.49	10.68	0.69	36.3 ± 22.3	41.9 ± 30.2
ALL	56,866	0.873	18.60	11.92	0.83	50.7 ± 48.1	50.9 ± 54.6

presents statistics on the detailed model-estimated average $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations at different hours in 2017 compared with the corresponding ground-based measurements for the validation results. It can be seen that the proposed method in this study has a slightly different performance in each hour, but the overall main evaluation indexes do not differ much. The main reason for this may be due to the reduced number of training samples from optical remote sensing satellites at sunrise (8:00 vs. 9:00) and sunset (16:00 vs. 17:00).

In addition, there are significant daily variations in air pollution at different $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution levels due to the difference in human activity intensity and natural conditions at each time period. The slope metric shows that the model has a certain degree of underestimation, which is a common problem in $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation studies, and some scholars believe that this is due to the large uncertainty in the aerosol retrievals and the small data samples under high pollution conditions [8,22,56].

Figure 10 illustrates the spatial and temporal distribution of hourly average $\mathrm{P}{\mathrm{M}}_{2.5}$ concentrations in 2017, where the point source distribution data are ground monitoring values. As can be observed in the figure, the spatial distribution of pollution across the study area is characterized by the same seasonal scale, with low $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution in the northwest region and high pollution in the southeast. From the annual average spatial distribution of $\mathrm{P}{\mathrm{M}}_{2.5}$ at different hours, the trend of $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution shows the characteristics of rising first and then falling from morning to evening. The pollution reaches a peak around 11:00 a.m., and $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution drops to a relatively low level in the evening, which may be caused by the increase in pollution emission in the morning and the superposition effect of pollution gathering from the previous night, etc. This kind of $\mathrm{P}{\mathrm{M}}_{2.5}$ pollution spatial and temporal distribution characteristic is also similar to some existing studies [53], especially that the increase in pollution emissions caused by anthropogenic activities, industrial production, etc., during the corresponding time of autumn and winter seasons are more relevant.

4. Conclusions

In this paper, we improved the estimation of $\mathrm{P}{\mathrm{M}}_{2.5}$ concentration across Beijing–Tianjing–Hebei regions using multi-source data by introducing the attention mechanism into random forest. The attention mechanism was used to co-weight the environmental similarity and the geographical distances, and the RF algorithm was applied to estimate hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ of BTH based on the Himawari-8/AHI AOD products. The experiment results demonstrated that our approach can more efficiently characterize the influence of spatio-temporal information hidden in adjacent ground monitoring sites. The hourly $\mathrm{P}{\mathrm{M}}_{2.5}$ estimates are well correlated with ground measurements in BTH, with an ${\mathrm{R}}^2$ of 0.887, an RMSE of 18.31 $\mathsf{\mu }$g/${\mathrm{m}}^3$, and an MAE of 11.17 $\mathsf{\mu }$g/${\mathrm{m}}^3$, indicating the good model performance. Furthermore, to simplify the model structure and improve the efficiency while maintaining accuracy, this paper thoroughly examines of the effectiveness of multi-source data in the estimation process, including the analysis of the effect of AOD data quality on the final estimation accuracy of the model and the ranking results of the feature importance of the random forest. Finally, this approach provides full-coverage $\mathrm{P}{\mathrm{M}}_{2.5}$ estimation in BTH by fusing satellite and model data.

In future work, we will pay attention to computation efficiency and model robustness, then expand the estimation area with reasonable spatial–temporal analysis and validation.