A Higher-Order Graph Convolutional Network for Location Recommendation of an Air-Quality-Monitoring Station

Abstract

The location recommendation of an air-quality-monitoring station is a prerequisite for inferring the air-quality distribution in urban areas. How to use a limited number of monitoring equipment to accurately infer air quality depends on the location of the monitoring equipment. In this paper, our main objective was how to recommend optimal monitoring-station locations based on existing ones to maximize the accuracy of a air-quality inference model for inferring the air-quality distribution of an entire urban area. This task is challenging for the following main reasons: (1) air-quality distribution has spatiotemporal interactions and is affected by many complex external influential factors, such as weather and points of interest (POIs), and (2) how to effectively correlate the air-quality inference model with the monitoring station location recommendation model so that the recommended station can maximize the accuracy of the air-quality inference model. To solve the aforementioned challenges, we formulate the monitoring station location as an urban spatiotemporal graph (USTG) node recommendation problem in which each node represents a region with time-varying air-quality values. We design an effective air-quality inference model-based proposed high-order graph convolution (HGCNInf) that could capture the spatiotemporal interaction of air-quality distribution and could extract external influential factor features. Furthermore, HGCNInf can learn the correlation degree between the nodes in USTG that reflects the spatiotemporal changes in air quality. Based on the correlation degree, we design a greedy algorithm for minimizing information entropy (GMIE) that aims to mark the recommendation priority of unlabeled nodes according to the ability to improve the inference accuracy of HGCNInf through the node incremental learning method. Finally, we recommend the node with the highest priority as the new monitoring station location, which could bring about the greatest accuracy improvement to HGCNInf.

1. Introduction

In recent years, with economic growth, environmental problems have become increasingly prominent and air pollution is receiving unprecedented attention [1,2,3,4]. The air-quality index (AQI) provides a number used by government agencies to communicate to the public how polluted the air is currently. As the AQI increases, an increasingly large percentage of the population is likely to experience increasingly severe adverse health effects [5,6,7]. To compute this AQI, an air-pollutant concentration from a monitor or model is required, such as carbon monoxide (CO), carbon dioxide (CO${}_2$), hydrocarbons (HC), nitrogen oxides (NO${}_x$), solid particulate matter (PM${}_{2.5}$ and PM${}_{10}$), etc. In order to reflect the air quality and its development trends in a timely and accurate manner, we need accurate air-quality-monitoring equipment.

However, it is unrealistic to establish a large number of monitoring equipment in different areas of the city due to the expensive construction and later maintenance costs [8]. Based on the premise of a limited number of monitoring equipment, a reasonable monitoring station location layout is the first basic link to accurately infer air quality. On the other hand, with the rapid development of urbanization and the continuous growth in the number of motor vehicles, the number of established monitoring equipment is too few to accurately reflect the air-quality distribution in an entire urban area.

Thus far, there is few research on the optimization methods of regional air-quality-monitoring networks and there is also no uniform standard for recommending optimal monitoring equipment locations. Therefore, we need to design a framework that could recommend optimal station locations to establish new monitoring equipment on the basis of the existing ones, which can bring about the greatest accuracy improvement to the inference model. In this way, it not only meets the accuracy requirements for air-quality inference but also saves economic costs.

In recent years, the research on the location selection is mainly divided into the following two categories: knowledge-driven approaches and data-driven approaches.

The knowledge-driven approaches use mathematical models [9,10] and physics knowledge [11] to solve the location recommendation problem through computational simulation. In addition, the United States, Europe, and Japan have also established regional air-quality-monitoring networks around photochemical smog pollution and aerosol pollution [12,13,14]. These methods all adopted a series of optimization techniques to determine the layout of network sites, including statistical analysis methods such as correlation analysis and cluster analysis [15,16] or mathematical methods such as multi-objective optimization [17,18]. In order to reach a stable state, the simulation process not only requires complex system programming but also consumes a lot of computing power. Simplifications and stationarity assumptions that may be unrealistic in modeling further degrade the model efficiency.

Recently, data-driven approaches for location recommendation have been developed based on fixed monitoring stations [19] or Taxi GPS trajectory [20,21] but without considering the road network spatial structure, such as road segment length, road types (highways, main roads, and streets), POI density, etc. Kang et al. [22] transformed the station recommendation into a graph problem, which is committed to covering the urban area with the least number of monitoring stations but without considering the influences of complex external factors, which might lead to geographically non-smooth values in air-quality distribution. Hsieh et al. [23] studied label propagation [24] on graphs and considered external influential factors but failed to capture the spatiotemporal correlation between nodes in graph.

In this paper, we want to solve a practical problem: how to recommend optimal station locations to establish air-quality-monitoring equipment based on existing ones to maximize the accuracy of the air-quality-inference model, so as to reflect the air-quality distribution and its development trend in a timely manner and accurately. This task is challenging for the following reasons:

(1) The air-quality distribution is affected by many complex external influential factors (such as weather, traffic volume, land use, etc). For commercial centers with heavy traffic flow, the air quality is often worse than that in parks or lakes. There will be geographical non-smooth data due to the influence of external influential factors, and it is difficult to obtain accurate air-quality distribution in unobserved areas through interpolation-based methods. For example, Figure 1 is a real-time record of a certain day at the Beijing Air-Quality-Monitoring Station. We can find that the PM${}_{10}$ concentration data are not smooth. The monitoring stations in the red circle are very geographically close, but the PM${}_{10}$ concentration varies greatly throughout the year. The most likely reason for this result is that the monitoring stations with low PM${}_{10}$ concentration are close to parks or lakes and that the stations with high PM${}_{10}$ concentration are located near commercial centers or main roads with heavy traffic.

(2) There is spatiotemporal interaction between different nodes in the urban spatiotemporal graph: At the same time, the air-quality value of a node is affected by other nodes; in the same space, the air-quality values of the same node at different times are correlated.

(3) Complex correlation between the monitoring station location recommendation model and the air-quality inference model. The task of the monitoring station location recommendation model is to select optimal locations to establish new monitoring stations to maximize the accuracy of an air-quality-inference model for inferring the air-quality distribution of an entire urban area. There is a close relationship between two models.

Aiming to solve the main research problems of this paper, the current research methods either fail to consider the urban spatial structure or the influence of external factors on air-quality distribution; the first problem is how to accurately infer air-quality distribution. However, the performance of an air-quality-inference model has a crucial impact on the subsequent monitoring-station location recommendation. The next most direct question is how to associate the monitoring-station location recommendation model with the air-quality-inference model to recommend optimal station locations. The existing methods cannot capture the correlation between nodes because of the performance limitation of the model, so the recommended station location is not optimal.

In order to solve the aforementioned challenges, we formulate monitoring station locations as an urban spatiotemporal graph (USTG) node recommendation problem in which each node represents a region with time-varying air-quality values and propose a two-step learning framework.

The main contributions of this research are as follows:

(1) We propose a variant of graph convolutional network (GCN) called higher-order graph convolutional network (HGCN). The ordinary GCN can capture the one-hop neighbor node’s spatiotemporal correlation. We improve the ordinary graph convolution network and design a high-order graph convolution network. Compared with an ordinary (one-hop) GCN, HGCN has a larger receptive field, which could capture more and higher-order neighbor node information.

(2) We designed an accurate air-quality-inference model (HGCNInf) based on the proposed HGCN to infer urban air-quality distribution. We applied the graph convolutional network to the field of air-quality monitoring for the first time. By modeling the urban area as an USTG, the HGCNInf can accurately infer the air quality of an entire urban area: the HGCN can effectively capture the spatiotemporal interactions of the air-quality distribution, and a fully connected neural network is used to extract the external influential factor features.

(3) We analyze and correlate the air-quality-inference model with the monitoring-station location recommendation model. By using the convolution weight parameters of HGCNInf, we design a GMIE based on the correlation degree between nodes in USTG reflecting the air-quality spatiotemporal changes, marking the recommendation priority of unlabeled nodes according to the ability to improve the inference accuracy of HGCNInf iteratively through the node incremental learning method. The recommended optimal node could bring about the greatest accuracy improvement to HGCNInf.

(4) We evaluate our model using Beijing air-quality data from 1 January 2015 to 31 December 2015, and the experimental results show that our approach far outperforms the state-of-the-art baseline methods.

2. Study Area and Data Requirement

2.1. Study Area

Beijing is China’s capital city (Figure 2), located at 40.250583${}^{\circ }$N and 116.462611${}^{\circ }$E, on the northwestern edge of the North China Plain. The average altitude of Beijing is 43.5 m above sea level. Beijing is located in a warm temperate and semi-humid area. The climate is a warm temperate and characterized by semi-humid continental monsoons. The average annual precipitation in the plain area is about 600 mm, and the annual average temperature is about 12.9 ${}^{\circ }$C.

The main sources of air pollution in Beijing are industrial exhaust gas and automobile exhaust emissions. Our study area covers the Haidian, Shijingshan, Fengtai, Xicheng, and Chaoyang districts, which are the main urban districts in Beijing. The air-quality data collected from the above areas can reflect the overall air-quality condition in Beijing.

2.2. Data Requirement

We evaluated our proposed model with Beijing air-quality data, which comes from the Beijing Municipal Ecological and Environment Monitoring Center. Air-quality data are collected every hour, and the time span is from 1 January 2015 to 31 December 2015. The data are from 17 monitoring stations. and a total of 17 × 8760 data points were collected. In the experiment, we divided the main urban area of Beijing into 30 × 38 sub-regions (1 km × 1 km), in which 17 sub-regions have established monitoring stations, while the air-quality distribution of the remaining 1123 sub-regions are unknown. The external factors include urban meteorological data, the number of points of interest (POIs) in the sub-region, and the geographic road network information in the sub-region (such as the length of highways and main roads, the number of intersections, etc.) [25]. These data features are represented as $F=\left\{f_1,f_2,\cdots ,f_m\right\}$, where $f_m$ represents the mth data feature. The types of POI are shown in

Table 1. The types of POIs.

1.	Parks	6.	Companies
2.	Schools	7.	Hotels
3.	Stadium	8.	Supermarkets
4.	Restaurants	9.	Vehicle repair stations
5.	Business center	10.	Gas stations

.

Based on urban geographic information, the urban area is divided into disjoint grids. Each grid is a sub-area of 1 km × 1 km and is associated with time-varying air quality values. As shown in Figure 3.

3. Preliminary and Methodology

3.1. Preliminary

3.1.1. Air-Quality Index

The air-quality index (AQI) is a number used by government agencies to communicate to the public how polluted the air is currently. As the AQI increases, an increasingly large percentage of the population is likely to experience increasingly severe adverse health effects. To compute the AQI, an air-pollutant concentration from a monitor or model is required. The function used to convert the air pollutant concentration to AQI varies by pollutants and is different in different countries. The air-quality index values were divided into ranges, and each range was assigned a descriptor and a color code.

3.1.2. Graph Convolutional Networks

Convolutional Neural Networks (CNNs) are mainly effective on Euclidean domain data with regular spatial structures. Generalizing convolutional neural networks on arbitrary graph-structured data is a hot topic in recent researches. Bruna et al. [26] generalized the convolution operator to a general graph via the graph-Laplacian spectrum. By transforming both the filter and the signal to the Fourier domain, convolutions on the graphs can be calculated by multiplying the filter and the signal and then by transforming the result back into the discrete domain. As a meaningful translation operator cannot be expressed in the node domain [26], Defferrard et al. [27] defined the convolution operator $\ast \mathcal{G}$ on graph in the Fourier domain. A signal x filtered by $g_w$ = $diag\left(w\right)$ is as follows:

$$\begin{array}{c}\hfill g_w\ast \mathcal{G}x=g_w\left(L\right)x=g_w\left(U\mathsf{\Lambda }{U}^{\tau }\right)x=U{g}_w\left(\mathsf{\Lambda }\right)U^{\tau }x.\end{array}$$

(1)

where U is a matrix of eigenvectors of the normalized graph Laplace matrix L and $\mathsf{\Lambda }$ is a diagonal matrix of eigenvalues of L.

However, evaluating Equation (1) is computationally expensive, as its computational complexity is $\mathcal{O}\left(N^2\right)$. To circumvent this problem, the filter $g_w\left(\mathsf{\Lambda }\right)$ is well approximated by a truncated expansion of K-order Chebyshev polynomials suggested in [28]. Based on [28], Defferrard et al. [27] designed fast localized convolutional filters on graphs to reduce the computational complexity.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill g_w\ast \mathcal{G}x=g_w\left(L\right)x& \approx \mathrm{U}\sum _{k=0}^{K-1}{W}_k^{\prime }{T}_k\left(\tilde{A}\right)U^T\mathrm{x}\hfill \\ & =\sum _{k=0}^{K-1}{W}_k^{\prime }{T}_k\left(\tilde{L}\right)\mathrm{x}\hfill \end{array}\end{array}$$

(2)

where $T_k\left(\tilde{L}\right)$ is a K-order polynomial about $\tilde{L}$, $\tilde{L}=\frac{2}{{\lambda }_{max}}L-I_N,and{\lambda }_{max}$ is the largest eigenvalue of the normalized graph Laplacian matrix L. The computational complexity is reduced from $\mathcal{O}\left(N^2\right)$ in Equation (2) to $\mathcal{O}\left(N\right)$.

Furthermore, Kipf et al. [29] designed a multi-layer graph convolution network for semi-supervised learning via a first-order approximation of localized spectral filters on graphs. The definition is as follows:

$$\begin{array}{c}\hfill Z={\tilde{D}}^{-\frac{1}{2}}\tilde{A}{\tilde{D}}^{-\frac{1}{2}}XW\end{array}$$

(3)

where Z is the convolved signal matrix, $\tilde{A}=A+I_N$ is the graph adjacency matrix with added self-connections, ${\tilde{D}}_{ii}={\mathsf{\Sigma }}_j{\tilde{A}}_{ij}$, and W is a trainable weight matrix. The computational complexity of this convolution operation is the same as $\tilde{A}X$.

As a typical type of graph, based on the spectral graph convolutions in [29], Yu et al. [30] designed a spatiotemporal graph convolutional network for traffic forecasting. Xu et al. [31] designed a spatiotemporal graph multi-fusion network for urban mobile source emission prediction. Sun et al. [32] designed multi-view graph convolutional networks to predict citywide crowd flows in irregular regions. Especially, Cui et al. [33] designed a traffic higher-order graph convolutional recurrent neural network for network-scale traffic learning and forecasting.

3.1.3. Urban Spatiotemporal Graph

We take each grid element as a node and to construct an urban spatiotemporal graph $G=(V;\epsilon ;A)$(USTG), where V is the set of nodes associated with the time-varying air-quality value; $\left|V\right|=N$ denotes the number of grid nodes; $\epsilon$ is the set of edges, indicating the connectivity between the nodes; and $A\in R^{N\times N}$ is the graph adjacency matrix. In particular, each vertex $v_i\in V$ in the USTG is a three-dimensional representation; the first two dimensions represent the geographic location information of the node, and the third dimension represents the air-quality value that changes with time. In USTG, V can be represented as $V=L\cup U$, where L represents the node set with air-quality-monitoring stations and U represents the node set without monitoring stations. We take the nodes in L as labeled nodes and the nodes in U as unlabeled nodes. Generally speaking, $\left|L\right| \ll \left|U\right|$. The spatial correlation of the spatiotemporal graph is reflected by the geographical distance characteristics between the nodes, and the temporal correlation is reflected by the air-quality value of different times associated with the nodes.

3.1.4. Inference Problem of Air Quality

Problem 1.

At time $\left\{T_i\mid i=1,2,\dots ,n\right\}$, based on USTG $G=(V;\epsilon ;A)$, the air-quality distribution $f_l\in R^{\left|L\right|\times P}$ of the labeled nodes L is used to infer the air-pollution distribution $f_u^{\ast }\in R^{\left|U\right|\times P}$ of the unlabeled nodes U. The task of air-quality inference is to learn a function $F(·)$ that maps $f_l\in R^{\left|L\right|\times P}$ to $f_u^{*}\in R^{\left|U\right|\times P}$:

$$\begin{array}{c}\hfill F\left(f_l\left(T_i\right),\mathrm{G}(V;\epsilon ;A)\right)=f_u^{*}\left(T_i\right).\end{array}$$

(4)

where $\left|L\right|$ is the number of labeled nodes, $\left|U\right|$ is the number of unlabeled nodes, and P is the number of features associated with each node after one-hot encoding processing.

3.1.5. Location Recommendation for the Monitoring Station

Problem 2.

The main objective of monitoring equipment site location recommendation is to choose optimal practical locations to establish new monitoring equipment at candidate locations. The optimal location can bring about the greatest accuracy improvement to the inference model. There is a relatively close relationship between the monitoring station recommendation model and the air-quality-inference model. We need to analyze and correlate these two models and to design an effective algorithm based on a correlation degree to complete the monitoring-station location recommendation.

3.2. Methodology

In this section, we introduce the air-quality-inference model and the monitoring-station location recommendation model. We first introduce the conception of higher-order graph convolutional networks. Then, we design an air-quality-inference model based on the improved HGCN, which is applied effectively to the field of atmospheric environment monitoring. Next, we analyze the air-quality inference model and correlate it with the air-quality-monitoring station location recommendation. We design a greedy algorithm for minimizing information entropy based on the correlation degree between nodes in USTG, marking the recommendation priorities of unlabeled nodes according to the ability to improve the inference model accuracies using the incremental learning method to complete the monitoring-equipment station location recommendation.

The proposed two-step air-quality-inference learning framework is shown in Figure 4.

(1) Air-quality inference model. We propose an effective air-quality inference model based on the proposed higher-order graph convolution network (HGCNInf) to infer the air-quality distribution, consisting of a higher-order graph convolution network (HGCN) and several fully connected neural networks. By inputing historical monitoring data and by fetching global information, the HGCN is used to capture the spatiotemporal interactions of air-quality distribution and the fully connected neural network is used to extract external influential factor features, so that HGCNInf can accurately infer the air-quality distribution of an entire urban area.

(2) Monitoring-station location recommendation model. HGCNInf takes the spatiotemporal interactions of pollutant distribution and external influential factors into account, in which the model convolution weight parameters actually represent the correlation degree between the nodes in USTG that reflect the spatiotemporal changes in air quality. The high correlation between a node and other nodes indicates that the air quality of the node is easier to infer. Therefore, for the station location recommendation, it should be chosen in such a node that has a low correlation with other nodes. In this paper, we introduce the information entropy index to evaluate the correlation degree between nodes. The node with the lower information entropy indicates that it has a higher correlation degree with the remaining nodes. Next, we designed a greedy algorithm for minimizing information entropy (GMIE) based on the correlation degree between nodes in USTG, marking the recommendation priorities of unlabeled nodes according to the ability to improve the inference accuracies of HGCNInf through the node incremental learning method to complete the monitoring station location recommendation.

3.2.1. Higher-Order Graph Convolutional Network

Some recent researches [34,35,36] have shown that, for graph node semi-supervised learning, random walk statistics can be used to effectively learn feature representations because it could preserve the completeness of the graph structure. Under special conditions, if the activation function of the first layer in the graph convolutional network is an identity function, then the multi-layer GCN model can also learn random walks. For the 2-layer GCN model proposed in [29], we assume that the activation function of the first layer is an identity function:

$$\begin{array}{c}\hfill \sigma \left(\widehat{A}X{W}^{\left(0\right)}\right)=\widehat{A}X{W}^{\left(0\right)}\end{array}$$

(5)

In this way, the GCN model can be further simplified to the following form:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill Z=f(X,A)& =softmax\left(\widehat{A}\left(\widehat{A}X{W}^{\left(0\right)}\right)W^{\left(1\right)}\right)\hfill \\ & =softmax\left({\widehat{A}}^{2}X{W}^{\left(0\right)}{W}^{\left(1\right)}\right)\hfill \\ & =softmax\left({\widehat{A}}^{2}X{W}^{\left(2\right)}\right)\hfill \end{array}\end{array}$$

(6)

where $W^{\left(2\right)}=W^{\left(0\right)}{W}^{\left(1\right)}$ is expressed as the weight parameter of ${\widehat{A}}^2$, $\widehat{A}={\tilde{D}}^{-\frac{1}{2}}\tilde{A}{\tilde{D}}^{-\frac{1}{2}}$. The two-layer one-hop graph convolutional network is converted into a one-layer two-hop convolutional network. Furthermore, ${\widehat{A}}^2$ can be expressed as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\widehat{A}}^2& =\left({\tilde{D}}^{-\frac{1}{2}}\tilde{A}{\tilde{D}}^{-\frac{1}{2}}\right)\left({\tilde{D}}^{-\frac{1}{2}}\tilde{A}{\tilde{D}}^{-\frac{1}{2}}\right)\hfill \\ & ={\tilde{D}}^{-\frac{1}{2}}\tilde{A}\left({\tilde{D}}^{-1}\tilde{A}\right){\tilde{D}}^{-\frac{1}{2}}\hfill \\ & \propto {\tilde{D}}^{-1}\tilde{A}\hfill \end{array}\end{array}$$

(7)

We can find that the 2-hop GCN network under certain conditions actually learns a one-step random walk, which can transfer the information of the node to the higher-order neighbor nodes. For the 2-layer GCN model proposed in [29], the flow of the input signal through the graph edges is reduced because the information features have gone through matrix multiplication (average neighbor node information) and a nonlinear activation function $\sigma (·)$. Therefore, it is not possible to directly take higher-order operations on $\widehat{A}$ based on the original multi-layer GCN network; otherwise, it causes network performance degradation. In order to make better use of the information of high-hop neighbor nodes, we increase the order of the graph convolution network while reducing the layer numbers of the network and then fuse the features extracted from each order network.

$$\begin{array}{c}\hfill {\widehat{A}}^k={\tilde{D}}^{-\frac{1}{2}}\tilde{A}\left[{\left({\tilde{D}}^{-1}\tilde{A}\right)}^{(k-1)}\right]{\tilde{D}}^{-\frac{1}{2}}\end{array}$$

(8)

$$\begin{array}{c}\hfill \begin{array}{c}G{C}^k=GCN\left({\widehat{A}}^k,X\right)={\widehat{A}}^{k}X{W}^{\left(k\right)}\end{array}\end{array}$$

(9)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill Z=f(X,A)& =G{C}^{\left\{k\right\}}\hfill \\ & =softmax\left(\left[G{C}^1,G{C}^2,\dots ,G{C}^k\right]\right)\hfill \end{array}\end{array}$$

(10)

Thus, the newly designed higher-order graph convolutional network uses fewer layers and parameters but can achieve better results.

3.2.2. The Air-Quality Graph Convolution

We first define the graph higher-order adjacency matrix.The one-hop neighbor matrix of the USTG is the adjacency matrix A, and the k-hop adjacency matrix can be obtained by the kth power of A. As shown in Figure 5, the multi-hop neighbor with respect to a node (red star) is the nodes that can be reached by the black edges in the graph.

We apply the proposed higher-order graph convolutional network to the field of air-quality monitoring. The problem of air-quality inference is specifically described in Section 3.1.4 and the proposed HGCNInf framework is shown in Figure 6.

In order to make the subsequent air-quality monitoring station location recommendation model more reasonable and interpretable, we modified the graph convolution network $G{C}^k$ described in Equation (9) to the following form.

$$\begin{array}{c}\hfill G{C}^k=GCN\left({\widehat{A}}^k,X\right)={\widehat{A}}^k{W}^{\left(k\right)}X\end{array}$$

(11)

where$\tilde{A}=D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}\in R^{N\times N}$, D is the degree matrix of A, and $W^{\left(k\right)}$ is the convolution weight parameter. The graph input signal $\mathrm{X}=\left[\begin{array}{c}{f}_l\hfill \\ f_u\hfill \end{array}\right]\in R^{N\times P}$, where $f_l\in R^{\lfloor L\mid \times P}$. is the air-quality distribution of the labeled nodes, initializing the air-quality distribution of unlabeled nodes to $f_u=\left[\begin{array}{ccc}0& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & 0\end{array}\right]\in R^{\left|U\right|\times P}$.

Then, we concatenated the K-hop features extracted from the graph signal, defined as follows:

$$\begin{array}{c}\hfill G{C}^{\left\{k\right\}}=\left[G{C}^1,G{C}^2,\dots ,G{C}^k\right]\end{array}$$

(12)

where $G{C}^{\left\{k\right\}}\in R^{N\times (K\times P)}$ is a set of K-hop features extracted by graph convolutional network. We fed $G{C}^{\left\{k\right\}}$ into a fully connected network $f_{fc1}$ to jointly train

$$\begin{array}{c}\hfill f_{fc1}:R^{N\times (K\times P)}\to R^{N\times P}\end{array}$$

(13)

$$\begin{array}{c}\hfill GC=f_{fc1}\left(G{C}^{\left\{k\right\}},W_{fc1}\right)\end{array}$$

(14)

where air-distribution feature $GC\in R^{N\times P}$ and the trainable parameter matrix $W_{fc1}\in R^{P\times (K\times P)}$.

We use the fully connected network $f_{fc2}$ to extract external factor features, defined as follows:

$$\begin{array}{c}\hfill f_{fc2}:R^{N\times M}\to R^{N\times P}\end{array}$$

(15)

$$\begin{array}{c}\hfill G{C}_f=f_{fc2}\left(f_m,W_{fc2}\right)\end{array}$$

(16)

where the trainable parameter matrix $W_{fc2}\in R^{P\times M}$; external factor features $G{C}_f\in R^{N\times P}$; and $f_m\in R^{N\times M}$, which is the M external factors.

We concatenated the air-quality distribution feature $GC\in R^{N\times P}$ with the external factor feature $G{C}_f\in R^{N\times P}$ and then fed them into the fully connected network $f_{fc3}$, defined as follows:

$$\begin{array}{c}\hfill f_{fc3}:R^{N\times P}\to R^{N\times P}\end{array}$$

(17)

$$\begin{array}{c}\hfill {\mathrm{GC}}_{all}=f_{fc3}\left(\left(\mathrm{GC}+G{C}_f\right),W_{fc3}\right)\end{array}$$

(18)

where the trainable parameter matrix $W_{fc3}\in R^{P\times P}$ and general feature $G{C}_{all}\in R^{N\times P}$.

Further, the air quality distribution $F_u\in R^{\left|U\right|\times P}$ of unlabeled nodes is obtained, defined as follows:

$$\begin{array}{c}\hfill F=softmax\left({\mathrm{GC}}_{all}\right)\end{array}$$

(19)

$$\begin{array}{c}\hfill F=\left[\begin{array}{c}{F}_l\hfill \\ F_u\hfill \end{array}\right]\in R^{N\times P}\end{array}$$

(20)

where $F_l\in R^{\left|L\right|\times P}$ is the air-quality distribution output of the labeled nodes and $F_u\in R^{\left|U\right|\times P}$ is the air-quality distribution output of the unlabeled node.

We minimized the cross-entropy loss of the output $F_l$ and the known training label Y. The cross-entropy loss is defined as follows:

$$\begin{array}{c}\hfill H\left(Y,F_l\right)=-diag\left(V_l\right)\sum \left[Y\ast log\left(F_l\right)\right]\end{array}$$

(21)

where ∗ represents the Hadamard product and where $diag\left(V_l\right)$ is a diagonal matrix where elements are set to 1 when $(\mathrm{i},\mathrm{i})\in V_l$ and 0 otherwise. In order to make the graph convolution features more stable and interpretable, we added regularization terms to the loss function.

(1) We added the L1-norm of the graph convolution weight matrix to the loss function as a regular term to make these weight matrices as sparse as possible. L1 regularization is defined as follows:

$$\begin{array}{c}\hfill {\mathrm{R}}_1={∥W_{gci}∥}_1=\sum _{i=1}^k\left|W_{gci}\right|\end{array}$$

(22)

(2) In order to limit the differences between adjacent features extracted from graph convolution, the convolution feature regularization term based on the L2 norm is added to the loss function. L2 regularization is defined as follows:

$$\begin{array}{c}\hfill {\mathrm{R}}_2={∥G{C}^{\left\{k\right\}}∥}_2=\sqrt{\sum _{i=1}^{k-1}{\left(G{C}^i-G{C}^{i+1}\right)}^2}\end{array}$$

(23)

Thus, the total loss function is defined as follows:

$$\begin{array}{c}\hfill Loss=H\left(Y,F_l\right)+\alpha ·R_1+\beta ·R_2\end{array}$$

(24)

where the parameters $\alpha$ and $\beta$ are used to control the weight of the regularization term.

Next, we minimized the model loss function to train the network as follows:

$$\begin{array}{cc}\hfill minLoss=& min[-diag\left(V_l\right)\sum \left[Y\ast log\left(F_l\right)\right]\\ & +\alpha {\sum }_{i=1}^k\left|W_{gci}\right|\\ & +\beta \mid {\sum }_{i=1}^{k-1}{\left(G{C}^i-G{C}^{i+1}\right)}^2]\end{array}$$

(25)

When the network training was completed, we found the value with the highest quantization probability from the air-quality distribution $F_u$ and defined $f_u^{*}$ as the inferred air-quality value. The definition is as follows:

$$\begin{array}{c}\hfill f_u^{\ast }=argmax\left(F_u\right)\end{array}$$

(26)

3.2.3. Air-Quality-Monitoring Station Location Recommendation

The main objective of this section is use the proposed HGCNInf to complete the location recommendation for a air-quality-monitoring station. The ultimate goal is to recommend an optimal station location from candidate locations (this paper defaults to all unlabeled nodes) that could bring about the greatest accuracy improvement to the inference model. Therefore, we need to correlate the air-quality-inference model with the monitoring-station location recommendation model. For the trained higher-order graph convolutional network, its weight parameters reflect the correlation degree between nodes. Since the designed HGCNInf takes into account both the spatiotemporal interactions of air pollutant distribution and the complex external factors, its convolution weight parameter represents the correlation degree between the nodes reflecting the spatiotemporal changes in air-quality distribution. This is of great significance to the monitoring-equipment location selection.

In the USTG, a higher correlation degree between a node and the remaining nodes indicate that the air quality of the node is easily inferred. Therefore, the monitoring station should be selected at a node that has the lowest correlation with the remaining nodes. We introduced the information entropy index to evaluate the correlation degree between nodes. The smaller the information entropy value, the higher the correlation degree between nodes. We used the convolution weight parameters of the trained HGCNInf to recommend optimal monitoring station locations that could also bring about the greatest accuracy improvement to HGCNInf. In this way, the two problems are well correlated.

Greedy algorithm based on information entropy minimization

We designed a greedy algorithm for minimizing information entropy (GMIE) based on the correlation degree between nodes in USTG to recommend optimal station locations. HGCNInf is used as the driving model through the incremental learning method, iteratively marking the recommendation priority of unlabeled nodes.

The basic framework is shown in Figure 7, and the specific steps are as follows from step 1 to step 6:

1. Calculate the average convolution weight parameter of the trained HGCNInf. The average convolution weight parameter is defined as follows:

$$\begin{array}{c}\hfill \overline{W}=\frac{1}{k}\sum _{i=1}^k{\tilde{A}}^i\left|W_{gci}\right|\end{array}$$

(27)

where k is the order of the higher-order graph convolutional network and $\tilde{A}$ is the standardized adjacency matrix. Next, calculate the information entropy of the average convolution weight parameter of each node in the unlabeled node U, which is defined as follows:

$$\begin{array}{c}\hfill H\left({\overline{W}}_u\right)=-\sum _{u\in U}{\overline{W}}_{u}log{\overline{W}}_u\end{array}$$

(28)

2. Find the unlabeled node $u^{\prime }\in U$ with the smallest information entropy value in $H\left({\overline{W}}_u\right)$, and mark its recommendation priority as the lowest. Based on the air-quality distribution $f_u$ inferred from HGCNInf, associate the inferred air-quality value with the node $u^{\prime }$ and then turn the node $u^{\prime }$ from the unlabeled nodes into the labeled nodes to constitute a new graph input signal data $f_l^{\prime }$. Remove the node $u^{\prime }\in U$ from the unlabeled node U, and the remaining unlabeled nodes are represented as $U^{\prime }$.

3. Based on the new graph input data $f_l^{\prime }$, drive HGCNInf to retrain and update the network convolution weight parameters, that is, update the correlation degree between the nodes in USTG reflecting the spatiotemporal changes in air-quality distribution and then derive the new air-quality distribution $f_u^{\prime }$ of unlabeled nodes. As described in step 1, new average convolution weight parameters ${\overline{W}}^{{}^{\prime }}$ and information entropy $H{\left({\overline{W}}_u\right)}^{{}^{\prime }}$ can be calculated.

4. Find the unlabeled node $u^{″}\in U^{\prime }$ with the smallest information entropy value in $H{\left({\overline{W}}_u\right)}^{{}^{\prime }}$, and mark its priority as the second lowest. Based on the new air-quality distribution $f_u^{\prime }$ inferred from the air-quality inference model, associate the inferred air-quality value with the node $u^{″}$, and then, turn the node $u^{″}$ from the unlabeled nodes into the labeled nodes to constitute a new graph input signal data $f_l^{″}$. Remove the node $u^{″}\in U^{\prime }$ from the unlabeled node $U^{\prime }$, and the remaining unlabeled nodes are represented as $U^{″}$.

5. Repeat step 1 to step 4 above until the recommendation priority of all unlabeled nodes U is marked.

6. Furthermore, input graph signal data at other moments and repeat step 1 to step 5.

Finally, analyze the results at multiple times and average the recommendation priority of unlabeled nodes, and then recommend the Top-N nodes with the highest average recommendation priority as the new monitoring station location. These Top-N sites with the highest average recommendation priority have the highest average information entropy, that is, the lowest correlation degree with the remaining nodes. Therefore, newly established monitoring equipment at these Top-N sites could bring about the greatest accuracy improvement to HGCNInf, so that it can reflect the air quality and its development trends in a timely and accurate manner.

4. Experiments and Results

4.1. Data Processing

The graph input data in this paper isare the PM${}_{10}$ values of Beijing and the complex external influential factor data. We used one-hot encoding to process PM${}_{10}$ data and used Z-Score Normalization to process the external influential factor data, the factor data were linearly transformed to be mapped into the interval [0, 1].

The adjacency matrix of the Beijing urban spatiotemporal graph was calculated by using the threshold Gaussian kernel weighting function based on the distance between nodes [30,37]. The calculation formula is as follows:

$$\begin{array}{c}\hfill W_{ij}=\left\{\begin{array}{cc}exp\left(-\frac{dist{\left(p_i,p_j\right)}^2}{{\sigma }^2}\right),& exp\left(-\frac{dist{\left(p_i,p_j\right)}^2}{{\sigma }^2}\right)\ge \epsilon \\ 0,& \mathrm{otherwise}\end{array}\right.\end{array}$$

(29)

where $dist{\left(p_i,p_j\right)}^2$ represents the Euclidean distance between the nodes $p_i$ and $p_j$, and ${\sigma }^2$ and $\epsilon$ are the thresholds used to control the distribution and sparsity of the adjacency matrix. In this experiment, ${\sigma }^2$ and $\epsilon$ are set to 10 and 0.05 respectively.

4.2. Experimental Settings

4.2.1. Evaluation

In order to evaluate and compare the performance between different models, we used two commonly used metrics: (1) Mean Absolute Percentage Error (MAPE); (2) Root Mean Squared Error (RMSE). They are defined as follows:

$$\begin{array}{c}\hfill \mathrm{MAPE}=\frac{1}{n}\sum _{t=1}^n\left|\frac{Y_i-{\widehat{Y}}_i}{Y_i}\right| \ast 100\%\end{array}$$

(30)

$$\begin{array}{c}\hfill \mathrm{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^N{(Y_i-{\widehat{Y}}_i)}^2}\end{array}$$

(31)

where ${\widehat{Y}}_i$ and $Y_i$ are the predicted values and the real values from selecting labeled nodes for model validation, and n is the number of all predicted nodes.

4.2.2. Baselines

We compared HGCNInf and GMIE with the following baseline models.

(1) HA: Historical average. HA takes the average value of air-quality data as the predicted value.

(2) SVR: Support vector regression [38]. SVR uses the historical AQI data of each monitoring station as training data to infer the air-quality value of unlabeled nodes.

(3) AQInf: Affinity-based AQI inference model [23]. For air-quality inference problem, this is the current state-of-the-art method. It is used to infer the air-quality distribution of the unlabeled nodes by using the AQI data of the labeled ones and by combining the external influential factors.

(4) GCNInf: An air-quality inference model based on graph convolutional networks (GCNInf). We built a two-layer graph convolutional networks as described in [29]. Similar to AQInf, GCNInf was used to infer the air-quality distribution of unlabeled nodes by using the AQI data of the labeled ones and by combining with the external influential factors.

All of the neural network model implementations are based on Pytorch version: 1.1.0, and evaluation and training are based on CPU Intel(R) Core(TM) i5-4210U CPU at 1.70GHz, NVIDIA GeForce 840M.

4.3. Experimental Results

4.3.1. The Effectiveness of HCNInf

In the experiment, since we only have real data of the labeled nodes (the sub-regions with monitoring stations), for the unlabeled nodes (the sub-regions without monitoring stations), there are no real data to verify the inference accuracy of each model. Therefore, we proposed to conduct cross validation by randomly selecting 12 nodes as labeled nodes from the 17 labeled ones, which generated 12 × 8760 training data, and the remaining 5 nodes were used to verify the inference accuracy of each model, which generated 5 × 8760 verification data.

(1) The performance comparison of air-quality-inference models

We trained the above air-quality-inference models separately three times and then took the average values as the final results.

Table 2. The performance comparison of inference models.

Model	Beijing Air-Quality Data Evaluation
	MAPE (%)		RMSE
	Without External Factors	With External Factors	Without External Factors	With External Factors
HA	56.35	/	30.01	/
SVR	49.94	/	23.00	/
AQInf	35.25	21.31	16.92	10.35
GCNInf	28.52	14.10	13.93	7.24
HGCNInf	19.18	7.71	9.99	3.90

and Figure 8 show the performance comparison of our proposed HGCNInf and other baseline models on the inferred Beijing air-quality distribution in two metrics. It can be found that HGCNInf is far superior to other baseline models both in the metric MAPE and RMSE.

In addition, comparing the results of the air-quality-inference models with external factors and the models without external factors, we can find that, for the same model, after considering complex external influential factors, the error in the air-quality inference decreased significantly. Therefore, the external influential factors play a significant role in improving the inference accuracy of air-quality distribution. We can conclude that the complex external influential factors are closely related to the air-quality distribution, which also explains why the monitoring stations that are geographically close to each other in the red circle differ greatly in the monitoring values throughout the year.

Furthermore, to test the robustness of each air-quality-inference model, we randomly discarded the same labeled nodes and observed the change degree in the air-quality-inference performance.

Table 3. The robustness comparison of inference models.

The Robustness of Each Air Quality Inference Model
Model		HA		SVR		AQInf		GCNInf		HGCNInf
	Metric	MAPE (%)	RMSE	MAPE (%)	RMSE	MAPE (%)	RMSE	MAPE (%)	RMSE	MAPE (%)	RMSE
Number
0		56.35	30.01	49.94	23.00	21.31	10.35	14.10	7.24	7.71	3.90
1		59.45	31.50	51.75	24.40	23.75	11.41	15.82	7.74	9.01	3.98
2		63.38	34.02	52.15	26.4	24.15	12.01	16.13	7.82	11.63	6.62
3		69.67	37.99	55.18	29.00	27.18	13.39	18.31	8.61	12.07	6.66
4		95.05	42.96	62.00	33.00	40.68	19.52	20.27	10.69	15.18	7.52
5		131.92	50.18	73.01	40.18	61.06	27.18	26.42	12.34	22.94	11.67

and Figure 9 show the inference results of HGCNInf and other baseline models on two evaluation metrics when we gradually randomly discarded the same labeled nodes. From the above chart, we can find that our proposed air-quality-inference model HGCNInf is far more robust than other baseline models.

(2) Training efficiency of HGCNInf We considered the influences of different order K of the graph convolution in HGCNInf on inference performance. As shown in the histogram of Figure 10a,b and

Table 4. The HGCNInf performance of different orders.

	Order	K = 1	K = 2	K = 3	K = 4	K = 5
Metric
MAPE (%)	Without external factors	23.75	23.09	19.18	20.27	20.27
	With external factors	15.18	15.18	7.71	11.63	11.63
RMSE	Without external factors	11.41	11.75	9.99	10.69	10.69
	With external factors	7.52	7.52	3.90	6.62	6.62

, they showed the performance comparison of different orders of HGCNInf. For Beijing’s air-quality-distribution data, when we chose the order K of the higher-order convolutional network to be 3, the model HGCNInf had the best performance. Figure 8c and

Table 5. The HGCNInf training time of different orders.

Training Time/(s)	K = 1	K = 2	K = 3	K = 4	K = 5
Without external factors	1.79	2.62	3.50	4.61	5.71
With external factors	5.83	9.49	15.26	17.39	20.75

show the average training times per 1000 times at each moment when the HGCNInf model takes different orders. The training time increases as the order of the higher-order convolutional network increases. In addition, when the HGCNInf model considers the external influential factors, the time consumed by training increases significantly. Comprehensively considering the model performance and training efficiency, we chose K = 3 as the order of the high-order convolutional network in the HGCNInf model, and the results are shown in Table 2.

(3) Weight visualization

To better understand the HGCNInf, we visualized a proportion of the average graph convolution weight W, graph adjacency matrix A, and their product of A*W, as shown in Figure 11.

4.3.2. The Effectiveness of GMIE

The main objective of this section is to recommend several optimal locations from the candidate locations to establish new monitoring stations. In this experiment, we verify whether the nodes recommended by our proposed monitoring-station location recommendation model (GMIE) can maximize the inference accuracy of HGCNInf for inferring the air-quality distribution of the entire city. Since there is no ground real air-quality data for unlabeled nodes, it is impossible to verify the specific accuracy improvement that the recommended nodes bring to the air-quality-inference models.

We adopt a cross-validation method to randomly select 7 nodes from 17 labeled nodes as training datas, and then, we reserve 5 labeled nodes as candidate sites to establish new monitoring equipment. The remaining 5 labeled nodes are used to verify the performance improvement that the recommended nodes bring to the air-quality-inference models.

Baselines. Since to our knowledge, AQInf+GEM is currently the state-of-the-art method for monitoring-station location recommendation [18], we compared our proposed HGCNInf+GMIE with GCNInf+GMIE and AQInf+GEM. The recommendation priorities of the candidate locations marked by each monitoring-station location recommendation model are shown in the Figure 12.

We use MAPE improvement and RMSE improvement to evaluate the effectiveness of monitoring-station location recommendation models. In theory, as the number of recommended station locations increases, the accuracy of the corresponding air-quality inference model should be greatly improved because more station locations are added into the labeled nodes. That is, MAPE and RMSE should be greatly reduced. Since the candidate locations provide the recommendation priority, the node with a higher recommendation priority can bring about greater improvement to the air-quality-inference model. Therefore, when gradually increasing the number of recommended nodes according to the recommendation priority, the declining slopes of MAPE and RMSE should become smaller and smaller or the rising slopes of MAPE improvement and RMSE improvement should become smaller and smaller.

We display how much both MAPE improvement and RMSE improvement models can be obtained by gradually adding more recommended station locations shown in

Table 6. The improved MAPE and RMSE results with varying numbers of recommended locations.

The Improved MAPE and RMSE Results with the Increase in the Number of Recommended Nodes
Model		HGCNInf+GMIE		GCNInf+GMIE		AQInf+GEM
	Metric	MAPE (%)	RMSE	MAPE (%)	RMSE	MAPE (%)	RMSE
Number
0		21.57	10.83	23.31	11.26	24.15	12.01
1		12.94	6.84	20.48	10.13	23.75	11.41
2		9.01	4.27	18.30	8.61	22.80	11.22
3		8.00	3.98	16.37	7.82	22.10	10.69
4		7.71	3.90	14.70	7.40	21.60	10.47
5		7.71	3.90	14.10	7.42	21.31	10.35

, and Figure 13 and Figure 14. We can find that the performance of each station location recommendation model is consistent with the theory and that our proposed model’s performance has far exceeded the state-of-the-art method, which can bring about the greatest accuracy improvement to HGCNInf for inferring the air quality.

5. Discussion

5.1. Air-Quality Inference

The knowledge-driven method is based on the formation mechanism of air pollutants and the actual physical diffusion and migration laws, and then a suitable model is established, and the air-quality distribution in urban areas not covered by the monitoring station is predicted through calculation and simulation. Gibson et al. [39] used a Gaussian plume diffusion model to predict the PM${}_{2.5}$ distribution of main points and routes. Rakowska et al. [40] constructed an urban street canyon model to monitor the impact of traffic flow on air quality. These methods are based on a complex mechanism knowledge, and it is difficult to establish a universal model for various environments due to the adopted empirical assumptions. For example, the Gaussian plume diffusion model that is widely used in the field of air pollution assumes that the distribution of air pollutants conform to the Gaussian distribution. However, in an urban street environment, the diffusion of mobile source pollutants on the road is affected by the buildings on both sides of the road, which does not satisfy this model.

Data-driven methods are also constantly developed and have achieved good results in fine-grained air-quality prediction in space. Zheng et al. [8] considered the influence of external factors related to air-quality distribution and proposed a semi-supervised learning method based on the co-training collaborative training framework [41] to infer the fine-grained air-quality distribution in urban space. However, the framework consists of two independent classifiers, and it is difficult to capture the complex interactions between different features. Chen et al. [42] used the K-nearest neighbor strategy to construct the relationship between external data sources (such as road network structure, traffic, etc.) and air-quality distribution. However, there is no exact theoretical standard for the specific value of K. The characteristics of the latest K sites may not be the most effective, and the degree of influence of different sites may change over time, which may lead to inconsistency problems. Hsieh et al. [23] designed an affinity-based air-quality-inference model AQInf (the affinity-based AQI inference model (AQInf)) based on graph label node propagation. However, due to the limitation of its own model performance, it is difficult for AQInf to capture the correlation between graph nodes that reflect the trend for air-quality temporal and spatial changes.

Our proposed HGCNInf consists of a higher-order graph convolution network (HGCN) and several fully connected neural networks (FNN). By inputing historical monitoring data and by fetching global information, the HGCN is used to capture the spatiotemporal interactions of air-quality distribution and the fully connected neural network is used to extract external influential factor features, so that HGCNInf could accurately infer the air-quality distribution of an entire urban area. As shown in Figure 8 and Figure 9, the prediction accuracy and robustness of HGCNInf are far better than those of each benchmark model. The experimental results show that our proposed model is also applicable even when there are very few label nodes available. In addition, the difference in the two evaluation indicators between the air quality inference model without external influencing factors and the external influencing factors added was compared. The same model after considering the influence of external factors improves the accuracy of urban air-quality distribution prediction and has a significant effect. Therefore, we can conclude that complex external influencing factors are closely related to the distribution of air quality, which also explains the phenomenon why the monitoring sites that are very close geographically in Figure 1 have large differences in values throughout the year.

5.2. Air-Quality-Monitoring Station Location Recommendation

HGCNInf takes the spatiotemporal interactions of pollutant distribution and external influential factors into account, for which the model convolution weight parameters actually represent the correlation degree between the nodes in USTG that reflects the spatiotemporal changes in air quality. Based on the correlation degree, we designed a GMIE that aimed to mark the recommendation priority of unlabeled nodes according to the ability to improve the inference accuracy of HGCNInf to complete the monitoring-station location recommendation. According to Figure 13 and Figure 14, our proposed model’s performance far exceeded the state-of-the-art method, which can bring about the greatest accuracy improvement to HGCNInf for inferring the air quality.

6. Conclusions

In this paper, we formulated the monitoring station location as a urban spatiotemporal graph (USTG) node recommendation problem. We designed an effective air-quality inference model HGCNInf based on a proposed higher-order graph convolution network for inferring the air-quality distribution throughout the city. HGCNInf can not only capture the spatiotemporal interactions of atmospheric pollutant distribution but also extract complex external influential factor features, so that HGCNInf could accurately infer the air-quality distribution of the entire urban area. In addition, HGCNInf can also learn the correlation between the nodes in USTG reflecting the air-quality distribution. Then, we designed a greedy algorithm for minimizing information entropy (GMIE) based on the correlation degree, marking the recommendation priority of unlabeled nodes according to the ability to improve the inference accuracy of HGCNInf to complete the task of recommending the optimal station location. The final recommended optimal station can bring HGCNInf the greatest accuracy improvement. By evaluating real-world Beijing air quality data, HGCNInf and GMIE far outperform the baselines, including the state-of-the-art method.

In the future, we will consider transfer learning methods to improve the applicability of the model to different regions and apply the proposed model to other areas of urban computing, such as traffic flow monitoring, urban regional population flow prediction, etc.