Time Series-Based PM_2.5 Concentration Prediction Model Incorporating Attention Mechanism

Abstract

As a key indicator of air quality, effective forecasting of ${\mathrm{PM}}_{2.5}$ concentration can provide key technical support for the scientific and precise implementation of air pollution prevention and control. However, predicting ${\mathrm{PM}}_{2.5}$ concentrations faces challenges such as multiple influencing factors, long-term temporal dependencies, and inherent nonlinearity. Furthermore, traditional Long Short-Term Memory (LSTM) networks not only fail to effectively grasp the dependency relationships in long-time-span data, but also encounter difficulties in fully integrating and exploiting the information of numerous influencing factors. In order to solve these problems, a novel prediction model (OVMD–PeepholeLSTM–attention) for ${\mathrm{PM}}_{2.5}$ concentration was presented in this study, which includes Peephole Long Short-Term Memory (PeepholeLSTM), optimal variational mode decomposition (OVMD) and an attention mechanism (AM). In this study, K modal components result from the initial decomposition of ${\mathrm{PM}}_{2.5}$ monitoring data using OVMD. The obtained components are then individually predicted by the PeepholeLSTM–attention model, and the final prediction is reconstructed. The proposed model was comprehensively evaluated on ${\mathrm{PM}}_{2.5}$ concentration monitoring data sets from Guangzhou and Shenzhen in China from 2020 to 2022, through a series of comparative experiments. The model proposed in this study is shown by experimental results to reduce mean absolute error (MAE) by approximately 39%, root mean square error (RMSE) by 45%, and increases the fitting coefficient ($R^2$) by 0.0457 in Guangzhou compared to the single PeepholeLSTM model. The corresponding improvements in Shenzhen are 45% for MAE, 51% for RMSE, and 0.0765 for $R^2$. This indicates that the model proposed in this paper exhibits higher accuracy in terms of predicting ${\mathrm{PM}}_{2.5}$ concentrations, and the research results can provide a basis for quantitative assessment and scientific decision-making for the sustainable development of urban ecological environments.

1. Introduction

Air quality in China has deteriorated markedly amid rapid economic growth, and ${\mathrm{PM}}_{2.5}$ pollution has become increasingly severe. These fine particles carry a multitude of harmful substances, remain suspended for long durations, and can travel considerable distances. Thus, they act as a carriers of various harmful pollutants in the air, significantly degradeing air quality and visibility. And they also simultaneously constitute serious threats to human health, such as impaired renal function, coronary artery disease, asthma, lung cancer, and other respiratory diseases [1,2,3,4]. Therefore, building a high-precision ${\mathrm{PM}}_{2.5}$ concentration prediction model is of key significance for achieving proactive warnings of air pollution and precise management and control. An accurate prediction model can quantify the spatiotemporal evolution patterns of air pollutants, providing quantitative decision-making support for the sustainable management of urban ecological environments. Its prediction results can directly support practical actions such as the dynamic formulation of pollution control measures, the scientific optimization of emission reduction plans, and the efficient allocation of environmental governance resources. Research on ${\mathrm{PM}}_{2.5}$ concentration prediction has been validated and applied in practical scenarios such as urban air environment monitoring and regional sustainable development assessment, both domestically and internationally, becoming an important technological support for the promotion of sustainable development in the ecological environment field.

At present, approaches to forecasting ${\mathrm{PM}}_{2.5}$ concentrations generally fall into two broad categories: mechanistic and machine learning models. Mechanistic approaches rely on fundamental physical and chemical principles for their implementation. These approaches simulate the diffusion, transport, and deposition processes of pollutant plumes, with the ultimate goal of estimating ambient pollutant concentrations [5,6,7,8]. These methods provide good interpretability. However, such models require prior knowledge of emission sources, meteorological conditions, and geographical features, and rely on empirical parameters and assumptions to estimate future concentration changes, which limits their applicability to specific scenarios [9]. For example, AERMOD is an empirical model primarily designed for simulating and predicting small-scale pollutant dispersion [5]. When third-generation air quality models are applied, complete emission inventories and accurate meteorological fields are required as inputs, which renders these models inapplicable in most cases [6]. Prediction methods based on mechanistic models simulate the physical evolution of atmospheric elements and their interregional transport. Although mechanism-based methods can simulate the physical evolution and interregional transport of atmospheric components, their prediction accuracy is largely constrained by the difficulty of obtaining precise input data as well as sufficient knowledge of emission sources, reaction mechanisms, and chemical kinetics [10,11].

Compared with mechanism-based approaches, machine learning methods are more effective in terms of capturing the nonlinear characteristics of ${\mathrm{PM}}_{2.5}$ and have become a research hotspot worldwide. Traditional machine learning encompasses a wide range of predictive models. Among these, Support Vector Machines (SVMs) stand out as representative and widely applied approaches. By employing Particle Swarm Optimization (PSO) to optimize a hybrid kernel SVM, Zhang et al. developed a PM_2.5 prediction model that achieved strong accuracy and efficiency [12]. The field of ${\mathrm{PM}}_{2.5}$ prediction has also seen the application of the Extreme Learning Machine (ELM)—a rapid learning algorithm for Single Hidden Layer Feedforward Neural Networks (SLFNs), where its optimized variants have demonstrated promising predictive performance [13,14]. Studies further indicate that meteorological factors, such as surface pressure, precipitation, and temperature, as well as other pollutants including PM₁₀, CO, and SO₂, play significant roles in influencing ${\mathrm{PM}}_{2.5}$ concentration levels [15,16]. Although traditional machine learning methods exhibit certain capabilities in handling nonlinear data, they are limited in extracting deeper representations from large-scale datasets [17]. Building on traditional machine learning, the advanced deep learning provides a new paradigm for atmospheric time series prediction by enabling hierarchical feature learning from large inputs. The deep learning field encompasses a variety of specialized architectures, notably Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Generative Adversarial Networks (GANs). RNNs have demonstrated particular advantages in time series modeling [18]. It is well-established that LSTM, the predominant RNN variant, is extensively applied to ${\mathrm{PM}}_{2.5}$ prediction [19,20]. By introducing memory cells and gating mechanisms, LSTM effectively addresses the long-term dependency problem and retains temporal information. Designed as a streamlined alternative to LSTM, the Gated Recurrent Unit (GRU) reduces model complexity and accelerates training, which has led to its wide adoption in ${\mathrm{PM}}_{2.5}$ prediction tasks [21,22,23,24]. However, the relationship between ${\mathrm{PM}}_{2.5}$ concentrations and meteorological factors is characterized by a dynamic interplay, where complexity stems from the interconnectedness of multiple variables rather than from any single factor. When considered independently, these factors cannot fully capture the interactive effects of multiple variables on ${\mathrm{PM}}_{2.5}$ concentrations, which limits prediction accuracy.

To elevate the accuracy of predictive models, numerous scholars have explored a range of avenues, such as optimizing the internal architecture of existing models and integrating heterogeneous model frameworks into a unified hybrid system. A prominent example of such optimized architectures is the Bidirectional Long Short-Term Memory (BiLSTM) network, an advanced iteration of the traditional LSTM that mitigates predictive deviations by effectively capturing sequential features in both forward and backward directions [25]. Other studies have focused on parameter optimization of LSTM models using optimization algorithms and their improved versions, including the Genetic Algorithm (GA) [26], Quantum Particle Swarm Optimization (QPSO) [27], Bayesian Optimization (BO) [28], and the Whale Optimization Algorithm (WOA) [29]. These algorithms perform global searches to automatically determine optimal hyperparameters (e.g., training epochs, learning rate, and batch size), thereby avoiding inefficient manual tuning and improving stability and accuracy. In addition, hybrid neural networks have been developed to exploit the complementary strengths of different architectures, improving accuracy, generalization, adaptability to complex data, and robustness against noise. For example, the fusion of convolutional and recurrent networks has emerged as a viable strategy to enhance the efficiency of prediction tasks in deep learning applications [30,31,32]. Ding et al. [33] proposed a hybrid model integrating LSTM and a weighted Random Forest (RF) for ${\mathrm{PM}}_{2.5}$ prediction. In this framework, by employing RF to assess the importance of input variables (e.g., temperature, wind speed, and historical ${\mathrm{PM}}_{2.5}$ concentrations), subsequently identifying and selecting the most relevant features. A fully connected network (FCN) was then employed to assign weights to these features, quantifying their relative influence. Finally, the LSTM processed the weighted feature sequence to capture long-term temporal dependencies, enabling it to predict ${\mathrm{PM}}_{2.5}$ concentrations over the subsequent six hours.

In order to optimize the model’s performance and improve the accuracy of predictions, previous studies have explored model enhancements and the fusion of different neural networks. Although some progress has been achieved, limitations remain in capturing complex feature relationships within the data. With the widespread application of attention mechanisms across various fields [34,35], the combination of data preprocessing and machine learning has provided new opportunities for improving prediction performance. Through probabilistic weighting, attention mechanisms extract long-term dependencies and minimize information loss, thereby enhancing overall predictive accuracy. It has been confirmed that ${\mathrm{PM}}_{2.5}$ prediction models combined with attention mechanisms achieve improved accuracy and generalization [36,37,38]. For example, aiming at two-day-ahead forecasts of ${\mathrm{PM}}_{2.5}$, Zhang et al. [39] developed a hybrid CNN–BiLSTM–attention model. This model is designed as a multi-modal fusion system, synthesizing strategies from convolutional, recurrent, and attention-based neural processing. Experiments that were evaluated on data from Shunyi District, Beijing, show that this model outperforms Lasso regression, ridge regression, and XGBoost in both short-term and long-term predictions. It is noteworthy that integrating an attention mechanism into the model can improve prediction accuracy, but it requires calculating attention weights at each time step [25,38,40].

In addition, the integration of data preprocessing techniques has been shown to improve ${\mathrm{PM}}_{2.5}$ prediction accuracy. This is achieved by simplifying raw sequences and decomposing them into different distinct subsequences, with each bearing a variety of feature information [37]. Notably, researchers have developed hybrid models by fusing neural networks with Empirical Mode Decomposition (EMD), its derivatives, and Variational Mode Decomposition (VMD); these models have since delivered significant improvements in ${\mathrm{PM}}_{2.5}$ prediction accuracy [37,38,41,42]. To address the challenges in ${\mathrm{PM}}_{2.5}$ concentration prediction within 0–24 h, Teng et al. [43] introduced a hybrid model that incorporates EMD and Sample Entropy (SE) into a BiLSTM framework. Experimental results indicated that this model outperformed single deep learning models, with short-term (within 6 h) prediction accuracy improved by at least 50%. These studies indicate that employing hybrid data decomposition techniques, coupled with further refinement through secondary decomposition, can achieve more detailed feature extraction and thus enhance the model’s predictive accuracy [44].

Overall, despite the significant progress made in ${\mathrm{PM}}_{2.5}$ prediction, numerous limitations remain unresolved in current research. For ${\mathrm{PM}}_{2.5}$ concentration time series characterized by seasonality and influenced by multiple complex factors, existing models fail to effectively address non-stationarity and lack refined decomposition of fluctuating components, often leading to mode mixing. Additionally, most models have a limited ability to focus on key information such as meteorological factors, face difficulties in multi-source data fusion, and some hybrid models suffer from structural redundancy. Although attention mechanisms have been introduced, they are often simply combined with existing models without fully exploiting their advantages. These techniques provide models with higher-quality inputs, enabling efficient fusion of multi-source data and collectively enhancing the capacity of the model for processing sophisticated data structures. Despite the progress achieved, substantial improvements are still required to realize deep collaboration among techniques and to enhance real-time prediction. Therefore, building a real-time ${\mathrm{PM}}_{2.5}$ concentration prediction model that can accurately forecast dynamic climate changes while offering high adaptability, high interpretability, and high precision is not only a practical requirement for precise air pollution control and sustainable ecological management, but also a key scientific challenge that urgently needs to be addressed.

To address these challenges, this paper proposes a high-accuracy ${\mathrm{PM}}_{2.5}$ concentration prediction method, with the technical process shown in Figure 1. Using ${\mathrm{PM}}_{2.5}$ monitoring data from Guangzhou and Shenzhen from 2020 to 2022 as the research object, after completing data supplementation, the seasonal variation characteristics of ${\mathrm{PM}}_{2.5}$ concentration are first analyzed. The correlations between pollution factors (${\mathrm{SO}}_2$, CO, ${\mathrm{O}}_3$, and ${\mathrm{NO}}_2$) and meteorological factors (precipitation, temperature, atmospheric pressure, etc.) with ${\mathrm{PM}}_{2.5}$ concentration are explored, thereby revealing the driving mechanisms of ${\mathrm{PM}}_{2.5}$ concentration changes. Subsequently, the preprocessed dataset is divided and normalized. Employing the ‘decomposition–prediction–reconstruction’ modeling approach, this is input into the OVMD–PeepholeLSTM–attention model (hereafter referred to as PeepholeLSTM-OA) to obtain the final ${\mathrm{PM}}_{2.5}$ concentration predictions. Finally, quantitative evaluation indicators such as MAE, RMSE, and $R^2$ are calculated to comprehensively assess the predictive performance of the model.

2. Research Data

2.1. Overview of the Study Area

Functioning as both the “dual engines” driving the Pearl River Delta region and major hubs in the global economy, Guangzhou and Shenzhen face severe challenges in air quality management during their rapid development (Figure 2). Studies indicate that Guangzhou’s ${\mathrm{PM}}_{2.5}$ concentrations display marked seasonal and regional differences—higher levels occur in winter under the influence of continental cold air masses, while summer conditions of high temperature and heavy rainfall facilitate pollutant dispersion and removal [45]. The mountainous terrain in the northern region helps disperse pollution, resulting in lower concentrations. In contrast, the southern plains experience higher concentrations due to poor dispersion conditions and dense industrial and transportation activities. During stable weather conditions, certain industrial zones in Shenzhen are prone to rising concentrations caused by emission accumulation. The complex topography, characterized by alternating mountains and seas, renders ${\mathrm{PM}}_{2.5}$ pollution subject to the combined effects of local emissions and regional transport, leading to distinct concentration trends across different areas.

In recent years, the governments of two regions have continuously advanced air quality improvements through multiple rounds of targeted initiatives, including industrial restructuring and precise pollution source management [46,47]. Even though the effectiveness of managing anthropogenic sources continues to become apparent, regional air quality is still influenced by a combination of natural and human factors, and the pressure for precise and proactive pollution control has not diminished. For example, in the spring of 2024, affected by the long-distance transport of northern dust storms, ${\mathrm{PM}}_{2.5}$ concentrations at some monitoring stations in Guangzhou exceeded 100 μg/m³ on a single day, revealing a composite pollution pattern characterized by “external input combined with local generation” [48]. In Shenzhen, some sites exceeded 35 μg/m³ for three consecutive days within a week, with a peak of 42 μg/m³, breaking the city’s long-standing relatively stable air quality pattern. These types of pollution events, caused jointly by seasonal changes, natural meteorological processes, and human economic activities, are characterized by suddenness, short duration, and significant fluctuations in concentration. They are difficult to effectively manage through traditional extensive control measures. However, high-resolution hourly ${\mathrm{PM}}_{2.5}$ concentration forecasts can accurately capture the real-time concentration changes of such complex pollution, providing critical data support for timely activation and precise implementation of urban pollution warnings and emergency control measures.

2.2. Data Acquisition and Processing

Air pollutant data were obtained from the National Urban Air Quality Real-Time Publishing Platform of the China National Environmental Monitoring Center. Meteorological data were retrieved from the ERA5 reanalysis dataset provided by the European Center for Medium-Range Weather Forecasts (ECMWF). The ERA5 dataset provides global meteorological records featuring a one-hour temporal resolution. The experiment is based on concurrent hourly data for air pollutants and meteorological factors, covering the period between January 2020 and December 2022, were collected and processed, yielding 24,709 samples for Guangzhou and 26,163 samples for Shenzhen. The environmental monitoring stations used in this study were 1345A in Guangzhou and 1359A in Shenzhen. Although this period fell within the special context of global pandemic prevention and control, during which anthropogenic emissions (such as from industrial production, transportation, and residential activities) experienced phased adjustments, the timeframe still maintains both data continuity and representativeness of regional pollution characteristics. It allowed for continuous monitoring of pollutants and meteorological data at stations in Guangzhou and Shenzhen throughout the entire period, enabling a comprehensive capture of the coupling patterns between natural and anthropogenic sources of ${\mathrm{PM}}_{2.5}$. The research findings therefore provide valuable reference for complex urban pollution control.

Due to the large sample size and high sampling frequency of hourly air pollution and meteorological data, there are unavoidable data losses in the data collection process caused by equipment malfunctions, maintenance, or extreme weather conditions. Missing data may lead to information loss, thereby affecting model training and predictive performance. Therefore, selecting an appropriate imputation method to ensure data completeness is an essential preprocessing step in model construction. In order to ensure the integrity of sample information and make the imputed values closer to the real situation, this study deletes all time points with complete missing values for all variables in the dataset. The MICE method is then used to fill in time points with only partial missing values.

After data imputation, time series plots of ${\mathrm{PM}}_{2.5}$ concentrations in Guangzhou and Shenzhen were obtained. As shown in Figure 3, both cities exhibited lower ${\mathrm{PM}}_{2.5}$ concentrations during spring and summer each year from 2020 to 2022. Concentrations gradually increased during autumn and winter, often peaking in winter and exhibiting a certain degree of periodicity in their variation.

Studies have shown that the variation mechanisms of ${\mathrm{PM}}_{2.5}$ concentrations are highly complex and dynamic. Air pollutants and meteorological factors exert diverse influences on ${\mathrm{PM}}_{2.5}$ fluctuations. Therefore, investigating the correlations and driving effects of these factors not only provides reasonable explanations for ${\mathrm{PM}}_{2.5}$ variation but also offers theoretical support for targeted pollution control measures. In this study, ${\mathrm{SO}}_2$, CO, ${\mathrm{O}}_3$, ${\mathrm{NO}}_2$, and ${\mathrm{PM}}_{10}$ were selected as air pollutant factors, while total precipitation (tp), dew point temperature (d2m), surface pressure (sp), wind components (v10 and u10), and temperature (t2m) were chosen as meteorological factors.

The correlations among pollutant factors, meteorological factors, and ${\mathrm{PM}}_{2.5}$ concentrations were evaluated using the Spearman correlation coefficient. The calculation is expressed by Equation (1) as follows:

$$\rho =1-{\displaystyle \frac{6\sum d_i^2}{n(n^2-1)}}$$

(1)

where $\rho$ denotes the Spearman correlation coefficient, $d_i$ represents the difference between the ranks of two variables, and n is the sample size. The value of $\rho$ ranges from −1 to 1. Specifically, a value of $\rho =1$ corresponds to a perfect positive linear relationship between variables, while $\rho =-1$ signifies a perfect negative linear relationship.

Figure 4 shows the Spearman correlation results. This analysis serves as an important preliminary quantitative basis for feature selection in subsequent deep learning model development, providing direct support for the selection of input features and the determination of parameters for the model. It indicates a very strong relationship between ${\mathrm{PM}}_{2.5}$ and ${\mathrm{PM}}_{10}$, with coefficients above 0.9 in both datasets. Other pollutant factors, including ${\mathrm{SO}}_2$, CO, ${\mathrm{O}}_3$, and ${\mathrm{NO}}_2$, also exhibited positive correlations with ${\mathrm{PM}}_{2.5}$ to varying degrees. Among the meteorological factors, all except surface pressure (sp) showed negative correlations with ${\mathrm{PM}}_{2.5}$, with total precipitation (tp) and dew point temperature (d2m) demonstrating particularly strong negative relationships. The above results confirm that the selected pollutants and meteorological factors are all significantly correlated with changes in ${\mathrm{PM}}_{2.5}$ concentrations, and no weakly correlated factors can be excluded. Therefore, this study includes all analyzed factors as core feature variables in the input layer of the deep learning model, providing complete data support for the model to capture the multidimensional driving features of ${\mathrm{PM}}_{2.5}$ concentrations. This also serves as prior knowledge to assist in the targeted adjustment of hyperparameter tuning.

3. Research Methods

In the current research on time series prediction of ${\mathrm{PM}}_{2.5}$ concentrations, the original monitoring data is highly non-stationary and significantly affected by noise, and it is difficult for a single deep learning model to accurately capture the key features and dynamic relationships of the sequence. To further improve the accuracy and robustness of prediction models and make them more applicable to practical regional air pollution control needs, this paper proposes a ${\mathrm{PM}}_{2.5}$ concentration prediction model, PeepholeLSTM-OA. This model integrates optimal variational mode decomposition (OVMD), Peephole Long Short-Term Memory network (PeepholeLSTM), and an attention mechanism (AM). OVMD effectively mitigates the non-stationarity and noise interference in the original data, and the attention mechanism is introduced into the PeepholeLSTM model, enabling the model to dynamically focus on the parts of the input sequence most relevant to the current output, thereby enhancing model performance.

3.1. Peephole Long Short-Term Memory Network (Peephole LSTM)

Peephole Long Short-Term Memory (PeepholeLSTM) is a variant of Long Short-Term Memory networks (LSTMs) [49]. It extends and optimizes the traditional LSTM, thereby enhancing its capability to process complex data. The structure of the PeepholeLSTM unit is illustrated in Figure 5. The ${\mathrm{PM}}_{2.5}$ concentration time series exhibits obvious long-term dependencies and phased abrupt change characteristics. In actual monitoring data, ${\mathrm{PM}}_{2.5}$ concentrations often show—under a relatively stable background level, influenced by changes in meteorological conditions (such as stagnant weather or temperature inversion) and sudden surges in human emissions—short-term spikes or rapid accumulation followed by slow decay, and there is usually a long interval between these high-pollution events. This characteristic means that, when updating the gating state, models need to accurately perceive and distinguish whether ‘the current change is sufficient to break the existing pollution accumulation state.’ The standard LSTM gating mechanism relies only on the current input and the previous hidden state. In long-span sequences, the influence of the cell state on gating decisions is indirect, which can easily lead to insufficient perception of the pollution accumulation level, especially near concentration spikes or turning points, where gate updates may lag. Peephole LSTM introduces direct access to the previous cell state in the input gate, forget gate, and output gate, enabling gating decisions to explicitly sense the current ${\mathrm{PM}}_{2.5}$ accumulation level and evolution trend. The update formulas for the gating unit states are provided in Equations (2)–(4).

$$f_t=\sigma (W_f·[C_{t-1},h_{t-1},x_t]+b_f)$$

(2)

$$i_t=\sigma (W_i·[C_{t-1},h_{t-1},x_t]+b_i)$$

(3)

$$o_t=\sigma (W_o·[C_t,h_{t-1},x_t]+b_o)$$

(4)

where $f_t$, $i_t$, and $o_t$ denote the forget gate, input gate, and output gate at time step t respectively; $\sigma$ is the Sigmoid activation function; W represents the gate weight matrix corresponding to each gate; $C_{t-1}$ is the previous cell state; $h_{t-1}$ is the previous hidden state; $x_t$ is the current input; and b denotes the bias vector.

3.2. Attention Mechanism

The core operation of the attention mechanism is computing a weight vector that quantifies the importance of each input sequence element to the current output, essentially implementing a weighted processing procedure. Its core idea is to enable the model to dynamically focus on the parts of the input sequence that are most relevant to the current output, by increasing the weight of important sequences and reducing the weight of non-important data, thereby improving the model’s performance. Especially in time series tasks, the attention mechanism helps the model automatically select historical data most relevant to the prediction, reducing reliance on all inputs. At the same time, it can also alleviate the issues of uneven temporal contributions and redundant multivariate input information in the formation process of ${\mathrm{PM}}_{2.5}$, thereby enhancing the model’s ability to capture key stages of pollution evolution. Combined with the cumulative memory characteristics of Peephole LSTM, the attention mechanism further strengthens the model’s capability to represent critical moments and key driving factors, which partly explains the improvement in the model’s predictive performance. Standard attention mechanisms typically involve three vectors: the Query vector (Q), which represents the current task’s focus information; the Key vector (K), which indexes input sequence elements for matching with Q; and the Value vector (V), which contains each element’s actual information and is aggregated via weighted summation. The attention mechanism can generally be divided into the following steps:

(1)

Calculate the attention scores. The attention score is derived from a similarity function between the Q and K. Common methods for calculating similarity are shown in

Table 1. Similarity function.

	Additive Attention	Dot-Product Attention	Scaled Dot-Product Attention	Bilinear Attention
Similarity	$v^{T}tanh(W_{q}q+W_{k}k)$	$q·k$	${\displaystyle \frac{q·k}{\sqrt{d_k}}}$	$q^{T}Wk$

.

(2)

Calculate attention weights. To obtain the final attention weights, the raw attention scores are normalized through the application of a softmax function. These weights represent the relative importance of each input element to the current output.

$${\alpha }_i={\displaystyle \frac{exp\left(\mathrm{score}(q,k_i)\right)}{{\sum }_{j=1}^{n}exp\left(\mathrm{score}(q,k_j)\right)}}$$

(5)

(3)

Weighted Summation. Use attention weights to perform a weighted sum of the Value vectors (V), resulting in the final attention value.

$$\mathrm{attention}=\sum _{i=1}^n{\alpha }_i{v}_i$$

(6)

3.3. Optimal Variational Mode Decomposition (OVMD)

The Variational Mode Decomposition (VMD) algorithm is an adaptive signal processing and modal decomposition method that can decompose a time series into K intrinsic mode functions (IMFs) with different central frequencies and bandwidths, thereby achieving effective separation of the original signal [50]. The expression of an amplitude–frequency modulated signal is given as follows:

$$u_k\left(t\right)=A_k\left(t\right)cos\left[{\phi }_k\left(t\right)\right]$$

(7)

where K denotes the number of decomposition modes, $A_k\left(t\right)$ represents the modal component, $u_k\left(t\right)$ is its amplitude, and ${\phi }_k\left(t\right)$ denotes the instantaneous phase of the mode component $u_k\left(t\right)$.

Through continuous iterative updates of modal components and central frequencies, it decomposes the original data into K intrinsic mode functions (IMFs) with distinct frequency characteristics. The efficacy of VMD and the subsequent predictive accuracy of the model are largely influenced by the choice of K. An insufficient number of decomposed modes will lead to insufficient decomposition and ineffective identification of data patterns. Conversely, an excessive modal decomposition can result in mode mixing between adjacent components. Therefore, in this study, the Optimal Variational Mode Decomposition (OVMD) algorithm was employed to decompose the ${\mathrm{PM}}_{2.5}$ monitoring data.

Unlike VMD, which requires empirical setting of hyperparameters, the OVMD algorithm determines the optimal value of K by analyzing the distribution of the center frequencies of the IMF components. In this way, the decomposition avoids both insufficient separation and mode mixing. The update step size $\tau$ is a core hyperparameter used in the OVMD decomposition process to control the convergence speed of signal decomposition iterations and to balance decomposition accuracy and computational efficiency. Its value directly affects the Residual Evaluation Index (REI), which in turn determines the rationality of the OVMD decomposition results: if $\tau$ is too large, it can lead to insufficient decomposition and larger residuals, while if it is too small, it increases the computational load and may cause over-decomposition. After the optimal K is obtained, the OVMD method reconstructs the IMF components generated under different step sizes $\tau$ and calculates their Relative Entropy Index (REI). The step size $\tau$ that yields the minimum REI value is selected as the optimal parameter, with the range of step size $\tau$ as [0, 1].

In this study, during the sample construction phase for ${\mathrm{PM}}_{2.5}$ concentration prediction, the original dataset was first divided into a training set and a test set according to a time sequence. OVMD decomposition was performed only on the training set data to determine the optimal number of modes K, the update step $\tau$, and other core parameters. Subsequently, the decomposition parameters determined from the training set were directly applied to the independent test set data for decomposition and subsequent prediction. A sliding window method was used throughout both the training and test sets to partition input and output sequences. The length of the sliding window was determined based on data characteristics and model input requirements, and the window moved continuously during iterations, which effectively reduced the impact of boundary effects on the error of real-time prediction results. The main decomposition procedure of OVMD is illustrated as follows:

(1)

The optimal number of decomposition modes (K) is determined by iteratively computing and analyzing the distribution of modal center frequencies across candidate K values.

(2)

The modal component sequences are reconstructed, and the Residual Evaluation Index (REI) between the reconstructed sequence and the original sequence is calculated to determine the optimal step size. The calculation of REI is given in Equation (8).

$$\mathrm{REI}=min{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\left[\sum _{k=1}^K{U}_k-f\right]}_i$$

(8)

where U denotes the number of decomposed modes, f represents the original signal, and N is the total number of signal samples.

3.4. Model Construction

The PeepholeLSTM-OA model adopts a “decomposition–prediction–reconstruction” framework, as illustrated in Figure 6.

In the decomposition stage, OVMD is employed to decompose the ${\mathrm{PM}}_{2.5}$ monitoring data, transforming the complex time series into multiple relatively stationary subsequences (IMFs). This adaptive decomposition alleviates non-stationarity and noise in the raw data, thereby providing high-quality inputs for subsequent prediction. In the prediction stage, PeepholeLSTM is used as the base learner. Compared with the traditional LSTM, PeepholeLSTM introduces direct connections from the cell state to the input, forget, and output gates, which enables the network to capture long-term dependencies more accurately within the gating mechanism and enhances its ability to model complex time series. Furthermore, an attention mechanism is incorporated into the PeepholeLSTM framework. The attention mechanism assigns weights to the contributions of different IMF components, allowing the model to focus on critical information while preserving more detailed data features, thus providing a solid foundation for performance improvement. In the reconstruction stage, the prediction results of all IMF components are aggregated through weighted summation to generate the complete ${\mathrm{PM}}_{2.5}$ concentration sequence. Through this “decomposition–prediction–reconstruction” process, the PeepholeLSTM-OA model achieves comprehensive extraction and utilization of multi-scale features, effectively addressing the non-linearity, non-stationarity, and multiple spatiotemporal characteristics of ${\mathrm{PM}}_{2.5}$ concentration series. Consequently, the model improves forecasting performance and evaluation metrics, offering a more reliable technical framework for air quality prediction and early warning. The specific prediction procedure is detailed as follows:

(1)

${\mathrm{PM}}_{2.5}$ monitoring data were processed for missing value imputation, feature selection, normalization, and split into training and testing sets. These data, together with pollutant and meteorological factors, were used as model inputs (x), then the data were split the dataset into a training set and a test set in an 8:2 ratio.

(2)

The training set ${\mathrm{PM}}_{2.5}$ time series were decomposed using OVMD. The optimal number of modes K was determined by iteratively evaluating the center frequencies of IMF components, and the update step size $\tau$ was set by minimizing the REI. After obtaining the OVMD decomposition parameters, the test set was then decomposed into K intrinsic modal functions (IMF 1, IMF 2, …, IMF K) of different frequency features. Each captured different frequency features such as trends, periodic variations, and high-frequency fluctuations in the original data, thus achieving ${\mathrm{PM}}_{2.5}$ data decomposition.

(3)

Each IMF subsequence was subjected to separate prediction via a PeepholeLSTM network, with the network leveraging the preceding 12 h of data to forecast the hourly ${\mathrm{PM}}_{2.5}$ concentration for the subsequent time step, obtaining prediction results for each IMF component. In this framework, the Peephole LSTM captured trend and seasonal patterns while learning nonlinear temporal dependencies and long-term correlations.

(4)

An attention mechanism was applied to assign weights ${\alpha }_i$ to each IMF prediction according to its importance, generating a weighted output.

(5)

The PeepholeLSTM-OA model predictions were fused and reconstructed on the test set. Model performance was evaluated using mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination ($R^2$).

3.5. Error Evaluation Indices

To evaluate model performance, the coefficient of determination ($R^2$) was used to measure regression line fitting accuracy. This metric ranges from 0 to 1, with values close to 1 signifying strong alignment between observed and predicted data. The MAE and RMSE were also computed to quantify the PeepholeLSTM-OA model’s predictive performance and generalization capability for ${\mathrm{PM}}_{2.5}$ concentrations. The formulas for $R^2$, MAE and RMSE are shown in Equations (9)–(11), respectively, as follows:

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}{{\sum }_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}$$

(9)

$$\mathrm{MAE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|y_i-{\widehat{y}}_i\right|$$

(10)

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}$$

(11)

where n denotes the number of data samples, ${\widehat{y}}_i$ represents the predicted value, $y_i$ denotes the observed value, and ${\overline{y}}_i$ is the mean of the observed values.

4. Analysis of ${\mathbf{PM}}_{\mathbf{2.5}}$ Prediction Results

For neural network and machine learning models, parameter selection significantly affects training cost and predictive performance. Proper parameter settings can enhance model training efficiency, improve prediction accuracy, and reduce the risk of overfitting. Among network parameters, the number of neurons is critical: increasing the number of neurons can improve the model’s fitting ability for complex time series, but excessive neurons may lead to overfitting and higher computational cost. Epoch denotes the number of times the entire training set is processed; an appropriate value balances generalization and computational efficiency. Batch size refers to the number of samples used to update the model in each iteration, and powers of two (e.g., 32, 64, 128) are commonly chosen to optimize computational performance. In this study, the neural network models (LSTM, GRU, and PeepholeLSTM) were configured with a single hidden layer containing 50 neurons, a fully connected output layer; an Adam optimizer with a learning rate of 0.001 was used to prevent overfitting during model training. The following were also used: Dropout rate (e.g., 0.2), weight decay coefficient (L2 = 1 × 10⁻⁴); activation function selected—ReLU; Mean squared error (MSE) loss function, 100 epochs; and a batch size of 64. Meanwhile, set an Early Stopping strategy so that when the validation loss does not decrease for 10 consecutive epochs, training is automatically terminated to avoid ineffective training and overfitting.

For the support vector regression (SVR) model, the kernel type is the most critical parameter, as it maps input data into a high-dimensional feature space to handle nonlinear relationships. The kernel bandwidth (gamma) controls the influence range of individual samples: a large gamma may cause overfitting, whereas a small gamma produces a smoother kernel, making the model more sensitive to overall trends. The penalty parameter (C) regulates the tolerance for training errors; a large C can lead to overfitting, while a small C may result in underfitting. In this study, the SVR model was configured with a radial basis function (RBF) kernel, C set to 100, and gamma set to 0.1. The model parameters are shown in

Table 2. Model parameter Settings.

Model	Parameter
	Units	epoch	batch_size
GRU	50	100	64
LSTM	50	100	64
PeepholeLSTM	50	100	64
SVR	Kernel	c	gamma
	RBF	100	0.1

.

Data from the Guangzhou and Shenzhen stations were separately input into the SVR, LSTM, GRU, PeepholeLSTM, and PeepholeLSTM–attention models for training and ${\mathrm{PM}}_{2.5}$ concentration prediction. The first 80% of the data were used for model training to capture temporal patterns and trends, while the remaining 20% were reserved as the test set to evaluate model performance. A time step of 12 was applied, using the previous 12 h to predict the ${\mathrm{PM}}_{2.5}$ concentration in the 13th hour. The fitting between the predicted and observed values for both stations is shown in Figure 7. Due to the large number of hourly data points, the comparison plot is not visually clear; therefore, 24 h averaged ${\mathrm{PM}}_{2.5}$ concentrations were calculated to generate a daily comparison plot. To further illustrate differences among models, enlarged plots of six representative weeks were additionally provided.

Figure 7 visualizes the predictive performance of different models by plotting the actual against the predicted ${\mathrm{PM}}_{2.5}$ concentrations on the test sets. The black curve represents observed values, while other colors indicate predictions from SVR, LSTM, GRU, PeepholeLSTM, and PeepholeLSTM–attention models. Overall, predictions from all five models approximately match the observed values for both Guangzhou and Shenzhen stations, capturing the general trends of ${\mathrm{PM}}_{2.5}$ variations. However, the SVR model exhibits noticeably poorer performance compared with the deep learning models. Although SVR captures the overall trend, its predictions at peak ${\mathrm{PM}}_{2.5}$ values often deviate substantially from observations and may even show trends opposite to the actual data. The LSTM, GRU, and PeepholeLSTM models demonstrate similar performance, effectively reflecting the true trends and outperforming SVR. Nevertheless, these models still show some discrepancies at peak ${\mathrm{PM}}_{2.5}$ values. Introducing the attention mechanism into PeepholeLSTM model (PeepholeLSTM–attention) leads to a further improvement in predictive performance.

Although the combined PeepholeLSTM–attention model improved predictive performance, peak ${\mathrm{PM}}_{2.5}$ values were still not well captured. Therefore, OVMD was applied to decompose the ${\mathrm{PM}}_{2.5}$ time series, and the resulting IMF components were predicted using the PeepholeLSTM–attention model. The final prediction of ${\mathrm{PM}}_{2.5}$ concentrations was obtained by reconstructing the predicted results of the IMFs.

OVMD was first applied to the Guangzhou station data. As shown in Figure 8, when the number of modes K reached 7, the center frequencies of IMFs for both Guangzhou and Shenzhen stations became stable, and further increases in K did not lead to significant changes. Thus, K = 7 was selected as the optimal decomposition number.

The optimal update step $\tau$ was determined by minimizing the REI value. Let $\tau$ traverse the interval [0, 1] for a search, with the update step set to 0.01. For each candidate value of $\tau$, calculate the Residual Evaluation Index (REI). By comparing the REI values for different $\tau$ values across the entire interval, finally select the $\tau$ that minimizes the REI as the optimal update step for OVMD decomposition. This ensures the decomposition process is efficient and the decomposition results are accurate, providing high-quality feature inputs for subsequent model training. As shown in Figure 9, the REI reached its minimum at $\tau$ = 0.85 for Guangzhou and $\tau$ = 0.88 for Shenzhen.

Accordingly, for both stations, OVMD was set to K = 7, with $\tau$ = 0.85 for Guangzhou, and $\tau$ = 0.88 for Shenzhen. The final decomposition results are shown in Figure 10. As can be seen from the figure, whether at Guangzhou Station or Shenzhen Station, the waveform fluctuations of the IMF1 component are the most dense and significantly higher than those of the subsequent components, which is a typical high-frequency fluctuation characteristic. This component has the fastest amplitude changes and the highest frequency, reflecting the short-term, rapid random fluctuations and high-frequency variations caused by sudden pollution events in the ${\mathrm{PM}}_{2.5}$ data. The waveform fluctuations of the IMF7 component are minimal, with overall changes being gentle, showing the long-term variation trend of ${\mathrm{PM}}_{2.5}$ concentration, which is a typical low-frequency trend characteristic. The fluctuation frequencies of the other components lie between high-frequency and trend components, containing certain short-term fluctuations as well as some trend changes, making them transitional mid-frequency components.

After OVMD decomposition, seven IMF components were obtained for each station, and each was individually predicted by sequentially inputting into the PeepholeLSTM–attention model with a time step of 12. The results of ${\mathrm{PM}}_{2.5}$ concentration prediction for the two sites were greatly improved, especially in terms of capturing the actual trends and peak positions more accurately. The prediction results of each IMF component are shown in Figure 11. The results indicate that the model has fully grasped the characteristic patterns of each IMF component (including the long-term stable trend of low-frequency components and the short-term fluctuation features of high-frequency components), and is able to accurately capture the variation details of each component. This enables precise prediction of the actual change trend and peak positions of ${\mathrm{PM}}_{2.5}$ concentrations, and also validates the rationality and effectiveness of combining OVMD decomposition with the Peephole LSTM–attention model.

To further reveal the mechanism of how the attention mechanism functions in predicting each IMF component and to clarify the model’s focus on features at different time steps of the input sequence, this study also visualizes the attention weight heatmaps during the prediction process of each modal component. Since high-frequency IMFs directly reflect the core fluctuation patterns of pollution events, accurately capturing their changes is key to improving prediction accuracy. Therefore, this paper only presents the heatmap of attention weight distribution for the high-frequency IMF1 component, which exhibits strong predictive performance (Figure 12). Analysis shows that the model demonstrates significantly differentiated attention characteristics for input time step features of different modal components; notably, the IMF components have the highest weights compared to the other 11 features, and this attention pattern aligns closely with the physical significance of each IMF component. For pollutants like ${\mathrm{PM}}_{2.5}$, which have strong temporal inertia, the model inherently assigns higher attention weights to time steps closer to the prediction moment, while weights for more distant time steps decay rapidly. Combined with historical ${\mathrm{PM}}_{2.5}$ data, when sudden changes in ${\mathrm{PM}}_{2.5}$ concentration occur (such as unexpected heavy pollution or sudden drops in concentration), the attention weights exhibit noticeable ‘abnormal peak shifts’.

The reconstructed ${\mathrm{PM}}_{2.5}$ concentration predictions obtained by combining all IMF components are presented in Figure 13. The results demonstrate that a superior overall predictive performance for the PeepholeLSTM-OA model closely approximates the observed data at both stations, achieving accurate fit for abrupt changes, as well as peak and trough values, thereby achieving accurate prediction of ${\mathrm{PM}}_{2.5}$ concentration variations.

To comprehensively assess model performance in ${\mathrm{PM}}_{2.5}$ concentration prediction, we not only visualized prediction outputs to gauge their ability to track concentration fluctuations but also employed evaluation metrics to quantify the true effectiveness of these predictions. These metrics provide a more direct reflection of model stability, sensitivity, and fitting effectiveness. The MAE, RMSE, and $R^2$ for each model are presented in

Table 3. Indicators of Experimental Models at Two Sites.

Model	Guangzhou			Shenzhen
	MAE	RMSE	${\mathit{R}}^{\mathbf{2}}$	MAE	RMSE	${\mathit{R}}^{\mathbf{2}}$
SVR	5.954	7.601	0.6600	4.826	6.349	0.6839
GRU	2.420	3.760	0.9312	2.587	4.096	0.9009
LSTM	2.535	3.863	0.9274	2.547	4.014	0.8964
PeepholeLSTM	2.368	3.705	0.9332	2.464	3.991	0.9024
PeepholeLSTM–attention	2.367	3.676	0.9343	2.454	3.952	0.9034
PeepholeLSTM-OA	1.442	2.025	0.9800	1.330	1.915	0.9789

. The results consistently indicated the superior performance of deep learning approaches (LSTM, GRU, and PeepholeLSTM) relative to SVR for the Shenzhen case, and consistent with results at Guangzhou, indicating that deep learning approaches are more suitable for long time series prediction tasks compared to traditional machine learning methods. Among the individual models, GRU achieved results comparable to LSTM, while PeepholeLSTM exhibited the best performance, a phenomenon consistently observed at both Guangzhou and Shenzhen sites. When the attention mechanism was integrated into the PeepholeLSTM, the combined model (PeepholeLSTM–attention) demonstrated further improvements in all three metrics, though the performance gains were marginal. Finally, with the introduction of the OVMD decomposition algorithm, the PeepholeLSTM-OA model achieved significant improvements over the single PeepholeLSTM. At the Guangzhou site, the MAE decreased by about 39%, the RMSE decreased by about 45%, and the $R^2$ increased by 0.0457; at the Shenzhen site, the MAE decreased by about 45%, the RMSE decreased by about 51%, and the $R^2$ increased by 0.0765. These results indicate that the stability, sensitivity to large errors, and fitting capability of the model were substantially enhanced.

In order to gain a more intuitive understanding of the prediction error distribution of the PeepholeLSTM-OA model, this study plotted the original error distribution chart (Figure 14). According to the statistics, the error range for the Guangzhou station is [−11.56, 7.60] μg/m³, with a standard deviation of 1.675 μg/m³ and a mean of −0.791 μg/m³; for the Shenzhen station, the error range is [−14.41, 12.66] μg/m³, with a standard deviation of 1.413 μg/m³ and a mean of −0.105 μg/m³. From this, it can be concluded that the model has excellent ${\mathrm{PM}}_{2.5}$ concentration prediction performance.

In summary, the OVMD–PeepholeLSTM–attention model proposed in this paper performs excellently in predicting ${\mathrm{PM}}_{2.5}$ concentrations at two sites. All evaluation metrics surpass those of the comparison models, particularly demonstrating significant advantages in capturing sudden concentration changes, peak positions, and long-term trends, thereby validating the rationality and effectiveness of the model’s design.

To further verify the authenticity of this excellent performance metric and the generalization ability of the model, and to eliminate the risk of overfitting, this study conducted K-fold cross-validation experiments. A 5-fold cross-validation strategy was used, where the original training set was randomly divided into five equally sized subsets. Each time, four subsets were selected as the training set and one subset as the validation set. The training and validation process was repeated five times, and the evaluation metrics of the five validation results were calculated to assess the stability of the model and eliminate the random errors caused by a single training session. The experimental results are shown in

Table 4. K-fold cross-validation results.

	Guangzhou			Shenzhen
	MAE	RMSE	${\mathit{R}}^{\mathbf{2}}$	MAE	RMSE	${\mathit{R}}^{\mathbf{2}}$
First fold	1.232	1.768	0.9773	1.226	1.703	0.9548
Second fold	1.766	2.293	0.9615	1.233	1.638	0.9725
Third fold	1.675	2.266	0.9490	2.418	3.574	0.9612
Fourth fold	1.874	2.574	0.9607	1.398	2.073	0.9738
Fifth fold	1.829	2.504	0.9600	2.006	2.724	0.9674

. The results of the 5-fold cross-validation show that the average $R^2$ value of the model across five validations is higher than 0.94, and the average MAE and RMSE values show little difference from the single training results, with no significant fluctuations, indicating that the model has good stability and that random errors from a single training session have been ruled out.

The analysis of the above experiments indicates that the prediction results of the PeepholeLSTM-OA model are highly reliable. It performs excellently in terms of stability, sensitivity to anomalous data, and the ability to uncover data patterns, making it practically meaningful for predicting changes in ${\mathrm{PM}}_{2.5}$ concentrations.

5. Conclusions

This study proposed the PeepholeLSTM-OA model for predicting the hourly ${\mathrm{PM}}_{2.5}$ concentrations in Guangzhou and Shenzhen, China, from 2020 to 2022, demonstrating superior performance compared with individual models. The key findings are summarized as follows.

(a)

Instance validation shows that the PeepholeLSTM-OA model outperforms the SVR, GRU, LSTM, PeepholeLSTM, and PeepholeLSTM–attention models across all evaluation metrics. The predicted results accurately reflect the true trends of the measured values and also show good performance in handling abrupt numerical changes.

(b)

In Guangzhou, the PeepholeLSTM-OA model achieves MAE, RMSE, and $R^2$ values of 1.442, 2.025, and 0.9800, respectively. Compared with the single PeepholeLSTM model, MAE decreases by approximately 39%, RMSE decreases by about 45%, and $R^2$ improves by 0.0457. In Shenzhen, the model achieves MAE, RMSE, and $R^2$ values of 1.330, 1.330, and 0.9789, respectively, with MAE reduced by approximately 45%, RMSE reduced by 51%, and $R^2$ improved by 0.0765 compared with the PeepholeLSTM model.

(c)

The PeepholeLSTM-OA model has excellent ${\mathrm{PM}}_{2.5}$ concentration prediction performance. This method can be integrated into global air quality monitoring systems and has broad applicability and potential in general scientific fields, providing technical support for building a sustainable atmospheric environment monitoring system. The model can provide data support for short-term, real-time precise management of regional air pollution, assist in early warning of short-term pollution peaks, and accurately capture the long-term evolution trend of ${\mathrm{PM}}_{2.5}$ concentrations through multi-step predictions. It provides a scientific basis for formulating sustainable policies for air pollution prevention and control, and has important practical value in promoting normalized and sustainable management of air pollution.

Future research could integrate attention mechanisms with reinforcement learning and leverage the predictive capabilities of the PeepholeLSTM-OA model to conduct intelligent decision-making studies related to pollution control. Based on this integrated framework, it is possible to further achieve unified decision-making for ’prediction-analysis-control’ of air pollution, contributing to the sustainable development of the atmospheric ecological environment.