Airborne Particulate Matter Modeling: A Comparison of Three Methods Using a Topology Performance Approach

Abstract

Understanding the behavior of suspended pollutants in the atmosphere has become of paramount importance to determine air quality. For this purpose, a variety of simulation software packages and a large number of algorithms have been used. Among these techniques, recurrent deep neural networks (RNN) have been used lately. These are capable of learning to imitate the chaotic behavior of a set of continuous data over time. In the present work, the results obtained from implementing three different RNNs working with the same structure are compared. These RNNs are long-short term memory network (LSTM), a recurrent gated unit (GRU), and the Elman network, taking as a case study the records of particulate matter ${\mathrm{PM}}_{10}$ and ${\mathrm{PM}}_{2.5}$ from 2005 to 2019 of Mexico City, obtained from the Red Automatica de Monitoreo Ambiental (RAMA) database. The results were compared for these three topologies in execution time, root mean square error (RMSE), and correlation coefficient (CC) metrics.

1. Introduction

Air quality is one of the main factors determining ecological and human comfort in a given geographic region [1,2,3]. For this, it has to be considered that air is made up of a mixture of various gases, vapors, and dust, whose origins are both natural and anthropogenic [4,5]. Growing industrial and residential development causes increases to various atmospheric pollutants (anthropogenic materials), leading to a decrease in air quality. The World Health Organization (WHO) conducts several studies periodically to assess air pollution, among other interest factors. For instance, a 2016 study estimated that more than 91% of the world population lives in places with poor air quality [6].

The behavior of air pollutants varies, as they are affected by various climatic and geographic variables [1,7,8,9], in this paper we will limit ourselves to the study of particulate matter (PM), which is a classification of pollutants that have begun to generate alerts in growing and overpopulated cities because of their chaotic behavior, which is complicated to model [6,10]. Algorithms such as artificial neural networks (ANN) have demonstrated their versatility in modeling the chaotic behavior of several variables, allowing the prediction of future behaviors with great adaptability to changes in the analysis data [11,12,13,14,15,16,17,18]. Each algorithm exhibits different characteristics. Their responses may vary despite using the same set of data.

1.1. Particulate Matter

Particulate matter is a sub-classification of air pollutants, which has different definitions depending on the aerospace diameter of the particles that compose it without considering their origin [4]. Based on this definition, the acronym PM refers to particulate matter, and the numbers below correspond to the maximum size of each classification [4,19].

Table 1. Classification of particulate matter [4].

Representation	Particle Size	Designation
${\mathrm{PM}}_{10}$	≤10 µm	Coarse material
${\mathrm{PM}}_{2.5}$	≤2.5 µm	Fine material

shows the size corresponding to each classification and the attributed name.

The main conditions caused by overexposure to PM include an aggravation of asthma and increased mortality from cardiovascular disease [7], because these particles are deposited in the pulmonary alveoli [20,21,22]. The problem is to recognize an overexposure to this type of pollutant. To do so, the first step is to generate a model that describes the chaotic behavior of PM material that can adapt to changes, with the objective being that future analysis can determine if conditions that exceed the established limits of pollutants are present.

To monitor PM concentrations, other pollutants and weather conditions, Mexico has several monitoring networks, including the Red Automatica de Monitoreo Ambiental (RAMA) in Mexico City and its surrounding areas. This network generates a database of continuous records with a determined frequency. The frequency of records is constant over time and can be referred to as a time series [11,12].

1.2. Deep Neural Network

Deep neural networks are the main technological architecture used in deep learning models. They are widely used in different fields of research, such as natural language processing [23], visual and auditory pattern recognition [23,24], signal processing, and time series processing, among others [15,25,26,27,28].

Among the current widely used algorithms for time-series modeling are recurrent neural networks (RNN), which are a classification of artificial neural networks (ANN) [29,30,31]. ANNs are a computational model inspired by biological neurons [28,32]. Warren McCulloch and Walter Pitts in 1943 were pioneers in proposing the computer model of an artificial neuron, called the logical threshold [31,33].

The first artificial neural networks were made up of simple neurons called perceptrons, as proposed by Rosenblatt in 1959 [28,31], which operate as a binary classifier (0 or 1) by means of Equation (1) [18].

$$f\left(x\right)=\{\begin{array}{cc}1& if w·x-u>0\\ 0& all other cases\end{array}$$

(1)

where w is a vector of real weights, $w·x$ is the scalar product, ans u is the threshold, which represents the degree of inhibition of the neuron. The value of $f\left(x\right)$ (0 or 1) is used to classify x in a binary classification problem [28].

Later, different network topologies emerged, such as convolutional neural networks (CNN) and recurrent neural networks (RNN), each with its own performance characteristics and therefore with different resulting parameters [28,33]. Lately, RNN topologies have been widely used to model chaotic behavior in time series [6,12,34].

The RNNs compared in this manuscript were the Elman network, the gate recurrent unit (GRU), and long short-term memory (LSTM) networks. Section 2, Section 3 and Section 4 describe their operation and main characteristics, respectively. Section 5 will explain the methodology implemented. In Section 6 the results obtained will be shown and a brief discussion will be conducted. Finally, the conclusions will be presented.

2. Elman Recurrent Neural Network

Also known as simple recurrent networks (SRN) [13]. An Elman recurrent neural networks is a two-layer backpropagation network, with feedback from the output of the hidden layer to the input of the network (Figure 1), which allows the network to detect time-varying patterns. The feedback contains a delay, allowing the network to retain values from the previous step for use in the current step, thus allowing the network to learn nonlinear patterns and predict their behavior. Equations (2) and (3) describe the basic operation of the Elman network [13,27,35].

Figure 1. Elman recurrent network base diagram [36].

Figure 1. Elman recurrent network base diagram [36].

$$h_t={\sigma }_h\left(W_h{x}_t+U_h{h}_{t-1}+b_h\right)$$

(2)

$$y_t={\sigma }_y\left(W_y{h}_t+b_y\right)$$

(3)

where

• $x_t$: input vector.
• $h_t$: hidden layer vector.
• $y_t$: output vector.
• W, U and b: parameter matrices and vector.
• ${\sigma }_h$ and ${\sigma }_y$: activation functions.

Elman networks with feedback can learn to recognize chaotic behaviors more quickly than a simple neural network, but at the cost of increased processing time [35]. Despite increasing the time required, it is smaller in comparison to that required by other recurrent networks. Widely used in image recognition (signature recognition, element detection in photographs), a time series in which the non-linearity of the problem is a secondary factor and processing time are crucial to solving the problem [27].

3. LSTM Recurrent Neural Network

The LSTM recurrent neural network was proposed by Hochreiter and Schmidhuber (1997) [6,37]. Conventional recurrent neural networks present problems in training because backpropagated gradients tend to grow large or fade with time. After all, the gradient depends not only on the present error but also on past errors. The accumulation of errors causes difficulties in memorizing long-term dependencies [38].

These problems are solved by LSTM networks, which incorporate a series of steps to decide which information is to be stored and which is to be erased with three gates that control the way information flows in and out of the unit [37], as shown in Figure 2.

• Input gates control when new information can enter the memory.
• Forget gates control when part of the information is forgotten, allowing the cell to discriminate between important and excessive data, thus leaving room for new data.
• Output gates control when it is used as the result of the memories in the cell.

The cell has a weighting optimization mechanism based on the resulting network output error, which controls each gate. This mechanism is usually implemented with the training algorithm, BPTT. Equations (4)–(8) describe the basic functioning of LSTM neurons.

$$i_t=\sigma \left(W| |xi\ast x_t+W_{hi}\ast h_{t-1}+W_{ci}\ast C_{t-1}+b_i\right)$$

(4)

$$f_t=\sigma \left(W| |xf\ast x_t+W_{hf}\ast h_{t-1}+W_{cf}\ast C_{t-1}+b_f\right)$$

(5)

$$c_t=i_t\ast tanh\left(W_{xc}\ast x_t+W_{hc}\ast h_{t-1}+b_c\right)+f_t\ast c_{t-1}$$

(6)

$$o_t=\sigma \left(W| |x0\ast x_t+W_{h0}\ast h_{h-1}+W_{c0}\ast C_t+b_0\right)$$

(7)

$$h_t=o_t\ast tanh\left(c_t\right)$$

(8)

where

• $x_t$: input vector.
• $f_t$: forget gate activation vector.
• $i_t$: input/update gate activation vector.
• W: matrix weights implemented.
• $o_t$: hidden state vector, better known as the LSTM unit output vector.

4. GRU Recurrent Neural Network

The GRU recurrent neural network was introduced in 2014 by Cho et al. [10,39]. A GRU neuron is similar to the LSTM neuron but lacks an output gate, thus maintaining two gates: an update gate and a reset gate, as seen in Figure 3 [10]. The update gate indicates how much of the previous cell contents to keep, and the reset gate defines how to incorporate the new input with the earlier cell contents [16,39].

Mainly used in polyphonic music modeling, speech modeling, and natural language processing, the GRU performs best on sets with a small number of data [16].

The Equations (9)–(12) describe the operation of a GRU neuron. Initializing t = 0 and the output vector $h_0$ = 0 [39].

$$z_t={\sigma }_g\left(W_x{x}_t+U_z{h}_{t-1}+b_z\right)$$

(9)

$$r_t={\sigma }_g\left(W_r{X}_t+U_r{h}_{t-1}+b_r\right)$$

(10)

$${\widehat{h}}_t={\varphi }_h\left(W_h{x}_t+U_h\left(r_t\ast h_{t-1}\right)+b_h\right)$$

(11)

$$h_t=\left(1-z| |t\right)\ast h_{t-1}+z_t\ast {\widehat{h}}_t$$

(12)

where

• $x_t$: input vector.
• $h_t$: output vector.
• ${\widehat{h}}_t$: candidate gate vector.
• $r_t$: reset gate vector.
• $z_t$: update gate vector.
• W, U, and b: parameter matrices and vector.
  Activation functions
• ${\sigma }_g$: sigmoid function.
• ${\varphi }_h$: hyperbolic tangent.

5. Case Study

In this section, the geographical area studied, characteristics of the database used, and the hardware characteristics for the analysis are explained.

5.1. Study Area

The study area corresponded to Mexico City and its surroundings (Mexico), where the RAMA network is located. Figure 4 shows the location of the monitoring stations [40,41]. The stations whose records are analyzed in this work, which are: TLA, MER, and XAL, belonging to the delegations of Tlalnepantla de Baz (State of Mexico), Venustiano Carranza (CDMX), and Ecatepec de Morelos (Mexico State), respectively [40], are highlighted in red.

5.2. PM Data

RAMA allows free access to its online database, which contains records of different pollutants since 1986 to date. At present, RAMA has 24 monitoring stations whose records have the following characteristics: they are daily records measured at a frequency of one hour with positive numerical values and records of atypical data denoted with a −99, which indicates that the sensor did not perform its correct operation for some external reason (malfunction, maintenance, etc.) [41]. In this study, the records in the interval from 2005 to 2019 of the particulate matter ${\mathrm{PM}}_{10}$ and ${\mathrm{PM}}_{2.5}$ were processed, which means that 131,472 data points were processed per monitoring station.

The behavior of PM is highly nonlinear, as shown in Figure 5, which shows the behavior of ${\mathrm{PM}}_{2.5}$ and ${\mathrm{PM}}_{10}$ for the first 16 days of January, 2019. Both records have a discontinuous interval, which represents no accurate record at that precise moment, and the missing values had to be proposed and interpolated to maintain continuity (section highlighted in orange).

5.3. Hardware and Software Used

The present work was performed on a personal computer with 6 GB RAM, an NVIDIA GEFORCE GTX graphics card, and an Intel CORE I7 7th Gen processor, using PYTHON 3.6 software in conjunction with the KERAS library for deep networks.

6. Proposed Methods

The following steps were performed for the implementation of the three different deep networks, using the methodology shown in Figure 6.

6.1. Preprocessing Data

Initially, the stations with the least amount of outlier data (measurement losses, disconnection due to maintenance, failures in the complete system, etc. [41]) were identified for both in the ${\mathrm{PM}}_{10}$ and ${\mathrm{PM}}_{2.5}$ records, so that the different recurrent neural network architectures evaluated only the enormous amount of data representing the actual behavior of particulate matter.

Table 2. Percentages of outliers.

${\mathrm{PM}}_{10}$		${\mathrm{PM}}_{2.5}$
MER	TLA	MER	TLA
11.02%	13.41%	13.33%	14.73%

shows the stations with the smallest percentages of outlier data between 2005 to 2019, which were located in the Venustiano Carranza (MER) and Tlalnepantla de Baz (TLA) delegations, being a commercial zone and an industrial zone, with 131,472 records existing from each station, respectively [40].

The next step was to replace these records with approximate values to have continuity in the data. For this purpose, the multiple imputation by chained equations (MICE) algorithm was implemented to identify the values before and after the outliers.

MICE makes a correlation between the valid data, performing an iterative series of predictive models. Each variable specified in the data set was imputed in each iteration using the other variables in the data set. In other words, the basic idea is to treat each variable with missing values as the dependent variable in a regression, with some or all of the remaining variables as predictors. This technique uses a procedure called predictive mean matching (PMM) to generate random draws from the predictive distributions determined by the fitted models. These lucky draws become the imputed values while maintaining behavior similar to the valid data [42].

6.2. Construction Model

The three RNNs were evaluated with the same hyperparameter, architecture, training data, and validation conditions. The architecture assessed consisted of three layers, an input layer, a hidden layer, and an output layer, which are described below [6]. Each neuron of the three different layers was configured depending on the evaluated network type, which varied among Elman, GRU, and LSTM networks.

• Input layer: 50 neurons.
• Hidden layer: 250 neurons.
• Output Layer: 1 neuron.

In

Table 3. Hiperparameters used in this contribution.

Hyperparameters	Changes
Bach size	96,128,256,512
Optimizer	Adam, Adamax
Number of epochs	12

, the number of hyperparameters to vary during each test are listed. This generated a total of 16 individual runs, which were each repeated ten times to determine the variability of the results obtained for both the ${\mathrm{PM}}_{10}$ and ${\mathrm{PM}}_{2.5}$ particle sizes.

Hyperparameters refer to the following features:

• Bach size. Defines the number of samples with which the network will work before modifying its internal parameters. Since the data are continuous records in a time series, it is necessary to divide them into sections. The network can recognize sections of the behavior using a smaller amount of memory than would be required to process all the records in a single iteration [43]. Another point of interest is the execution time, which with a small batch size tends to increase [44]. It is important to find a balance point between the execution time and the memory used; in other words, with a small batch size, the execution time increases but the error obtained is lower when working with a large batch size and consequently a shorter execution time. The batch size to be used depends on the amount of data used during the training of the network, so that the network recognizes sections continuously and is able to model the continuous behavior to predict a future behavior [43,44,45]. Essentially, the batch size allows the network to divide the problem into sections whose output variables are compared with the expected variables, obtaining an error. From this error, the network is updated and the model is improved [46]. Most of the values used corresponded to the parameter of $2^n$, as shown in the literature [18]. In contrast, 96 corresponds to the multiple of 12 h represented for the prediction of exceedances exposed in [6,10].
• Optimizer. Methods by which the algorithms are used to readjust their internal parameters, thus improving the learning of the implemented algorithm. For the present work, we used the optimizers Adam and Adamax, which are widely used with time series data, because they employ a stochastic gradient method based on the adaptive estimation of first and second-order moments [15,31,47].
• Number of epochs. Defines the number of times the algorithm will use the entire training data set. During each epoch, each data sample will have the opportunity to update the internal parameters of the network [31,44], in this case, 12 epochs were used, since by using a more significant number of epochs, the network evaluation error remains practically constant with only an occasional, slight variation [6].

6.3. Model Evaluation

With its purpose of evaluating the robustness of an algorithm applied in a system with numerous variables that can affect or generate perturbations in the input parameters, the total sensitivity model evaluation method was used. Todorov et al. evaluated different stochastic approaches to evaluate the sensitivity indexes, allowing a comparison to be made between the input parameters with respect to their influence on the points of interest [48]. To evaluate the results of the time series analysis we used the root mean square error (RMSE) [49] and the correlation coefficient (CC) [50]. The variable “M” represents the values of the records modeled by the recurrent networks (Elman, LSTM, and GRU), the variable “R” is the actual records, and “n” the number of total data.

RMSE represents the difference between the data generated by the network and the actual data [49].

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(M_i-R_i\right)}^2}$$

(13)

CC generates values between −1 and 1, being the closest to one in a positive way means that the predicted values (M) are closer to the real values (R). In other words, if the CC is equal to 1, there is no difference between the modeled data and the real data, while if a negative value is obtained it means that a mirror behavior to the real data was obtained [50]. Equation (14) shows the sections corresponding to the calculation.

$$CC=\frac{S_{MR}}{\sqrt{S_M{S}_R}}$$

(14)

where:

$$S_{MR}=\frac{{\sum }_{i=1}^n\left(M_i-\overline{M}\right)\left(R_i-\overline{R}\right)}{n-1}$$

(15)

$$S_M=\frac{{\sum }_{i=1}^n\left(M_i-\overline{M}\right)}{n-1}$$

(16)

$$S_R=\frac{{\sum }_{i=1}^n\left(R_i-\overline{R}\right)}{n-1}$$

(17)

7. Results and Discussion

Comparison between the behavior of the real data (solid blue line) and the neural network model (solid orange line) for both ${\mathrm{PM}}_{10}$ and ${\mathrm{PM}}_{2.5}$ is shown in Figure 7. This comparison was made using an LSTM neural network in 2019.

Furthermore, Figure 8 and Figure 9 shows the repeatability and the range corresponding to the CC, obtained with different batch sizes (96,128,256, and 512) and optimizers (Adam and Adamax), using the three network architectures evaluated for the behavior of the ${\mathrm{PM}}_{2.5}$ and ${\mathrm{PM}}_{10}$ particulate material, respectively.

Figure 8 shows the variability of the CC when using the Adam optimizer (on the right) and the Adamax optimizer (on the left) in ${\mathrm{PM}}_{2.5}$ particle modeling using the three network architectures. Figure 8a,b correspond to the results of the Elman network architecture, where the best results shown were using the Adamax optimizer, especially using a batch size of 512, with an average CC of 0.757. In contrast, with the use of the Adam optimizer for a batch size of 96, atypical data were generated, demonstrating that the network had problems in cases where the behavior of the data was highly chaotic and that for this neural network, it is better to use the Adamax optimizer.

In the case of the use of the GRU (Figure 8c,d) and LSTM (Figure 8e,f) networks, it was observed that the behavior was very similar; batch sizes of 96 obtained the best results, regardless of the optimizer used. Highlighting the following differences, in the use of the GRU network, errors were observed when the batch sizes were greater than 256, regardless of the optimizer implemented. The high repeatability obtained by the Adamax optimizer is notable. In the case of the LSTM network in conjunction with the Adamax optimizer, for tests with batch sizes of 128 and 256, it was observed that the results obtained are consistent while maintaining high repeatability with values of approximately 0.76, compared to those obtained with the use of the Adamax optimizer.

Figure 9 shows the results obtained from the CC of the ${\mathrm{PM}}_{10}$ particle modeling. The implemented Elman architecture (Figure 9a,b) presented the least variation in the results obtained with a batch size of 512 and the Adam optimizer, with a mean of 0.79. Curiously, highest variation occurred when the Elman architecture was presented with the batch size of 96. With the Adamax optimizer, there was a greater variation with batch sizes of 128 and 256. The best results came from the batch size of 96, followed by the batch size of 512 with approximate means of 0.799 and 0.797, respectively. For Figure 9c,d, corresponding to the GRU architecture, there was a decrease in the CC with an increase to the implemented batch size, regardless of the optimizer, as is highlighted in the greater compactness of the results with batch sizes of 256 and 512 with the Adam optimizer, with mean CC values of 0.795 and 0.794, respectively. In the case of the results with the Adamax optimizer a constant reduction was shown as the batch size increased, maintaining a practically constant amplitude of 0.002. Finally, items Figure 9e,f, corresponding to the LSTM architecture, show that with batch sizes of 96, 128, and 256, they had average CC values of 0.79 for both optimizers, with greater compactness observed with the Adam optimizer, while for a batch size of 512 there was a considerable difference. In the LSTM architecture, the optimizer does not show a representative change for this application.

In Figure 10 and Figure 11, the variability obtained from the RMSE is shown. These Figures show greater variability in the results obtained compared to those obtained from the correlation coefficient. In Figure 8a and Figure 10a, it is observed that, for the Elman network, there was an outlier point when using the Adam optimizer and a batch size of 96, indicating that when the Elman network was modeling the behavior of ${\mathrm{PM}}_{2.5}$ many of the results were below the expected performance. Conversely, with the rest of the batch sizes evaluated, high repeatability of approximately 9.9 was maintained with a 512 batch size, showing the best possible results for this topology.

Regarding the Adamax optimizer (Figure 10b and Figure 11b) with batch sizes of 256 and 512 there was greater variability in the results obtained, and an average of 10.25 was maintained in both cases. With batch sizes set to 96 and 128, a similar consistency was observed, with a mean of approximately 9.6. In Figure 10c,d and Figure 11c,d, corresponding to the GRU architecture, the results present a higher variability, which is more visible with batch sizes of 256 and 512 for both optimizers. With batch sizes of 96 and 128, the Adamax optimizer showed higher repeatability in the results than the Adam optimizer. Finally, Figure 10e,f and Figure 11e,f correspond to the implementation of the LSTM topology. This network presented different behavior between them; in the case of the application of the Adam optimizer there was a greater variation in the results with batch size of 256, with an average of 10.4, but this variation decreased with batch size of 512, obtaining an average of 9.9.

In contrast, with batch sizes of 128 and 96, LTSM topology presented a similar behavior. In the case of the Adamax optimizer, a more significant variation is found with a batch size of 512 and an approximate mean of 10.7 was obtained. For batch sizes of 256 and 96, similar results were obtained, highlighting an outlier for batch size of 256, with a standard for both of them of 9.6. Finally, the 128 batch size showed a low variability of the results obtained after the ten tests performed.

Figure 11a,b corresponds to the results of the Elman network implementation, which produced a wide variety of possible results with the application of the Adam optimizer with batch sizes of 96, while larger batch sizes showed higher repeatability and lower variance. However, for all three cases, outliers are shown which seem larger than the general average. Despite the high repetitiveness of the results, the Elman network seemed to have complications modeling the behavior of the particulate matter ${\mathrm{PM}}_{10}$ using the Adam optimizer, in the sense that the variance was higher. In the case of the Adamax optimizer implementation, the results show a comparative improvement, with high repetitiveness in the 512 batch size, with a mean of 16.6. In the case of smaller batch sizes, the mean improved as the batch size was reduced, albeit with higher variability.

In Figure 11c,d the GRU network showed different behavior for each optimizer, in which the optimizer Adam and the batch sizes of 96, 256, and 512 presented a variation in their results of 0.6 in the RMSE. For the case of the batch size of 128, there was greater repeatability of the results, but with two outliers, one at each side of the boxplot. Using the Adamax optimizer for batch sizes of 96 and 128, there was a variation in the results of 0.2 with a mean of 16.3, in contrast to the variation present with higher batch sizes.

Finally, Figure 11e,f indicate that, in the LSTM network, the expected results was observed. This means that the RMSE increased with larger batch sizes. Also, there is a difference between using the Adam and the Adamax optimizers, especially in the results of the batch size of 96. The Adam optimizer obtained results with a mean of 16.1, with a single outlier. In the case of the batch size of 128, the same low variance was observed, but without the outlier. With the Adamax optimizer, the results with greater repetitiveness were presented with the batch size of 256, for the batch size of 128 it had a mean of 16.25 with a variance of 0.5, which was greater than that present in the batch size of 96 maintaining a similar mean.

Finally, Figure 12 and Figure 13 show the behaviors of the execution times of each of the tests in the modeling of ${\mathrm{PM}}_{2.5}$ and ${\mathrm{PM}}_{10}$ particulate matter, respectively. In both Figures, the same behavior is observed regardless of the optimizer implemented. In other words, the execution time was higher as the batch size decreased. This was especially so with a batch size of 96, regardless of the topology implemented. However, the execution times in the modeling of the ${\mathrm{PM}}_{2.5}$ particles were higher with the Adamax optimizer, with the slowest execution being that of the LSTM network, followed by the GRU network. The Elman network was the fastest.

For the ${\mathrm{PM}}_{10}$ particle modeling, Figure 13 indicates that the behavior recorded was practically the same, regardless of the optimizer implemented. This is also consistent with previous results in which the Elman network was the fastest topology, followed by the GRU network and the LSTM network.

Summarizing the above,

Table 4. Best hyperparameters for ${\mathrm{PM}}_{2.5}$ particle modeling.

Neural Network	Optimizer	Batch Size	CC	RMSE	Times(s)
Elman	Adam	512	0.7573	9.6801	224.16
GRU	Adamax	96	0.7588	9.6332	4421.91
LSTM	Adamax	128	0.7575	9.6449	2056.16

and

Table 5. Best hyperparameters for PM₁₀ particle modeling.

Neural Network	Optimizer	Batch Size	CC	RMSE	Times(s)
Elman	Adam	512	0.7977	16.6869	256.72
GRU	Adam	96	0.8004	16.2505	2544.90
LSTM	Adamax	128	0.7992	16.2492	2536.40

show the best combination of hyperparameters for each network in the modeling of ${\mathrm{PM}}_{2.5}$ and ${\mathrm{PM}}_{10}$ particles, respectively, indicating the average of the corresponding metrics obtained.

8. Conclusions

Evaluating three different recurrent topologies implementing a similar base architecture for ${\mathrm{PM}}_{2.5}$ and ${\mathrm{PM}}_{10}$ particle modeling it was observed that factors such as the optimizer used affect the results obtained. Yhe Adam optimizer showed the best results in general. At the same time, it showed greater variability in particular instances when implemented in an Elman network, in contrast with the Adamax optimizer, which maintained a lower variability in the results obtained, only having the problem of the execution time, which for a batch size of 96 was close to double that of the Adam optimizer. Mentioning the batch size, it was observed that the best results in terms of repeatability were those obtained with a batch size of 256, and in terms of the final results, comparing the CC and RMSE shows that the appropriate batch size was 128, both producing the mean values in this set of tests. The evaluated recurrent networks present great capacity for modeling the behavior of nonlinear elements such as particulate matter. Through the CC, it was determined that there was no over-training, the evaluated records from different stations maintaining repeatability in the results demonstrates the robustness of the topologies presented. Furthermore, execution times are an important factor to consider. Producing similar results obtained with different execution times, helps one determine which RNN structure is appropriate for a defined task. Considering the RMSE and CC results together with the execution time, the best network topology implemented overall used GRU neurons, but if fast results are required, the Elman network showed a higher processing capacity in a reduced time. Its only drawback is the existence of problems in in determining chaotic patterns, which must be investigated further, but if there is a large quantity of data and the execution time is not an issue to be tackled, the LSTM network is a great option for maintaining a low variability and high repeatability in the results obtained.