# Detection of Changes in Ground-Level Ozone Concentrations via Entropy

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. The Methodology

_{i,t}, i = 1,…, N; t = 1,…, T, be the ozone concentration data collected in T days from N monitoring stations. In general, X

_{i,t}are not normally distributed or even approximately normally distributed. To tackle this problem, we can first transform the data by applying the Box–Cox power transformation,with the parameter λ:

_{i,t}(λ), t = 1, …, T, for each fixed i, it is assumed that Z

_{i,t}(λ), t = 1,…,T, is an autoregressive time series with period 2L. Thus, to model the data, we employ the Fourier series expansion to reflect its periodic properties, while using the autoregressive formulation to describe its autocorrelation structure as follows:

_{i}

_{,0}(λ), a

_{i},

_{j}(λ), b

_{i},

_{j}(λ), j = 1,…, p, c

_{i,k}(λ), k = 1,…, q are unknown regression coefficients, p is the order of the truncated Fourier series, q is the lag order of the autoregressive representation and ε

_{i,t}(λ), t = 1,…, N, are random errors.

_{i,t}(λ)}, which should be allowed to vary in space and time. To tackle this problem, we can borrow the strength of a simultaneous autoregressive (SAR) model, which is often used in spatial statistics for modelling the spatial correlation of quantities of interest in a region and the regression relation between quantities of interest and explanatory variables. The parameter estimation for a SAR model can be given by employing the maximum likelihood method [18] or a Bayesian method [19]. Put ε

_{·t}= (ε

_{1,}

_{t},…, ε

_{N},

_{t})′. We model {ε

_{·}

_{t}} by the following SAR model:

_{N}is an N × N identity matrix, {ρ

_{t}} are spatial parameters, W is a weight matrix and ϵ

_{t}= (ϵ

_{1,}

_{t},…, ϵ

_{N,t})′ are independently normally distributed random errors with zero means and diagonal covariance matrix ${\sigma}_{t}^{2}{I}_{N}$. Thus, the density function of ε

_{·t}is:

_{t}}. Denote the number of sudden changes by g and denote these g change-points by ${k}_{1}^{*},\cdots ,{k}_{g}^{*}$, such that $1<{k}_{1}^{*}<{k}_{2}^{*}<\cdots <{k}_{q}^{*}<T$. Thus, h

_{t}can be expressed as:

_{A}(t) is an indicator function of the set A, i.e.,

_{l}≠ 0 for l = 1,…, g. The aim of this paper is to estimate g and ${k}_{1}^{*},\dots ,{k}_{g}^{*}$, which can be done by the method given in [21]. Let $m=\lfloor \sqrt{T}\rfloor $ and p = ⌊T/m⌋, where ⌊c⌋ denotes the largest integer less than or equal to c. Denote θ = (θ

_{1},…, θ

_{p})′. By Jin, Shi and Wu (2013) [21], the estimate of θ is given by:

_{T}> 0, γ

_{T}> 0 are chosen by the Bayesian information criterion (BIC), and the penalty function pλ

_{T},γ

_{T}(|u|) satisfies the following assumption:

^{2}/(2 log(log(3m))), $a=\sqrt{b/(2\mathrm{log}(\mathrm{log}(3m)))}$ and $D=3m({Q}_{\widehat{k}}-{Q}_{T-(p-j-1)m})/{Q}_{T-(p-j-1)m}$. By Theorem 3.1.1 in [22], we have:

**Step 1.**Select all of the stations, such that at least one pair of ozone concentration observations from any two of these stations is not missing.**Step 2.**For the data from each station, do the following: Fit the temporal model (2) to the data. Since the data are not normally distributed, we transform the data by using the Box–Cox transformation given in (1). λ is chosen, such that the residuals obtained by fitting the temporal model are normally distributed. Test if the residuals are dependent.**Step 3.**Compute the sample covariance of the residuals resulting from fitting two temporal models to the data from two stations. Find the relationship between the covariance and the distance between the two stations, and then, construct the spatial weights matrix W. For example, if the sample covariance is decreasing as the distance between the corresponding two stations is increasing, we can use the inverse of the distance as the corresponding off-diagonal element in the spatial weight matrix W.Use the matrix W to establish the simultaneous autoregressive (SAR) model at each time. Estimate the parameters of the SAR model by using the residuals obtained by fitting N temporal models to the ozone concentration data.**Step 4.**Estimate the entropy h_{t}of the SAR model at each time t and denote it by ĥ_{t}. Apply the change-point detection method given in [21] to the entropy time series {ĥ_{t}} to detect multiple change-points.

## 3. Application to Real Ozone Concentration Data

_{i,t}, i = 1,…, 19; t = 1,…, 2684, formed by 2684 (22 years × 122 days) daily maximum eight-hour moving averages of hourly ozone concentration data recorded in micrograms per cubic meter from each of the 19 stations, which are displayed in Figure 1. Figure 2 displays the locations of the 19 stations and their indexes. The numbers of missing data at nine of the stations are under 200, while the numbers of missing data at the other five stations are between 400 and 800. The remaining five stations have a number of missing data close to 1000. Figure 3 presents the box-and-whisker plots of the data collected at each station. It is clear that the data at each station are not normally distributed. Thus, we apply the Box–Cox power transformation (1) to the data {X

_{i,t}} and obtain the transformed data {Z

_{i,t}(λ)}, for each λ ∈ {0.3, 0.31, 0.32,⋯, 0.6}. The final value of λ will be decided later.

_{i,t}(λ)} by least squares and obtain the estimates ${\widehat{\beta}}_{i,t}(\mathrm{\lambda})$, j = 1,⋯, 5, of the parameters β

_{j,i}(λ), j = 1,⋯, 5. We compute the residuals $\{{\widehat{\epsilon}}_{i,t}(\mathrm{\lambda})\}$ by:

_{i,t}(λ)} are approximately normally distributed. Thus, we can choose λ in terms of p-values of a normality test on $\{{\widehat{\epsilon}}_{i,t}(\mathrm{\lambda})\}$ for each fixed pair of λ and i. In this application, the Pearson chi-squared test (R code: pearson.test) is employed. By applying this test to the residuals $\{{\widehat{\epsilon}}_{i,t}(\mathrm{\lambda})\}$ for fixed λ and i, we obtain the p-value p

_{i}(λ). Let p(λ) = Median {p

_{i}(λ), i = 1,⋯, 19} for each λ ∈ {0.3, 0.31, 0.32,⋯, 0.6}. $\mathrm{\lambda}=\widehat{\mathrm{\lambda}}$ is chosen, such that $\widehat{\mathrm{\lambda}}=\mathrm{arg}{\mathrm{max}}_{\mathrm{\lambda}}p(\mathrm{\lambda})$, which turns out to be 0.48. Hence, $\widehat{\mathrm{\lambda}}=0.48$ is used in the Box–Cox power transformation (1) hereafter.

_{i,t}} for each fixed i are modelled as:

_{j,i}, j = 0, 1,…, 5 by the least squares method. Denote these estimates by ${\widehat{\beta}}_{j,i},j=0$, 1,…, 5. We can then compute the residuals ${\widehat{\epsilon}}_{i,t}$ for t = 1,…, T. $\left\{{\widehat{\beta}}_{j,i}\right\}$ and $\left\{{\widehat{\epsilon}}_{i,t}\right\}$ are plotted respectively in Figures 4 and 5. To examine if the model has fitted the data from each station well, we compute ${R}_{i}^{2}$ (the coefficient of determination) obtained by fitting the model (8) to the data from each of 19 monitoring stations. ${R}_{i}^{2}$, i = 1,…, 19 are displayed in Table 1, which shows that the values of ${R}_{i}^{2}$ are all larger than 0.95. We also compute the p-value p

_{i}obtained by performing Pearson chi-square test on $\{{\widehat{\epsilon}}_{i,t},\phantom{\rule{0.2em}{0ex}}t=1,\dots ,T\}$ for i = 1,…, 19, which are also displayed in Table 1. From this table, it can be observed that only three p-values of the Pearson chi-square test are smaller than 0.01. Further, for each time series $\{{\widehat{\epsilon}}_{i,t},\phantom{\rule{0.2em}{0ex}}t=1,\dots ,2684\}$, we compute the Box–Pierce test statistic ([23])for each of the two null hypotheses H

_{0}: ρ(1) = ρ(2) = ρ(3) = ρ(4) = 0 and H

_{0}: ρ(1) = ρ(2) = ⋯ = ρ(7) = 0, where ρ(k) is the autocorrelation at lag k (R code: Box.test). The box-and-whisker plot of the p-values from the Box–Pierce test is displayed in Figure 6, which shows that both null hypotheses cannot be rejected, i.e., the residuals can be considered as uncorrelated at Lags 1 to 7.

_{i}

_{,1}, s

_{i}

_{,2}) is the rectangular coordinate of the location of the i-th station. It can be seen that the covariance decreases as the distance increases. Thus, we construct the spatial weight matrix W = (w

_{i,j})

_{19×19}in (3) by letting all of its diagonal elements {w

_{i,i}} be zeros and off-diagonal elements {w

_{i,j}, i ≠ j} be the inverse distances between the stations i and j, i.e., w

_{i,j}= 1/d

_{i,j}.

_{·t}by $(({\widehat{\epsilon}}_{1,t}-\overline{\widehat{\epsilon}}{.}_{t}),\cdots ,{({\widehat{\epsilon}}_{19,t}-\overline{\widehat{\epsilon}}{.}_{t}))}^{\prime}$ in Model (3). By Ord (1975) [18], we obtain the maximum likelihood estimates ${\widehat{\rho}}_{t}$ and ${\widehat{\sigma}}_{t}^{2}$ of (3), and then, we obtain the estimate of ${\widehat{\Sigma}}_{t}={\widehat{\sigma}}_{t}^{2}{\left({I}_{19}-{\widehat{\rho}}_{t}W\right)}^{-2}$. Thus, we obtain ${\widehat{h}}_{t}=\frac{1}{2}\mathrm{log}\left[{\left(2\pi e\right)}^{n}\left|{\widehat{\Sigma}}_{t}\right|\right]$, an estimate of the differential entropy defined in (4).

_{t}}. Let ${S}_{i=1}^{19}{\left({\widehat{\epsilon}}_{i,t}-\overline{\widehat{\epsilon}}{.}_{t}\right)}^{2}/18$ be the sample variance. The sample mean $\overline{\widehat{\epsilon}}{.}_{t}$, Moran’s I ${\mathrm{I}}_{t}$ and ${\widehat{\rho}}_{t}$ are respectively displayed in Figure 8. We apply the change-point detection method given in [21] to each time series of $\overline{\widehat{\epsilon}}{.}_{t}$, ${\mathrm{I}}_{t}$ and ${\widehat{\rho}}_{t}$ and cannot find any change-point. Thus, if we only consider the time series $\overline{\widehat{\epsilon}}{.}_{t}$, ${\mathrm{I}}_{t}$ and ${\widehat{\rho}}_{t}$, we have to claim that there is no change in the ozone concentration in the Toronto region. In contract, by applying the same method to both time series $\left\{{S}_{t}^{2}\right\}$ and $\left\{{\widehat{\sigma}}_{t}^{2}\right\}$, we detect the same change-point at 456 (29 August 1991). If we also apply the same method to the time series {ĥ

_{t}}, we find three change-points, 1585 (30 September 2000), 1837 (7 June 2003) and 2183 (17 September 2005). The sample variance ${S}_{t}^{2}$, error variance ${\widehat{\sigma}}_{t}^{2}$ and entropy ĥ

_{t}are respectively displayed in Figure 9.

## 4. Conclusion

_{t}, a function of ${\widehat{\rho}}_{t}$ and ${\widehat{\sigma}}_{t}^{2}$, can be used for detecting changes in ground-level ozone concentration data. As demonstrated in Section 3, when the same change-point detection method is applied to each of the time series $\left\{\overline{\widehat{\epsilon}}{.}_{t}\right\}$, $\left\{{\mathrm{I}}_{t}\right\}$, $\left\{{\widehat{\rho}}_{t}\right\}$, $\left\{{S}_{t}^{2}\right\}$, $\left\{{\widehat{\sigma}}_{t}^{2}\right\}$ and {ĥ

_{t}}, the time series that is the best for detection of multiple change-points is {ĥ

_{t}}. This may be due to the fact that the entropy can be used to measure various spatial uncertainties, including both spatial variance and spatial dependence, and is able to extract more information from the data than some other statistics, e.g., ${\widehat{\rho}}_{t}$ and ${\widehat{\sigma}}_{t}^{2}$. The proposed methodology is also applicable to other climate data.

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Rodríguez, S.; Reyes, H.; Pérez, P.; Vaquera, H. Selection of a Subset of Meteorological Variables for Ozone Analysis: Case Study of Pedregal Station in Mexico City. J. Environ. Sci. Eng. A
**2012**, 1, 11–20. [Google Scholar] - Özbay, B.; Keskin, G.A.; Doğruparmak, Ş.Ç.; Ayberk, S. Multivariate Methods for Ground-level Ozone Modeling. Atmos. Res
**2011**, 102, 57–65. [Google Scholar] - Shu, Y.; Lam, N.S.N. Spatial Disaggregation of Carbon Dioxide Emissions from Road Traffic Based on Multiple Linear Regression Model. Atmos. Environ
**2011**, 45, 634–640. [Google Scholar] [CrossRef] - Baur, D.; Saisana, M.; Schulze, N. Modelling the Effects of Meteorological Variables on Ozone Concentration—a Quantile Regression Approach. Atmos. Environ
**2004**, 38, 4689–4699. [Google Scholar] - Munir, S.; Chen, H.; Ropkins, K. Modelling the Impact of Road Traffic on Ground Level Ozone Concentration Using a Quantile Regression Approach. Atmos. Environ
**2012**, 60, 283–291. [Google Scholar] - Chelani, A.B. Nonlinear dynamical analysis of ground level ozone concentrations at different temporal scales. Atmos. Environ
**2010**, 44, 4318–4324. [Google Scholar] [CrossRef] - Niu, X.F. Nonlinear Additive Models for Environmental Time Series, with Applications to Ground-Level Ozone Data Analysis. J. Am. Stat. Assoc
**1996**, 91, 1310–1321. [Google Scholar] - Weng, Y.C.; Chang, N.B.; Lee, T.Y. Nonlinear Time Series Analysis of Ground-level Ozone Dynamics in Southern Taiwan. J. Environ. Manag
**2008**, 87, 405–414. [Google Scholar] - Jin, B.; Chan, E.; Wu, Y. Hierarchical Bayesian Spatio-temporal Modelling of Regional Ozone Concentrations and Network Design. J. Environ. Stat
**2011**, 3, 1–32. [Google Scholar] - Sahu, S.K.; Bakar, K.S. Hierarchical Bayesian Autoregressive Models for Large Space-time Data with Applications to Ozone Concentration Modelling. Appl. Stoch. Models Bus. Ind
**2012**, 28, 395–415. [Google Scholar] [CrossRef] - Gyarmati-Szabó, J.; Bogachev, L.V.; Chen, H. Modelling Threshold Exceedances of Air Pollution Concentrations via Non-homogeneous Poisson Process with Multiple Change-points. Atmos. Environ
**2011**, 45, 5493–5503. [Google Scholar] [CrossRef] - Achcar, J.A.; Fernández-Bremauntz, A.A.; Rodrigues, E.R.; Tzintzun, G. Estimating the Number of Ozone Peaks in Mexico City Using a Non-homogeneous Poisson Model. Environmetrics
**2008**, 19, 469–485. [Google Scholar] [CrossRef] - Smith, R.L.; Shively, T.S. Point Process Approach to Modeling Trends in Tropospheric Ozone Based on Exceedances of a High Threshold. Atmos. Environ
**1995**, 29, 3489–3499. [Google Scholar] - de Nazelle, A.; Arunachalam, S.; Serre, M.L. Bayesian Maximum Entropy Integration of Ozone Observations and Model Predictions: an Application for Attainment Demonstration in North Carolina. Environ. Sci. Technol
**2010**, 44, 5707–5713. [Google Scholar] [CrossRef] - Fuentes, M.; Chaudhuri, A.; Holland, D.M. Bayesian Entropy for Spatial Sampling Design of Environmental Data. Environ. Ecol. Stat
**2007**, 14, 323–340. [Google Scholar] - Martínez, J.; García, P.J.; Alejano, L.; Reyes, A. Detection of Outliers in Gas Emissions from Urban Areas Using Functional Data Analysis. J. Hazard. Mater
**2011**, 186, 144–149. [Google Scholar] - Sancho, J.; Martínez, J.; Pastor, J.J.; Taboada, J.; Piñeiro, J.I.; García-Nieto, P.J. New Methodology to Determine Air Quality in Urban Areas Based on Runs Rules for Functional Data. Atmos. Environ
**2014**, 83, 185–192. [Google Scholar] - Ord, K. Estimation Methods for Models of Spatial Interaction. J. Am. Stat. Assoc
**1975**, 70, 120–126. [Google Scholar] - Oliveira, V.D.; Song, J.J. Bayesian Analysis of Simultaneous Autoregreesive Models. Sankhyā
**2008**, 70-B(2), 323–350. [Google Scholar] - Ahmed, N.A.; Gokhale, D.V. Entropy Expressions and Their Estimators for Multivariate Distributions. IEEE Trans. Inf. Theory
**1989**, 35, 688–692. [Google Scholar] - Jin, B.; Shi, X.; Wu, Y. A Novel and Fast Methodology for Simultaneous Multiple Structural Break Estimation and Variable Selection for Nonstationary Time Series Models. Stat. Comput
**2013**, 23, 221–231. [Google Scholar] [CrossRef] - Csörgő, M.; Horvath, L. Limit Theorems in Change-Point Analysis; Wiley: Chichester, UK, 1997. [Google Scholar]
- Box, G.E.P.; Pierce, D.A. Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models. J. Am. Stat. Assoc
**1970**, 65, 1509–1526. [Google Scholar] - Simmons, G. Canadian Regulation of Air Pollution from Motor Vehicles; Researched and prepared by Greg Simmons for Greenpeace and the Sierra Legal Defence Fund, 2002; Sierra Legal Defence Fund: Vancouver, BC, Canada, 2002. [Google Scholar]
- United Nations. Canadian On-road Vehicle and Engine Emission Regulations; Informal document No.4 (44th GRPE, 10-14 June 2002, agenda item 10.). Available online: www.unece.org/fileadmin/DAM/trans/doc/2002/wp29grpe/TRANS-WP29-GRPE-44-inf04e.pdf accessed on 29 April 2015.

**Figure 3.**The respective box-and-whisker plots of the ozone concentration data from the 19 stations.

**Figure 4.**The respective box-and-whisker plots of ${\widehat{\beta}}_{j,i},\phantom{\rule{0.2em}{0ex}}j=0,1,\dots ,5$.

**Figure 6.**The respective box-and-whisker plots of p-values of the Box–Pierce test on the respective two null hypotheses H

_{0}: ρ(1) = ρ(2) = ρ(3) = ρ(4) = 0 and H

_{0}: ρ(1) = ρ(2) = ⋯ = ρ(7) = 0.

**Figure 8.**Respective plots of $\overline{\widehat{\epsilon}}{.}_{t},$${\mathrm{I}}_{t}$, and ${\widehat{\rho}}_{t}$.

**Table 1.**The respective coefficient of determination, ${R}_{i}^{2}$, and the p-value, p

_{i}, for i = 1,⋯, 19.

Station ID
| ||||||||||
---|---|---|---|---|---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |

${R}_{i}^{2}$ | 0.9538 | 0.9662 | 0.9662 | 0.9647 | 0.96708 | 0.9641 | 0.9714 | 0.9657 | 0.9716 | 0.9493 |

p_{i} | 0.3568 | 0.7291 | 0.0110 | 0.0547 | 0.06204 | 0.4119 | 0.5094 | 0.5559 | 0.3411 | 0.3453 |

Station ID
| |||||||||
---|---|---|---|---|---|---|---|---|---|

11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | |

${R}_{i}^{2}$ | 0.9742 | 0.9711 | 0.9700 | 0.9742 | 0.97906 | 0.9745 | 0.9754 | 0.9752 | 0.9785 |

p_{i} | 0.4131 | 0.7438 | 0.0140 | 0.4816 | 0.01905 | 0.0056 | 0.1779 | 0.0001 | 0.0001 |

© 2015 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wu, Y.; Jin, B.; Chan, E.
Detection of Changes in Ground-Level Ozone Concentrations via Entropy. *Entropy* **2015**, *17*, 2749-2763.
https://doi.org/10.3390/e17052749

**AMA Style**

Wu Y, Jin B, Chan E.
Detection of Changes in Ground-Level Ozone Concentrations via Entropy. *Entropy*. 2015; 17(5):2749-2763.
https://doi.org/10.3390/e17052749

**Chicago/Turabian Style**

Wu, Yuehua, Baisuo Jin, and Elton Chan.
2015. "Detection of Changes in Ground-Level Ozone Concentrations via Entropy" *Entropy* 17, no. 5: 2749-2763.
https://doi.org/10.3390/e17052749