# Towards Hyper-Dimensional Variography Using the Product-Sum Covariance Model

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

^{(3)}) for horizontal, vertical, and temporal axes, thus representing the simplest hyper-dimensional case above the two-dimensional variogram space originally used to develop the product-sum model.

## 2. Theory

_{h}denotes covariance and γ

_{h}variogram. To simplify notation, we will use symbol “V” for variogram and “COV” for covariance, and the number in the superscript next to the symbols will denote their dimensionality: V

^{(1)}stands for one-dimensional, V

^{(2)}two-dimensional variogram, etc., with corresponding covariance COV

^{(1)}, COV

^{(2)}, etc. Common spatio-temporal modeling (only horizontal spatial and temporal distance considered) belongs to V

^{(2)}class.

#### 2.1. Original Product-Sum Model and Modeling Procedure

_{s,t}(h

_{s},h

_{t}) = a

_{1}C

_{s}(h

_{s})C

_{t}(h

_{t})+a

_{2}C

_{s}(h

_{s})+a

_{3}C

_{t}(h

_{t})

_{s}and C

_{t}are valid spatial and temporal covariance models, respectively. De Iaco et al. [22] proved that for positive definiteness, it is sufficient that a

_{1}> 0, a

_{2}≥ 0 and a

_{3}≥ 0.

^{(1)}spatial (h

_{s}= 0) and temporal (h

_{t}= 0) variograms using the data, and then combined these models to obtain the final spatio-temporal variogram model:

_{s,t}(h

_{s},h

_{t})= γ

_{s,t}(h

_{s},0) + γ

_{s,t}(0, h

_{t}) – kγ

_{s,t}(h

_{s},0)γ

_{s,t}(0,h

_{v})

_{s,t}(h

_{s},0) and γ

_{s,t}(0, h

_{t}) are spatio-temporal variograms for h

_{s}= 0 and h

_{t}= 0, respectively.

^{(1)}). The only condition k has to fulfill to create an admissible covariance model is:

^{(1)}models, and the existence of tolerance per se makes the procedure somewhat subjective. In what follows, we present a new approach that does not require defining tolerances.

#### 2.2. Modeling of the Hyper-Dimensional Variogram Based on the Product-Sum Model

^{(1)}class of covariances C

_{s}and C

_{t}is not dependent on the dimensionality of the basic covariance models, and the same applies to corresponding variograms (Equation (3)). The minimum and necessary conditions to assure validity of the product-sum model are: (1) that basic COV

^{(1)}building blocks are valid models, and (2) that constants a

_{1}-a

_{3}are subjected to constraints mentioned earlier in Equation (2) (De Iaco et al. 2001). Thus, the product-sum model validity holds for any basic covariance(s) dimensionality (equivalent to C

_{s}and C

_{t}in Equation (2)), as long as they represent a valid model (for validity criteria, see Chiles and Delfiner, 2012). Subsequently, a COV

^{(2)}class of covariance modeled in the first step starting from basic COV

^{(1)}components can be used as a basic covariance in the subsequent steps, thus increasing the dimensionality of the resulting covariance.

_{s}and C

_{t}in the sense that if these terms exchange places in the equation, it will still converge to the same expression given that the constants a

_{1}–a

_{3}are data-driven, (note that constants a

_{1}–a

_{3}are not directly modeled, but rather implicitly through global and partial sills estimated from the data; please see Equations (4)–(8) in De Iaco et al. [22] for further clarifications). We will later show that fitting the model to data can be done sequentially, mimicking the approach from the original paper by De Iaco et al. [22], or, as we propose, all at once, thus avoiding the need to define tolerances.

^{(n)}type of covariance starting with COV

^{(n−1)}and COV

^{(1)}would have k

_{n-1}value:

_{1}–k

_{n−1}would have to be estimated from the data. This expression allows us to extend the model to an arbitrary number of dimensions.

#### 2.2.1. Sequential Hierarchical Modeling

^{(3)}class of the covariance, we could start by separately modeling one COV

^{(2)}class of the covariance using the product-sum model, and a COV

^{(1)}class of the covariance using basic one-dimensional model, and then combining them again using the product-sum model into a final COV

^{(3)}. The approach is depicted in Scheme 1.

_{h,v}(h

_{h},h

_{v})

^{(2)}= a

_{1}C

_{h}(h

_{h})

^{(1)}C

_{v}(h

_{v})

^{(1)}+a

_{2}C

_{h}(h

_{h})

^{(1)}+a

_{3}C

_{v}(h

_{v})

^{(1)}

_{h,v}(h

_{h},h

_{v})

^{(2)}and C

_{t}(h

_{t})

^{(1)}are combined into a final product-sum model:

_{h,v,t}(h

_{h},h

_{v},h

_{t})

^{(3)}= a

_{4}C

_{h,v}(h

_{h},h

_{v})

^{(2)}C

_{t}(h

_{t})

^{(1)}+a

_{5}C

_{h,v}(h

_{h},h

_{v})

^{(2)}+a

_{6}C

_{t}(h

_{t})

^{(1)}

_{h,v,t}(h

_{h},h

_{v},h

_{t})

^{(3)}=

**a**C

_{1}a_{4}_{h}(h

_{h})

^{(1)}C

_{v}(h

_{v})

^{(1)}C

_{t}(h

_{t})

^{(1)}+

**a**C

_{2}a_{4}_{h}(h

_{h})

^{(1)}C

_{t}(h

_{t})

^{(1)}+

**a**C

_{3}a_{4}_{v}(h

_{v})

^{(1)}C

_{t}(h

_{t})

^{(1)}+

… +

**a**C

_{1}a_{5}_{h}(h

_{h})

^{(1)}C

_{v}(h

_{v})

^{(1)}+

**a**C

_{2}a_{5}_{h}(h

_{h})

^{(1)}+

**a**C

_{3}a_{5}_{v}(h

_{v})

^{(1)}+

**a**C

_{6}_{t}(h

_{t})

^{(1)}

_{h,t}(h

_{h},h

_{t}) in Equation (8), the initial values for a

_{1}–a

_{6}would be different, yet their products (shown in bold) in Equation (10) would be the same, as they ultimately come from data, which again points to the symmetrical properties of the product sum model with respect to its basic covariance components. The resulting covariance is guaranteed to yield a valid model, as the product of constants a

_{1·}a

_{4}that multiplies the first term in Equation (10) is always >0, while all other products of constants are ≥0. The four underlined covariance product terms of the final model represent a Hadamard product [38] of two or more positive definite matrices. According to Schur product theorem, a Hadamard product of two positive definite matrices necessarily gives a positive definite matrix [39]. Thus, the resulting model is guaranteed to be valid.

_{s,t}(h

_{s},0) and γ

_{s,t}(0, h

_{t}) were modeled by pre-determining a spatial and temporal tolerance within which a pair of data can still be considered collocated or coincident. However, in practice the required tolerance might need to be very large to yield enough data points, which would lead to inaccuracies in the modeling of both one-dimensional and higher-dimensional variograms, given that lower dimensional variograms are building blocks of higher dimensional variogram (see Section 2.2.2 for further discussion).

#### 2.2.2. Modeling “All at Once”

_{s,t}(h

_{s},0)and γ

_{s,t}(0,h

_{t}):

_{s}and r

_{t}were, respectively, the vector lag with spatial tolerance ${\delta}_{s}$ and the lag with temporal tolerance ${\delta}_{t}$. |N(r

_{s})| and |M(r

_{t})| are the cardinalities of the following sets:

^{(n)}variogram corresponding to Equation (3). into an experimental (or alternatively a raw) variogram, and replacing k using Equation (7). Then the V

^{(n−1)}variogram component on the right-hand side of the expression is again substituted using Equation (3) and Equation (7), and maximal dimensionality of the basic variograms is again reduced by one. The procedure is recursively repeated until all variogram terms on the right-hand side have dimensionality one. While the expression itself can be very large, the number of parameters to estimate all at once is relatively small and computationally feasible. For example, to model V

^{(3)}variogram using common 3-parameter exponential or Gaussian model, it would be required to simultaneously estimate only 9 parameters (nugget, 3 times V

^{(1)}sill and range parameters, and two k-parameters). For temporally evolving anisotropic full 3D space, only 12 estimated parameters would be required. Generally, the number of parameters for an n-dimensional variogram to be estimated is 3n.

_{1}–k

_{n−1}play important roles, since the sill

^{(2)}–sill

^{(n)}values are obtained after the k value is optimized in each step. However, in modeling “all at once,” k constants are substituted following Equation (7), and do not appear as important entities over the course of modeling.

## 3. Application

^{(3)}variogram offers an adequate representation of the variability, and treated time, horizontal and vertical distances as orthogonal components in variogram space, and thus modeled separately. The vertical and temporal directions were modeled using Gaussian, and horizontal using an exponential model. After modeling the covariance, we kriged the PAR to a regular ~10 × 10 km grid mimicking the kriging approach in Tadić et al. [6], and obtained the average PAR for May 2006 shown in Figure 1.

^{−2}s

^{−1}higher (Figure 1, red circle) than the average of GPP calculated from the spatially-variable PAR (Figure 1, blue line). The difference between these two estimates is 24% of the observed standard deviation in daytime half-hourly GPP at a site near Lamont, Oklahoma [43], and indicates the importance of spatial variability in climate drivers of GPP. This likely represents an upper bound on such effects at this site, as the GPP-PAR curve used here has a high degree of nonlinearity to illustrate the utility of our method for a general class of problems. Related problems for which this method is applicable include modeling spatiotemporal variability in soil moisture and leaf area index, and the nonlinear influence of these variables on evapotranspiration and surface energy partitioning [45].

## 4. Conclusions

## 5. Software and/or Data Availability Section

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Chilès, J.-P.; Delfiner, P. Kriging. In Geostatistics: Modeling Spatial Uncertainty, 2nd ed.; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2012. [Google Scholar] [CrossRef]
- Stein, M. Statistical methods for regular monitoring data. J. R. Stat. Soc. Ser. B Stat. Methodol.
**2005**, 67, 667–687. [Google Scholar] [CrossRef] - Montero, J.M.; Fernández-Avilés, G.; Mateu, J. Spatial and Spatio-Temporal Geostatistical Modeling and Kriging; Wiley: Chichester, UK, 2015. [Google Scholar]
- Tadić, J.M.; Ilić, V.; Biraud, S. Examination of geostatistical and machine-learning techniques as interpolators in anisotropic atmospheric environments. Atmos. Environ.
**2015**, 111, 28–38. [Google Scholar] [CrossRef][Green Version] - Hammerling, D.M.; Michalak, A.M.; O’Dell, C.; Kawa, S.R. Global CO
_{2}distributions over land from the Greenhouse Gases Observing Satellite (GOSAT). Geophys. Res. Lett.**2012**, 39, L08804. [Google Scholar] [CrossRef] - Tadić, J.M.; Qiu, X.; Yadav, V.; Michalak, A.M. Mapping of satellite Earth observations using moving window block kriging. Geosci. Model Dev.
**2015**, 8, 1–9. [Google Scholar] [CrossRef] - Cressie, N.; Wikle, C. Statistics for Spatio-Temporal Data; Wiley: Hoboken, NJ, USA, 2011; 588p. [Google Scholar]
- Gneiting, T.; Genton, M.G.; Guttorp, P. Geostatistical space-time models, stationarity, separability and full symmetry. In Statistics of Spatio-Temporal Systems; Finkenstaedt, B., Held, L., Isham, V., Eds.; Monographs in Statistics and Applied Probability; Chapman & Hall/CRC Press: Boca Raton, FL, USA, 2007; pp. 151–175. [Google Scholar]
- Kyriakidis, P.C.; Journel, A.G. Geostatistical space–time models: A review. Math. Geol.
**1999**, 31, 651–684. [Google Scholar] [CrossRef] - Snepvangers, J.J.J.C.; Heuvelink, G.B.M.; Huisman, J.A. Soil water content interpolation using spatio-temporal kriging with external drift. Geoderma
**2003**, 112, 253–271. [Google Scholar] [CrossRef] - Stein, M. Space–Time Covariance Functions; Technical Rep. 4; Center for Integrating Statistical and Environ Science, University of Chicago: Chicago, IL, USA, 2004. [Google Scholar][Green Version]
- Horrell, M.T.; Stein, M.L. Half-spectral space–time covariance models. Spat. Stat.
**2017**, 19, 90–100. [Google Scholar] [CrossRef] - Rodrigues, A.; Diggle, P. A class of convolution-based models for spatio-temporal processes with non-separable covariance structure. Scand. J. Stat.
**2010**, 37, 553–567. [Google Scholar] [CrossRef] - Zastavnyi, V.; Porcu, E. Characterization theorems for the gneiting class of space-time covariances. Bernoulli
**2011**, 17, 456–465. [Google Scholar] [CrossRef] - De Iaco, S.; Myers, D.; Posa, D. On strict positive definiteness of product and product-sum covariance models. J. Stat. Plan. Inference
**2011**, 141, 1132–1140. [Google Scholar] [CrossRef] - De Iaco, S.; Posa, D. Predicting Spatio-Temporal Random Fields: Some Computational Aspects. Comput. Geosci.
**2012**, 41, 12–24. [Google Scholar] [CrossRef] - De Iaco, S.; Posa, D. Positive and Negative Non-Separability for Space-Time Covariance Models. J. Stat. Plan. Inference
**2013**, 143, 378–391. [Google Scholar] [CrossRef] - De Iaco, S.; Posa, D.; Myers, D.E. Characteristics of Some Classes of Space-Time Co-variance Functions. J. Stat. Plan. Inference
**2013**, 143, 2002–2015. [Google Scholar] [CrossRef] - De Iaco, S.; Palma, M.; Posa, D. A General Procedure for Selecting a Class of Fully Symmetric Space-Time Covariance Functions. Environmetrics
**2016**, 112, 212–224. [Google Scholar] [CrossRef] - Heuvelink, G.B.M.; Pebesma, E.; Gräler, B. Space-Time Geostatistics published in Encyclopedia of GIS; Springer International Publishing: Cham, Switzerland, 2017; pp. 1919–1926. [Google Scholar] [CrossRef]
- De Iaco, S.; Posa, D. Strict positive definiteness in geostatistics. Stoch. Environ. Res. Risk Assess.
**2018**, 32, 577–590. [Google Scholar] [CrossRef] - De Iaco, S.; Myers, D.; Posa, D. Space-time analysis using a general product–sum model. Stat. Probab. Lett.
**2001**, 52, 21–28. [Google Scholar] [CrossRef] - Tadić, J.M.; Michalak, A.M.; Iraci, L.; Ilić, V.; Biraud, S.C.; Feldman, D.R.; Built, T.; Johnson, M.S.; Loewenstein, M.; Jeong, S.; et al. Elliptic Cylinder Airborne Sampling and Geostatistical Mass Balance Approach for Quantifying Local Greenhouse Gas Emissions. Environ. Sci. Technol.
**2017**, 51, 10012–10021. [Google Scholar] [CrossRef] - Brock, F.V.; Crawford, K.C.; Elliott, R.L.; Cuperus, G.W.; Stadler, S.J.; Johnson, H.W.; Eilts, M.D. The Oklahoma Mesonet—A technical overview. J. Atmos. Ocean. Technol.
**1995**, 12, 5–19. [Google Scholar] [CrossRef] - McPherson, R.A.; Fiebrich, C.A.; Crawford, K.C.; Kilby, J.R.; Grimsley, D.L.; Martinez, J.E.; Basara, J.B.; Illston, B.G.; Morris, D.A.; Kloesel, K.A.; et al. Statewide monitoring of the mesoscale environment: A technical update on the Oklahoma Mesonet. J. Atmos. Oceanic Technol.
**2007**, 24, 301–321. [Google Scholar] [CrossRef] - Tadić, J.M.; Qiu, X.; Miller, S.; Michalak, A.M. Spatio-temporal approach to moving window block kriging of satellite data V1.0. Geosci. Model Dev.
**2017**, 10, 709–720. [Google Scholar] [CrossRef] - Christakos, G. On the problem of permissible covariance and variogram models. Water Resour. Res.
**1984**, 20, 251–265; [Google Scholar] [CrossRef] - Dimitrakopoulos, R.; Luo, X. Spatiotemporal modeling: Covariances and ordinary kriging systems. In Quantitative Geology and Geostatistics, Geostatistics for the Next Century; Dimitrakopoulos, R., Ed.; Springer: Dordrecht, The Netherlands, 1994; pp. 88–93. [Google Scholar]
- Rouhani, S.; Hall, T.J. Space-Time Kriging of Groundwater Data. In Geostatistics; Armstrong, M., Ed.; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1989; Volume 2, pp. 639–651. [Google Scholar]
- De Cesare, L.; Myers, D.E.; Posa, D. Spatio-temporal modelling of SO
_{2}in Milan district. In Geostatistics Wollongong; Baafi, E.Y., Schofield, N.A., Eds.; Kluwer Academic Publishing: Dordrecht, The Netherlands, 1996; pp. 1031–1042. [Google Scholar] - Cressie, N.; Huang, H.C. Classes of nonseperable, spatio-temporal stationary covariance functions. J. Am. Stat. Assoc.
**1999**, 94, 1–53. [Google Scholar] [CrossRef] - Guo, L.; Lei, L.; Zeng, Z. Spatiotemporal correlation analysis of satellite-observed CO
_{2}: Case studies in China and USA. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Melbourne, Australia, 21–26 July 2013. [Google Scholar] - Zeng, Z.; Lei, L.; Guo, L.; Zhang, L.; Zhang, B. Incorporating temporal variability to improve geostatistical analysis of satellite-observed CO
_{2}in China. Chin. Sci. Bull.**2013**, 58, 1948–1954. [Google Scholar] [CrossRef] - Zeng, Z.-C.; Lei, L.; Strong, K.; Jones, D.B.A.; Guo, L.; Liu, M.; Deng, F.; Deutscher, N.M.; Dubey, M.K.; Griffith, D.W.T.; et al. Global land mapping of satellite-observed CO2 total columns using spatio-temporal geostatistics. Int. J. Digit. Earth
**2017**, 10, 426–456. [Google Scholar] [CrossRef][Green Version] - De Cesare, L.; Myers, D.; Posa, D. Estimating and modeling space–time correlation structures. Stat. Prob. Lett.
**2001**, 51, 9–14. [Google Scholar] [CrossRef] - De Cesare, L.; Myers, D.E.; Posa, D. Product–sum covariance for space–time modeling: An environmental application. Environmetrics
**2001**, 12, 11–23. [Google Scholar] [CrossRef] - Salmon, M.M. Introduction to Logic and Critical Thinking, 6th ed.; Cengage Learning: Boston, MA, USA, 2012. [Google Scholar]
- Million, E. The Hadamard Product. 2007. Available online: http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf (accessed on 9 January 2019).
- Mathias, R. Matrix completions, norms and Hadamard products. Proc. Am. Math. Soc.
**1993**, 117, 905–918. [Google Scholar] - Baldocchi, D. FLUXNET: A new tool to study the temporal and spatial variability of ecosystem-scale carbon dioxide, water vapor, and energy flux densities. Bull. Am. Meteorol. Soc.
**2001**, 82, 2415–2434. [Google Scholar] [CrossRef][Green Version] - Vilà-Guerau de Arellano, J.; Ouwersloot, H.G.; Baldocchi, D.; Jacobs, C.M.J. Shallow cumulus rooted in photosynthesis. Geophys. Res. Lett.
**2014**, 41, 1796–1802. [Google Scholar] [CrossRef] - Lambers, H.; Stuart Chapin, F.S.; Pons, T.L. Photosynthesis. In Plant Physiological Ecology; Springer: New York, NY, USA, 2008; pp. 11–99. [Google Scholar]
- Williams, I.N.; Riley, W.J.; Kueppers, L.M.; Biraud, S.C.; Torn, M.S. Separating the effects of phenology and diffuse radiation on gross primary productivity in winter wheat. J. Geophys. Res. Biogeosci.
**2016**, 121, 1903–1915. [Google Scholar] [CrossRef] - Haas, T.C. Lognormal and moving window methods of estimating acid deposition. J. Am. Stat. Assoc.
**1990**, 85, 950–963. [Google Scholar] [CrossRef] - Bagley, J.E.; Kueppers, L.M.; Billesbach, D.P.; Williams, I.N.; Biraud, S.C.; Torn, M.S. The influence of land cover on surface energy partitioning and evaporative fraction regimes in the U.S. Southern Great Plains. J. Geophys. Res. Atmos.
**2017**, 122, 5793–5807. [Google Scholar] [CrossRef] - Romanowicz, R.; Young, P.; Brown, P.; Diggle, P. A recursive estimation approach to the spatio-temporal analysis and modelling of air quality data. Environ. Model. Softw.
**2006**, 21, 759–769. [Google Scholar] [CrossRef] - Tadić, J. Hyperdimensional Variography Code. Available online: https://www.researchgate.net/publication/313387764_Hyperdimensional_Variography_Code (accessed on 10 January 2019).
- Creative Commons. Available online: https://creativecommons.org/licenses/ (accessed on 10 January 2019).

**Scheme 1.**The sequence of necessary steps (1…(n−1)), to model COV

^{(n)}class of covariance using the product-sum model sequentially, starting from one-dimensional basic covariance (variogram) models COV

_{1}

^{(1)}-COV

_{n}

^{(1)}.

**Figure 1.**Application of kriged shortwave radiation (i.e., PAR) to estimate gross primary productivity (GPP) over a 2.5° longitude × 2.5° latitude region (~220 km) in central Oklahoma. The black curve illustrates the nonlinear relationship between GPP and PAR from a canopy model. The histogram shows the distribution of PAR values within the region on May 30 of 2006 (16:00–6:30 local time). Due to the saturation of GPP at high PAR, estimates of GPP from the spatially-averaged PAR (red circle) are higher than the actual regional-scale GPP (blue line).

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Tadić, J.M.; Williams, I.N.; Tadić, V.M.; Biraud, S.C.
Towards Hyper-Dimensional Variography Using the Product-Sum Covariance Model. *Atmosphere* **2019**, *10*, 148.
https://doi.org/10.3390/atmos10030148

**AMA Style**

Tadić JM, Williams IN, Tadić VM, Biraud SC.
Towards Hyper-Dimensional Variography Using the Product-Sum Covariance Model. *Atmosphere*. 2019; 10(3):148.
https://doi.org/10.3390/atmos10030148

**Chicago/Turabian Style**

Tadić, Jovan M., Ian N. Williams, Vojin M. Tadić, and Sébastien C. Biraud.
2019. "Towards Hyper-Dimensional Variography Using the Product-Sum Covariance Model" *Atmosphere* 10, no. 3: 148.
https://doi.org/10.3390/atmos10030148