# dCATCH—A Numerical Package for d-Variate near G-Optimal Tchakaloff Regression via Fast NNLS

^{*}

## Abstract

**:**

## 1. Introduction

- use of d-variate Vandermonde-like matrices at X in a discrete orthogonal polynomial basis (obtained by discrete orthonormalization of the total-degree product Chebyshev basis of the minimal box containing X), with automatic adaptation to the actual dimension of ${\mathbb{P}}_{m}^{d}\left(X\right)$;
- few tens of iterations of the basic Titterington multiplicative algorithm until near G-optimality of the design is reached, with a checked G-efficiency of say $95\%$ (but with a design support still far from sparsity);
- Tchakaloff-like compression of the resulting near G-optimal design via NNLS solution of the underdetermined moment system, with concentration of the discrete probability measure by sparse re-weighting to a support $\subset X$, of cardinality at most ${\mathbb{P}}_{2m}^{d}\left(X\right)$, keeping the same G-efficiency;
- iterative solution of the large-scale NNLS problem by a new accelerated version of the classical Lawson-Hanson active set algorithm, that we recently introduced in Reference [3] for $2d$ and $3d$ instances and here we validate on higher dimensions.

## 2. G-Optimal Designs

## 3. Computing near G-Optimal Compressed Designs

- $C=\mathrm{dCHEBVAND}(n,X)$
- $[U,jvec]=\mathrm{dORTHVAND}(n,X,u,jvec)$
- $[pts,w]=\mathrm{dNORD}(m,X,gtol)$

- $[pts,w,momerr]=\mathrm{dCATCH}(n,X,u)$,

#### Accelerating the Lawson-Hanson Algorithm by Deviation Maximization (LHDM)

- $[x,resnorm,exitflag]=\mathrm{LHDM}(A,b,options)$.

## 4. Numerical Examples

#### 4.1. Complex 3d Shapes

#### 4.2. Hypercubes: Chebyshev Grids

#### 4.3. Hypercubes: Low-Discrepancy Points

**Remark**

**1.**

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Dessole, M.; Marcuzzi, F.; Vianello, M. dCATCH: A Numerical Package for Compressed d-Variate Near G-Optimal Regression. Available online: https://www.math.unipd.it/~marcov/MVsoft.html (accessed on 1 June 2020).
- Bos, L.; Vianello, M. CaTchDes: MATLAB codes for Caratheodory—Tchakaloff Near-Optimal Regression Designs. SoftwareX
**2019**, 10, 100349. [Google Scholar] [CrossRef] - Dessole, M.; Marcuzzi, F.; Vianello, M. Accelerating the Lawson-Hanson NNLS solver for large-scale Tchakaloff regression designs. Dolomit. Res. Notes Approx. DRNA
**2020**, 13, 20–29. [Google Scholar] - Atkinson, A.; Donev, A.; Tobias, R. Optimum Experimental Designs, with SAS; Oxford University Press: Oxford, UK, 2007. [Google Scholar]
- Pukelsheim, F. Optimal Design of Experiments; SIAM: Philadelphia, PA, USA, 2006. [Google Scholar]
- Celant, G.; Broniatowski, M. Interpolation and Extrapolation Optimal Designs 2-Finite Dimensional General Models; Wiley: Hoboken, NJ, USA, 2017. [Google Scholar]
- Mandal, A.; Wong, W.K.; Yu, Y. Algorithmic searches for optimal designs. In Handbook of Design and Analysis of Experiments; CRC Press: Boca Raton, FL, USA, 2015; pp. 755–783. [Google Scholar]
- De Castro, Y.; Gamboa, F.; Henrion, D.; Hess, R.; Lasserre, J.B. Approximate optimal designs for multivariate polynomial regression. Ann. Stat.
**2019**, 47, 127–155. [Google Scholar] [CrossRef] [Green Version] - Dette, H.; Pepelyshev, A.; Zhigljavsky, A. Improving updating rules in multiplicative algorithms for computing D-optimal designs. Comput. Stat. Data Anal.
**2008**, 53, 312–320. [Google Scholar] [CrossRef] [Green Version] - Torsney, B.; Martin-Martin, R. Multiplicative algorithms for computing optimum designs. J. Stat. Plan. Infer.
**2009**, 139, 3947–3961. [Google Scholar] [CrossRef] - Bloom, T.; Bos, L.; Levenberg, N.; Waldron, S. On the Convergence of Optimal Measures. Constr. Approx.
**2008**, 32, 159–169. [Google Scholar] [CrossRef] [Green Version] - Bos, L. Some remarks on the Fejér problem for lagrange interpolation in several variables. J. Approx. Theory
**1990**, 60, 133–140. [Google Scholar] [CrossRef] [Green Version] - Calvi, J.P.; Levenberg, N. Uniform approximation by discrete least squares polynomials. J. Approx. Theory
**2008**, 152, 82–100. [Google Scholar] [CrossRef] - Kiefer, J.; Wolfowitz, J. The equivalence of two extremum problems. Can. J. Math.
**1960**, 12, 363–366. [Google Scholar] [CrossRef] - Putinar, M. A note on Tchakaloff’s theorem. Proc. Am. Math. Soc.
**1997**, 125, 2409–2414. [Google Scholar] [CrossRef] - Titterington, D. Algorithms for computing D-optimal designs on a finite design space. In Proceedings of the 1976 Conference on Information Science and Systems; John Hopkins University: Baltimore, MD, USA, 1976; Volume 3, pp. 213–216. [Google Scholar]
- Burkardt, J. MONOMIAL: A Matlab Library for Multivariate Monomials. Available online: https://people.sc.fsu.edu/~jburkardt/m_src/monomial/monomial.html (accessed on 1 June 2020).
- Bos, L.; Piazzon, F.; Vianello, M. Near optimal polynomial regression on norming meshes. In Sampling Theory and Applications 2019; IEEE Xplore Digital Library: New York, NY, USA, 2019. [Google Scholar]
- Bos, L.; Piazzon, F.; Vianello, M. Near G-optimal Tchakaloff designs. Comput. Stat.
**2020**, 35, 803–819. [Google Scholar] [CrossRef] - Tchakaloff, V. Formules de cubatures mécaniques à coefficients non négatifs. Bull. Sci. Math.
**1957**, 81, 123–134. [Google Scholar] - Carathéodory, C. Über den Variabilitätsbereich der Fourier’schen Konstanten von positiven harmonischen Funktionen. Rendiconti Del Circolo Matematico di Palermo (1884–1940)
**1911**, 32, 193–217. [Google Scholar] [CrossRef] [Green Version] - Sommariva, A.; Vianello, M. Compression of Multivariate Discrete Measures and Applications. Numer. Funct. Anal. Optim.
**2015**, 36, 1198–1223. [Google Scholar] [CrossRef] [Green Version] - Pleśniak, W. Multivariate Jackson Inequality. J. Comput. Appl. Math.
**2009**, 233, 815–820. [Google Scholar] [CrossRef] [Green Version] - Lawson, C.L.; Hanson, R.J. Solving Least Squares Problems; SIAM: Philadelphia, PA, USA, 1995; Volume 15. [Google Scholar]
- Van Benthem, M.H.; Keenan, M.R. Fast algorithm for the solution of large-scale non-negativity-constrained least squares problems. J. Chemom.
**2004**, 18, 441–450. [Google Scholar] [CrossRef] - Hardin, D.; Michaels, T.; Saff, E. A Comparison of Popular Point Configurations on S
^{2}. Dolomit. Res. Notes Approx. DRNA**2016**, 9, 16–49. [Google Scholar] - Bloom, T.; Bos, L.; Calvi, J.; Levenberg, N. Polynomial Interpolation and Approximation in ${\mathbb{C}}^{d}$. Ann. Polon. Math.
**2012**, 106, 53–81. [Google Scholar] [CrossRef] [Green Version] - De Marchi, S.; Piazzon, F.; Sommariva, A.; Vianello, M. Polynomial Meshes: Computation and Approximation. In Proceedings of the CMMSE 2015, Rota Cadiz, Spain, 6–10 July 2015; pp. 414–425. [Google Scholar]
- Dick, J.; Pillichshammer, F. Digital Nets and Sequences-Discrepancy Theory and Quasi—Monte Carlo Integration; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
- Golub, G.H.; Van Loan, C.F. Matrix Computations, 3rd ed.; Johns Hopkins University Press: Baltimore, MD, USA, 1996. [Google Scholar]
- Slawski, M. Nonnegative Least Squares: Comparison of Algorithms. Available online: https://sites.google.com/site/slawskimartin/code (accessed on 1 June 2020).
- Beghi, A.; Marcuzzi, F.; Martin, P.; Tinazzi, F.; Zigliotto, M. Virtual prototyping of embedded control software in mechatronic systems: A case study. Mechatronics
**2017**, 43, 99–111. [Google Scholar] [CrossRef] - Beghi, A.; Marcuzzi, F.; Rampazzo, M. A Virtual Laboratory for the Prototyping of Cyber-Physical Systems. IFAC-PapersOnLine
**2016**, 49, 63–68. [Google Scholar] [CrossRef] - Lasserre, J.B. The moment-SOS hierarchy. Proc. Int. Cong. Math.
**2018**, 4, 3791–3814. [Google Scholar] - De Klerk, E.; Laurent, M. A survey of semidefinite programming approaches to the generalized problem of moments and their error analysis. In World Women in Mathematics 2018-Association for Women in Mathematics Series; Springer: Cham, Switzerland, 2019; Volume 20, pp. 17–56. [Google Scholar]
- Martinez, A.; Piazzon, F.; Sommariva, A.; Vianello, M. Quadrature-based polynomial optimization. Optim. Lett.
**2020**, 35, 803–819. [Google Scholar] [CrossRef]

**Figure 1.**Multibubble test case, regression degree $m=10$. (

**a**) The evolution of the cardinality of the passive set P along the iterations of the three LH algorithms. (

**b**) Multibubble with 1763 compressed Tchakaloff points, extracted from 18,915 original points.

**Figure 2.**The evolution of the cardinality of the passive set P along the iterations of the three LH algorithms for Chebyshev nodes’ tests.

**Figure 3.**The evolution of the cardinality of the passive set P along the iterations of the three LH algorithms for Halton points’ tests.

LS | Least Squares |

NNLS | Non-Negative Least Squares |

LH | Lawson-Hawson algorithm for NNLS |

LHI | Lawson-Hawson algorithm with unconstrained LS Initialization |

LHDM | Lawson-Hawson algorithm with Deviation Maximization acceleration |

dCATCH | d-variate CAratheodory-TCHakaloff discrete measure compression |

dCHEBVAND | d-variate Chebyshev-Vandermonde matrix |

dORTHVAND | d-variate Vandermonde-like matrix in a weighted orthogonal polynomial basis |

dNORD | d-variate Near G-Optimal Regression Designs |

LHDM | Lawson-Hawson algorithm with Deviation Maximization acceleration |

**Table 3.**Results for the multibubble numerical test: $compr=M/mean\left(cpts\right)$ is the mean compression ratio obtained by the three methods listed; ${t}_{LH}/{t}_{Titt}$ is the ratio between the execution time of LH and that of the Titterington algorithm; ${t}_{LH}/{t}_{LHDM}$ (${t}_{LHI}/{t}_{LHDM}$) is the ratio between the execution time of LH (LHI) and that of LHDM; $cpts$ is the number of compressed Tchakaloff points and $momerr$ is the final moment residual.

Test | LH | LHI | LHDM | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|

m | M | compr | ${\mathit{t}}_{\mathit{LH}}/{\mathit{t}}_{\mathit{Titt}}$ | ${\mathit{t}}_{\mathit{LH}}/{\mathit{t}}_{\mathit{LHDM}}$ | cpts | momerr | ${\mathit{t}}_{\mathit{LHI}}/{\mathit{t}}_{\mathit{LHDM}}$ | cpts | momerr | cpts | momerr |

10 | 18,915 | 11/1 | 40.0/1 | 2.7/1 | 1755 | $3.4\times {10}^{-8}$ | 3.2/1 | 1758 | $3.2\times {10}^{-8}$ | 1755 | $1.5\times {10}^{-8}$ |

**Table 4.**Results of numerical tests on $M={\left(2km\right)}^{d}$ Chebyshev’s nodes, with $k=4$, with different dimensions and degrees: $compr=M/mean\left(cpts\right)$ is the mean compression ratio obtained by the three methods listed; ${t}_{LH}/{t}_{Titt}$ is the ratio between the execution time of LH and that of Titterington algorithm; ${t}_{LH}/{t}_{LHDM}$ (${t}_{LHI}/{t}_{LHDM}$) is the ratio between the execution time of LH (LHI) and that of LHDM; $cpts$ is the number of compressed Tchakaloff points and $momerr$ is the final moment residual.

Test | LH | LHI | LHDM | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

d | m | M | compr | ${\mathit{t}}_{\mathit{LH}}/{\mathit{t}}_{\mathit{Titt}}$ | ${\mathit{t}}_{\mathit{LH}}/{\mathit{t}}_{\mathit{LHDM}}$ | cpts | momerr | ${\mathit{t}}_{\mathit{LHI}}/{\mathit{t}}_{\mathit{LHDM}}$ | cpts | momerr | cpts | momerr |

3 | 6 | 110,592 | 250/1 | 0.4/1 | 3.1/1 | 450 | $5.0\times {10}^{-7}$ | 3.5/1 | 450 | $3.4\times {10}^{-7}$ | 450 | $1.4\times {10}^{-7}$ |

4 | 3 | 331,776 | 1607/1 | 0.2/1 | 2.0/1 | 207 | $8.9\times {10}^{-7}$ | 3.4/1 | 205 | $9.8\times {10}^{-7}$ | 207 | $7.9\times {10}^{-7}$ |

5 | 2 | 1,048,576 | 8571/1 | 0.1/1 | 1.4/1 | 122 | $6.3\times {10}^{-7}$ | 1.5/1 | 123 | $3.6\times {10}^{-7}$ | 122 | $3.3\times {10}^{-7}$ |

**Table 5.**Results of numerical tests on Halton points: $compr=M/mean\left(cpts\right)$ is the mean compression ratio obtained by the three methods listed; ${t}_{LH}/{t}_{Titt}$ is the ratio between the execution time of LH and that of Titterington algorithm; ${t}_{LH}/{t}_{LHDM}$ (${t}_{LHI}/{t}_{LHDM}$) is the ratio between the execution time of LH (LHI) and that of LHDM; $cpts$ is the number of compressed Tchakaloff points and $momerr$ is the final moment residual.

Test | LH | LHI | LHDM | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

d | m | M | compr | ${\mathit{t}}_{\mathit{LH}}/{\mathit{t}}_{\mathit{Titt}}$ | ${\mathit{t}}_{\mathit{LH}}/{\mathit{t}}_{\mathit{LHDM}}$ | cpts | momerr | ${\mathit{t}}_{\mathit{LHI}}/{\mathit{t}}_{\mathit{LHDM}}$ | cpts | momerr | cpts | momerr |

10 | 2 | 10,000 | 10/1 | 41.0/1 | 1.9/1 | 990 | 1.1 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-8}$ | 1.9/1 | 988 | 9.8 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-9}$ | 990 | 9.4 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-9}$ |

10 | 2 | 100,000 | 103/1 | 6.0/1 | 3.1/1 | 968 | 3.6 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-7}$ | 2.8/1 | 973 | 2.7 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-7}$ | 968 | 4.2 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-7}$ |

4 | 5 | 10,000 | 10/1 | 20.2/1 | 2.3/1 | 997 | 9.7 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-9}$ | 2.4/1 | 993 | 1.3 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-8}$ | 997 | 2.1 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-9}$ |

4 | 5 | 100,000 | 103/1 | 2.0/1 | 3.8/1 | 969 | 6.6 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-7}$ | 3.8/1 | 964 | 6.3 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-7}$ | 969 | 5.3 $\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-7}$ |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Dessole, M.; Marcuzzi, F.; Vianello, M.
dCATCH—A Numerical Package for d-Variate near G-Optimal Tchakaloff Regression via Fast NNLS. *Mathematics* **2020**, *8*, 1122.
https://doi.org/10.3390/math8071122

**AMA Style**

Dessole M, Marcuzzi F, Vianello M.
dCATCH—A Numerical Package for d-Variate near G-Optimal Tchakaloff Regression via Fast NNLS. *Mathematics*. 2020; 8(7):1122.
https://doi.org/10.3390/math8071122

**Chicago/Turabian Style**

Dessole, Monica, Fabio Marcuzzi, and Marco Vianello.
2020. "dCATCH—A Numerical Package for d-Variate near G-Optimal Tchakaloff Regression via Fast NNLS" *Mathematics* 8, no. 7: 1122.
https://doi.org/10.3390/math8071122