The Multivariate Theory of Connections

This paper extends the univariate Theory of Connections, introduced in (Mortari, 2017), to the multivariate case on rectangular domains with detailed attention to the bivariate case. In particular, it generalizes the bivariate Coons surface, introduced by (Coons, 1984), by providing analytical expressions, called constrained expressions, representing all possible surfaces with assigned boundary constraints in terms of functions and arbitrary-order derivatives. In two dimensions, these expressions, which contain a freely chosen function, g(x, y), satisfy all constraints no matter what the g(x, y) is. The boundary constraints considered in this article are Dirichlet, Neumann, and any combinations of them. Although the focus of this article is on two-dimensional spaces, the final section introduces the Multivariate Theory of Connections, validated by mathematical proof. This represents the multivariate extension of the Theory of Connections subject to arbitrary-order derivative constraints in rectangular domains. The main task of this paper is to provide an analytical procedure to obtain constrained expressions in any space that can be used to transform constrained problems into unconstrained problems. This theory is proposed mainly to better solve PDE and stochastic differential equations.


Introduction
The Theory of Connections (ToC), as introduced in [1], consists of a general analytical framework to obtain constrained expressions, f (x), in one-dimension. A constrained expression is a function expressed in terms of another function, g(x), that is freely chosen and, no matter what the g(x) is, the resulting expression always satisfies a set of n constraints. ToC generalizes the one-dimensional interpolation problem subject to n constraints using the general form, f x = g x + ∑ k = 1 n η k p k x , (1) where p k (x) are n user-selected linearly independent functions, η k are derived by imposing the n constraints, and g(x) is a freely chosen function subject to be defined and nonsingular where the constraints are specified. Besides this requirement, g(x) can be any function, including, discontinuous functions, delta functions, and even functions that are undefined in some domains. Once the η k coefficients have been derived, then Equation (1) satisfies all the n constraints, no matter what the g(x) function is.
Constrained expressions in the form given in Equation (1) are provided for a wide class of constraints, including constraints on points and derivatives, linear combinations of constraints, as well as infinite and integral constraints [2]. In addition, weighted constraints [3] and point constraints on continuous and discontinuous periodic functions with assigned period can also be obtained [1]. How to extend ToC to inequality and nonlinear constraints is currently a work in progress.
The Theory of Connections framework can be considered the generalization of interpolation; rather than providing a class of functions (e.g., monomials) satisfying a set of n constraints, it derives all possible functions satisfying the n constraints by spanning all possible g(x) functions. This has been proved in Ref. [1]. A simple example of a constrained expression is, This equation always satisfies df dx x 1 = ẏ 1 and df dx x 2 = ẏ 2 , as long as ġ x 1 and ġ x 2 are defined and nonsingular. In other words, the constraints are embedded into the constrained expression.
Constrained expressions can be used to transform constrained optimization problems into unconstrained optimization problems. Using this approach, fast least-squares solutions of linear [4] and nonlinear [5] ODE have been obtained at machine error accuracy and with low (actually, very low) condition number. Direct comparisons of ToC versus MATLAB's ode45 [6] and Chebfun [7] have been performed on a small test of ODE with excellent results [4,5].
In particular, the ToC approach to solve ODE consists of a unified framework to solve IVP, BVP, and multi-value problems. The extension of differential equations subject to component constraints [8] has opened the possibility for ToC to solve in real-time a class of direct optimal control problems [9], where the constraints connect state and costate.
This study first extends the Theory of Connections to two-dimensions by providing, for rectangular domains, all surfaces that are subject to: (1) Dirichlet constraints; (2) Neumann constraints; and (3) any combination of Dirichlet and Neumann constraints. This theory is then generalized to the Multivariate Theory of Connections which provide in n-dimensional space all possible manifolds that satisfy boundary constraints on the value and boundary constraints on any-order derivative.
This article is structured as follows. First, it shows that the one-dimensional ToC can be used in two dimensions when the constraints (functions or derivatives) are provided along one axis only. This is a particular case, where the original univariate theory [1] can be applied with basically no modifications. Then, a two dimensional ToC version is developed for Dirichlet type boundary constraints. This theory is then extended to include Neumann and mixed type boundary constraints. Finally, the theory is extended to n-dimensions and to incorporate arbitrary-order derivative boundary constraints followed by a mathematical proof validating it.

Manifold Constraints in One Axis, Only
Consider the function, f (x), where f : ℝ n ℝ 1 , subject to one constraint manifold along the ith variable, For instance, in 3-D space, this can be the surface constraint, f x, y, z y = π = c x, π, z . All manifolds satisfying this constraint can be expressed using the additive form provided in Ref. [1], is a freely chosen function that must be defined and nonsingular at the constraint coordinates. When m manifold constraints are defined along the x i -axis, then the 1-D methodology [1] can be applied as it is. For instance, the constrained expression subject to m constraints along the x i variable evaluated at x i = w k , where k ∈ [1, m], that is, Note that this equation coincides with the Waring interpolation form (better known as Lagrangian interpolation form) [10] if the free function vanishes, g(x) = 0.

Example #1: Surface Subject to Four Function Constraints
The first example is designed to show how to use Equation (3) with mixed, continuous, discontinuous, and multiple constraints. Consider the following four constraints, c x, − 2 = sin 2x , c x, 0 = 3cosx x + 1 mod 2 , c x, 1 = 9e −x 2 , and c x, 3 = 1 − x .
This example highlights that the constraints and free-function may be discontinuous by using the modular arithmetic function. The result is a surface that is continuous in x at some coordinates (at y = −2, 1, and 3) and discontinuous at y = 0. The surfaces shown in Figures 1 and 2 were obtained using two distinct expressions for the free function, g(x, y).

Example #2: Surface Subject to Two Functions and One Derivative Constraint
This second example is provided to show how to use the general approach given in Equation (1) and described in [1], when derivative constraints are involved. Consider the following three constraints, c x, − 2 = sin 2x , c y x, 0 = 0, and c x, 1 = 9e −x 2 .
The three constraints imply the constraints, sin 2x = g x, − 2 + η 1 − 2η 2 + 4η 3 0 = g y x, 0 + η 2 9e −x 2 = g x, 1 + η 1 + η 2 + η 3 , from which the values of the η k coefficients, can be derived. After substituting these coefficients into Equation (4), the constrained expression that always satisfies the three initial constraints is obtained. Using this expression and two different free functions, g(x, y), we obtained the surfaces shown in Figures 3 and 4, respectively. The constraint c y (x, 0) = 0, difficult to see in both figures, can be verified analytically.

Connecting Functions in Two Directions
In this section, the Theory of Connections is extended to the two-dimensional case. Note that dealing with constraints in two (or more) directions (functions or derivatives) requires particular attention. In fact, two orthogonal constraint functions cannot be completely distinct as they intersect at one point where they need to match in value. In addition, if the formalism derived for the 1-D case is applied to 2-D case, some complications arise. These complications are highlighted in the following simple clarifying example.
Consider the two boundary constraint functions, f (x, 0) = q(x) and f (0, y) = h(y). Searching the constrained expression as originally done for the one-dimensional case implies the expression, f x, y = g x, y + η 1 p 1 x, y + η 2 p 2 x, y .
To obtain the values of η 1 and η 2 , the determinant of the matrix to invert is p 1 (x, 0) p 2 (0, y) − p 1 (0, y) p 2 (x, 0). This determinant is y by selecting p 1 (x, y) = 1 and p 2 (x, y) = y, or it is x by selecting p 1 (x, y) = x and p 2 (x, y) = 1. Therefore, to avoid singularities, this approach requires paying particular attention to the domain definition and/or on the user-selected functions, p k (x, y). To avoid dealing with these issues, a new (equivalent) formalism to derive constrained expressions is devised for the higher dimensional case.
The Theory of Connections extension to the higher dimensional case (with constraints on all axes) can be obtained by re-writing the constrained expression into an equivalent form, highlighting a general and interesting property. Let us show this by an example. Equation (2) can be re-written as, These two components, A(x) and B(x), of a constrained expression have a specific general meaning. The term, A(x), represents an (any) interpolating function satisfying the constraints while the B(x) term represents all interpolating functions that are vanishing at the constraints. Therefore, the generation of all functions satisfying multiple orthogonal constraints in n-dimensional space can always be expressed by the general form, f (x) = A(x) + B(x), where A(x) is any function satisfying the constraints and B(x) must represent all functions vanishing at the constraints. Equation f (x) = A(x) + B(x) is actually an alternative general form to write a constrained expression, that is, an alternative way to generalize interpolation: rather than derive a class of functions (e.g., monomials) satisfying a set of constraints, it represents all possible functions satisfying the set of constraints.
To prove that this additive formalism can describe all possible functions satisfying the constraints is immediate. Let f (x) be all functions satisfying the constraints and y(x) = A(x) + B(x) be the sum of a specific function satisfying the constraints, A(x), and a function, B(x), representing all functions that are null at the constraints. Then, y(x) will be equal to f , representing all functions that are null at the constraints.
As shown in Equation (5), once the A(x) function is obtained, then the B(x) function can be immediately derived. In fact, B(x) can be obtained by subtracting the A(x) function, where all the constraints are specified in terms of the g(x) free function, from the free function g(x).
For this reason, let us write the general expression of a constrained expression as, where A(g(x)) indicates the function satisfying the constraints where the constraints are specified in term of g(x).
The previous discussion serves to prove that the problem of extending Theory of Connections to higher dimensional spaces consists of the problem of finding the function, A(x), only. In two dimensions, the function A(x) is provided in literature by the Coons surface [11], f (x, y). This surface satisfies the Dirichlet boundary constraints, f 0, y = c 0, y , f 1, y = c 1, y , f x, 0 = c x, 0 , and f x, 1 = c x, 1 , (7) where the surface is contained in the x, y ∈ or, equivalently, where ℳ c x, y = 0 c x, 0 c x, 1 c 0, y −c 0, 0 −c 0, 1 c 1, y −c 1, 0 −c 1, 1 and v z = 1 1 − z z .

Mortari and Leake
Page 6 Since the f (x, y) boundaries match the boundaries of the c(x, y) constraint function, then the identity, f x, y = v T x ℳ f x, y v y , holds for any f (x, y) function. Therefore, the B(x) function can be set as, representing all functions that are always zero at the boundary constraints, as g(x, y) is a free function.

Theory of Connections Surface Subject to Dirichlet Constraints
Equations (8) and (9) can be merged to provide all surfaces with the boundary constraints defined in Equation (7) in the following compact form, where, again, A(x, y) indicates an expression satisfying the boundary function constraints defined by c(x, y) and B(x, y) an expression that is zero at the boundaries. In matrix form, where g(x, y) is a freely chosen function. In particular, if g(x, y) = 0, then the ToC surface becomes the Coons surface. c 0, y = sin −π/4 cos 4y + π/3 c 1, y = sin 3 − π/4 cos 4y + π/3 , and Figure 5 (right) shows a ToC surface that is obtained using the free function, g x, y = 1 3 cos 4πx sin 6πy − x 2 cos 2πy .
Mortari and Leake Page 7 For generic boundaries defined in the rectangle x, y ∈ [x i , x f ] × [y i , y f ], the ToC surface becomes, Equation (12) can also be set in matrix form, Note that all the ToC surfaces provided are linear in g(x, y), and, therefore, they can be used to solve, by linear/nonlinear least-squares, two-dimensional optimization problems subject to boundary function constraints, such as linear/nonlinear partial differential equations.

Multi-Function Constraints at Generic Coordinates
Equation (12) can be generalized to many function constraints (grid of functions). Assume a set of n x function constraints c(x k , y) and a set of n y function constraints c(x, y k ) intersecting at the n x n y points p ij = c(x i , y j ), then all surfaces satisfying the n x n y function constraints can be expressed by, Mortari and Leake Page 8 Again, Equation (13) can be written in compact form, Two examples of ToC surfaces are given in Figure 6 in the x, y ∈ [−2, 1] × [1,3] domain.

The formulation provided in Equation
where v z : = 1, 2z 3 − 3z 2 + 1, z 3 − 2z 2 + z, −2z 3 + 3z 2 , z 3 − z 2 T The ToC in 2D with function and derivative boundary constraints is simply, where the ℳ matrix and the v vectors are provided by Equations (17) and (16) (6) is not unique. Therefore, the p(x) and q(y) functions must satisfy p(0) = 1 and q(0) = 1. The simplest expressions satisfying these equations can be obtained by selecting p(x) = 1 and q(y) = 1. In this case, the associated ToC surface is given by, f x, y = 1 1 g x, y c x, 0 − g x, 0 c 0, y − g 0, y g 0, 0 − c 0, 0 1 1 Note that any functions satisfying p(0) = 1 and q(0) = 1 can be adopted to obtain the ToC surface satisfying the constraints f (0, y) = c(0, y) and f (x, 0) = c(x, 0). This is because there are infinite Coons-type surfaces satisfying the constraints. Consequently, the vectors v(x) and v(y) are not unique.

Constraints: c(0, y) and c y (x, 0)
For these boundary constraints, the Coons-type surface is expressed by, Therefore, the p(x) and q(y) functions must satisfy p(0) = 1 and q y (0) = 1. One solution is p(x) = 1 and q(y) = y. Therefore, the associated ToC surface is given by, f x, y = 1 1 g x, y c y x, 0 − g y x, 0 c 0, y − g 0, y g y 0, 0 − c y 0, 0 1 y .

Constraints
In this case, the Coons-type surface satisfying the boundary constraints is in the form,

Generic Mixed Constraints
Consider the case of mixed constraints, In this case, the surface satisfying the boundary constraints is built using the matrix, and v y = 1 q 1 y, y 1 , y 2 , y 3 q 2 y, y 1 , y 2 , y 3 q 3 y, y 1 , y 2 , y 3 are vectors made of the (not unique) function vectors v(x) and v(y) whose expressions can be found by satisfying the constraints (as done in the previous four subsections) along with a methodology similar to that given in Section 5.

Constraints
This section provides the Multivariate Theory of Connections, as the generalization to ndimensional rectangular domains with arbitrary-order boundary derivatives of what is presented above for two-dimensional space. Using tensor notation, this generalization is represented in the following compact form, where n is the number of orthogonal coordinates defined by the vector x = {x 1 , x 2 , …, x n }, v i k x k is the i k th element of a vector function of the variable x k , ℳ is an n-dimensional tensor that is a function of the boundary constraints defined in c(x), and g(x) is the freefunction.
In Equation (20) Additionally, let us define an operator, called the boundary constraint operator, whose purpose is to take the dth derivative with respect to coordinate x k and then evaluate that function at x k = p. Equation (21) shows the idea.
In general, for a function of n variables, the boundary constraint operator identifies an n − 1dimensional manifold. As the boundary constraint operator is used throughout this proof, it is important to note its properties when acting on sums and products of functions. Equation

The v Vectors
Each vector, v k , is associated with the ℓ k constraints that are specified by k c p k and substituting the values of α ij , we obtain v i 1 = 1, 1 − x 1 , x 1 .

Proof
This section demonstrates that the term A(x) from Equation (20) Mortari and Leake Page 18 Mathematics (Basel). Author manuscript; available in PMC 2020 May 29.
The boundary constraint operator applied to v k yields, Since the only nonzero terms are associated with i k = 1, q, we have, Applying the boundary constraint operator to the n − 1-dimensional ℳ tensor where index i k = q has no effect, because all of the functions already have coordinate x k substituted for the value p q−1 (see Equation (24)). Moreover, applying the boundary constraint operator to the ℳ tensor where index i k = 1 causes all terms in the sum within the parenthesis in Equation From Equation (23), we note that boundary constraint operators that take a derivative follow the usual product rule when applied to a product. Moreover, we note that all of the v vectors except v i k do not depend on x k , thus applying the boundary constraint operator to them results in a vector of zeros. Applying the boundary constraint operator to v i k yields, and applying the boundary constraint operator to ℳ yields, k b p q − 1 d q − 1 ℳ i 1 i 2 …1…i n = = k b p q − 1 d q − 1 ℳ i 1 i 2 …1…i n , i k = 1 = 0, i k ≠ 1.

Substituting these simplifications into
Again, all of the vectors v were designed such that their first component is 1, and the value of the element of ℳ for all indices equal to 1 is 0. Therefore, Equation (32) simplifies to, which proves Equation (20) works for arbitrary-order derivative boundary constraints.
In conclusion, the term A(x) from Equation (20)

Conclusions
This paper extends to n-dimensional spaces the Univariate Theory of Connections (ToC), introduced in Ref. [1]. First, it provides a mathematical tool to express all possible surfaces subject to constraint functions and arbitrary-order derivatives in a boundary rectangular domain, and then it extends the results to the multivariate case by providing the Multivariate Theory of Connections, which allows one to obtain n-dimensional manifolds subject to anyorder derivative boundary constraints.
In particular, if the constraints are provided along one axis only, then this paper shows that the univariate ToC, as defined in Ref. [1], can be adopted to describe all possible surfaces satisfying the constraints. If the boundary constraints are defined in a rectangular domain, then the constrained expression is found in the form f (x) = A(x) + B(x), where A(x) can be any function satisfying the constraints and B(x) describes all functions that are vanishing at the constraints. This is obtained by introducing a free function, g(x), into the function B(x) in such a way that B(x) is zero at the constraints no matter what the g(x) is. This way, by spanning all possible g(x) surfaces (even discontinuous, null, or piece-wise defined) the resulting B(x) generates all surfaces that are zero at the constraints and, consequently, f (x) = A(x) + B(x), describes all surfaces satisfying the constraints defined in the rectangular boundary domain. The function A(x) has been selected as a Coons surface [11] and, in particular, a Coons surface is obtained if g(x) = 0 is selected. All possible combinations of Dirichlet and Neumann constraints are also provided in Appendix A.
The last section provides the Multivariate Theory of Connections extension, which is a mathematical tool to transform n-dimensional constraint optimization problems subject to constraints on the boundary value and any-order derivative into unconstrained optimization problems. The number of applications of the Multivariate Theory of Connections are many, especially in the area of partial and stochastic differential equations: the main subjects of our current research.