Geometric Interpretation of Errors in Multi-Parametrical Fitting Methods Based on Non-Euclidean Norms

Abstract: The paper completes the multi-parametrical fitting methods, which are based on metrics induced by the non-Euclidean Lq-norms, by deriving the errors of the optimal parameter values. This was achieved using the geometric representation of the residuals sum expanded near its minimum, and the geometric interpretation of the errors. Typical fitting methods are mostly developed based on Euclidean norms, leading to the traditional least–square method. On the other hand, the theory of general fitting methods based on non-Euclidean norms is still under development; the normal equations provide implicitly the optimal values of the fitting parameters, while this paper completes the puzzle by improving understanding the derivations and geometric meaning of the optimal errors.


Introduction
The keys to evaluating an experimental result-e.g., compare it with the result anticipated by theories-require first the right selection of potential statistical tools and techniques for correctly processing and analyzing this result. This "processing and analyzing" involves two general types of approximation problems: One problem concerns a function fitting to given set of data. The other problem arises when a function is given analytically by an explicit mathematical type but we would like to find an alternative function with simpler form.
The widely used, traditional fitting method of least squares involves minimizing the sum of the squares of the residuals, i.e., the squares of the differences between the function f (x) and the approximating function that represents the statistical model, V(x). However, the least-square method is not unique. For instance, the absolute deviations minimization can also be applied. Generally, as soon as the desired norm of the metric space is given, the respective method of deviations minimization is defined. The least-square method is based on the Euclidean norm, while the alternative absolute deviations method is based on the uniform or Taxicab norm. In general, an infinite number of fitting methods can be defined, based on the metric space induced by the L q -norm; this case is studied here in detail.
Given the metric induced by the L q -norm, the functional of the total L q -normed residuals [7][8][9][10][11][12], noted also as total deviations (TD), between the fixed f (x) and the approximating V(x; p k ) functions in the domain D, is given by: Stats 2019, 2

427
The functional of total deviations, TD q ( p k ) q , is expanded (Taylor series) near its local minimum: where is the total deviation function at its global minimum, while is the Hessian matrix at this minimum, where all the components are positive, i.e., A 0 , By expanding the approximating function V(x; p k ) near the TD's minimum, [7] showed the following equations: and where The normal equations are given by where we set The purpose of this paper is to present the geometric interpretation of the errors of the optimal parameter values, derived from a multi-parametrical fitting, based on a metric induced by the non-Euclidean L q -norm. In Section 2, we derive the smallest possible value of the variation of the total deviations from its minimum, δTD, also called, the error of the total deviations value. In Section 3, we describe the geometric interpretation of the errors of the optimal parameter values, while in Section 4, we use this geometry to derive the exact equations that provide these errors. In Section 5, we apply the developed formulation for the 1-dim and 2-dim cases. Finally, Section 6 summarizes the conclusions.

The Error of the Total Deviation Values
The total deviations functional, TD a ( p k ) q , has a minimum value A 0 (q). The difference between these functionals cannot be arbitrarily small. Here we derive the smallest possible value of the variation of the total deviations from its minimum, δTD, also called, the error of the total deviations value.
First, we mention that the transition of the continuous to the discrete way for describing the values of x, can be realized as follows: while the expression of the total deviations is given by for large values of N, where L is the total length of the domain D, and the resolution of x-values is x res = L/N. In the discrete case, it is sufficient to express the total deviations simply by where we set Then, we calculate the error of the total deviations values, δTD, near the local minimum of TD q ( p k ) q , that is, for p k = p k * , ∀ k = 1, 2, . . . , n. Thus, In the case of a large number of sampling elements, we adopt the continuous description, i.e., and S(u) is the distribution of u-values in their domain D u , that is, since x-values are equidistributed in their domain D. Therefore, where the number of the sampling elements, N, can be varied by 1, thus δN = 1. Hence, where Moreover, we show that the far right part of Equation (12) is zero. Indeed: where and thus, we obtain: leading to the set of the following n normal equations: hence: Similarly, for the continuous way of x-values, we have: The result of Equation (20d) will be used in Section 4 on the expression of the optimal errors.

The Uncertainty Manifold
We define the deviation of the total deviations functional from its minimum, ∆TD ≡ TD q ( p k ) q − TD q ( p k * ) q > 0, which is expressed with the quadratic form: where we set δp k ≡ p k − p k * , ∀k = 1, . . . , n.
Given a particular value of ∆TD, each of these parameter deviations, e.g., the k-th component δp k , has a maximum value δp k,max . This maximum value δp k,max of each parameter deviation δp k , depends on the value of ∆TD. The smallest possible value of δp k,max is deduced when ∆TD also reaches its smallest value. The smallest possible value of δp k,max interprets the error δp k * of the optimal parameter values p k * , ∀k = 1, 2, . . . , n; this is achieved when the particular value ∆TD is given by the smallest possible value of a deviation from the TD's minimum, δTD. In Section 2, we showed that δTD equals: There are cases, where the total deviations value is subject to an experimental, reading, or any other type of a non-statistical error; this is, in general, called the resolution value T res . Then, the smallest possible value δTD is meaningful only when it stays above the threshold of T res ; in other words, δTD ≥ T res or, if A 0 /N ≤ T res , then δTD = T res . Hence, The quadratic form in Equation (21) is positive definite, and thus it defines an n-dimensional paraboloid (hypersurface with a local minimum) immersed into an (n+1)-dimensional space. The corresponding n + 1 axes are given by the n parameter deviations δp k n k=1 and the deviation ∆TD, describing thus, the (n+1)-dimensional space as where D ∆TD = ∆TD ≥ δTD > 0\∆TD ∈ is the domain of the deviation values, ∆TD. Given a fixed value of ∆TD, and that can be the value of the smallest deviation, i.e., δTD = ∆TD( δp k ), the set of the parameter deviations δp k n k=1 defines a locus of an n-dimensional ellipsoid, rotated with respect to the axes δp k n k=1 . This n-dimensional ellipsoid is bounded by the (n − 1)-dimensional locus of intersection between the n-dimensional paraboloid ∆TD = ∆TD( δp k ) and the n-dimensional hyperplane ∆TD = δTD.
The n-dimensional ellipsoid is called uncertainty manifold, denoted by U n , for short. This is a manifold with an edge, meaning thus, its boundary, denoted by ∂U n . In general, the edge of an n-dimensional manifold is an (n − 1)-dimensional manifold. Here, the edge ∂U n involves the (n − 1)-dimensional locus of intersection between the n-dimensional paraboloid ∆TD = ∆TD( δp k ) and the n-dimensional hyperplane ∆TD = δTD. The n-dimensional cuboid, which encloses the uncertainty manifold's edge ∂U n , is also a manifold with an edge and is denoted by Uc n . Its edge is an (n − 1)-dimensional manifold denoted by ∂Uc n .
For example, consider the case of two-parametrical approximating functions, V(x; p 1 , p 2 ). Then, the quadratic form of Equation (21) defines the two-dimensional paraboloid ∆TD = ∆TD(δp 1 , δp 2 ), immersed into the three-dimensional space with Cartesian axes given by (x ≡ δp 1 , y ≡ δp 2 , z ≡ ∆TD). The two-dimensional ellipsoid is defined by the space bounded by the locus δTD = ∆TD(δp 1 , δp 2 ), which is the intersection of the two-dimensional paraboloid ∆TD = ∆TD(δp 1 , δp 2 ) and the two-dimensional hyperplane ∆TD = δTD. For visualizing this example, see Figure 1. Next, we will use the concept of the hyper-dimensional uncertainty manifold to derive the expressions of the errors of the optimal parameter values.

Derivation of the Errors of the Optimal Parameter Values
The expressions of the errors of the optimal parameter values-or simply, optimal errors-are well-known in the case of the least-square and other Euclidean based fitting methods. In [7], we have used the error expression, which is caused by the curvature, in order to have an estimate of the optimal errors (for applications, see [11][12][13][14][15][16][17][18]). Here, we will see the formal geometric derivation of the optimal error expressions.
First, we note that the edge of the uncertainty manifold, U n  , has a number of n extrema, denoted by These components are given by the condition: ( )  The intersection between the paraboloid and a constant hyper-plane ∆TD = δTD is a rotated n-dimensional ellipsoid, or a rotated ellipsis for the case of n = 2, that is the uncertainty manifold ∂U 2 (enclosing U 2 ). The rectangular adjusted on the extrema of ∂U 2 denotes the manifold ∂Uc 2 (enclosing Uc 2 ).
Next, we will use the concept of the hyper-dimensional uncertainty manifold to derive the expressions of the errors of the optimal parameter values.

Derivation of the Errors of the Optimal Parameter Values
The expressions of the errors of the optimal parameter values-or simply, optimal errors-are well-known in the case of the least-square and other Euclidean based fitting methods. In [7], we have used the error expression, which is caused by the curvature, in order to have an estimate of the optimal errors (for applications, see [11][12][13][14][15][16][17][18]). Here, we will see the formal geometric derivation of the optimal error expressions.
First, we note that the edge of the uncertainty manifold, ∂U n , has a number of n extrema, denoted by C (k) n k=1 , and they are related to the errors of the parameters optimal values, δp k * n k=1 , as follows: The position vector → ∆ (µ) of the corresponding point C (µ) , ∀µ = 1, 2, . . . , n, consists of n components each, i.e., These components are given by the condition: ∀ν = 1, 2, . . . , n, with ν µ.

The Case of n = 1
Let us begin with the case of a one-dimensional paraboloid, given simply by the parabola corresponding to uni-parametrical approximating functions. The locus of intersection between this parabola and the line ∆TD = δTD (that is, the one-dimensional hyperplane) are the two points δp ± = ± δTD/A 2 (q). The uncertainty manifold U 1 is the one-dimensional ellipsoid, defined by the line segment δp − ≤ δp ≤ δp + , which is enclosed by the points δp ± . In this case, the edge of the uncertainty manifold ∂U 1 is restricted to the zero-dimensional space composed only by the two points δp ± . The manifolds U 1 and Uc 1 coincide (similarly with their edges, ∂U 1 and ∂Uc 1 , respectively). Hence, (32)

The Case of n = 2
The case of bi-parametrical approximating functions is characterized by the two-dimensional paraboloid, which is illustrated in Figure 1. The locus of intersection between this paraboloid and the plane ∆TD(δp 1 , δp 2 ) = δTD is given by the rotated ellipse: written suitably as after the rotation transformation where Then, where the diagonal matrix has the following elements which are the eigenvalues of the matrix A 2 (q). The ellipsis' major/minor axes in Equation (35) are: while the rotation angle θ in Equation (36) is given by The uncertainty manifold U 2 is the rotated 2-dim ellipsoid in the (δp 1 , δp 2 ) axes, defined by: or, in the rotated axes (δp 1 , δp 2 ), is simply given by: which is enclosed by the ellipse corresponding to the equal sign of Equation (45), that is the edge of the uncertainty manifold, ∂U 2 . Finally, the errors are and

Conclusions
The paper presented the geometric interpretation of the errors of the optimal parameter values, derived from a multi-parametrical fitting, based on a metric induced by the non-Euclidean L q -norm. Typical fitting methods are mostly developed based on Euclidean norms, leading to the traditional least-square method. On the other hand, the theory of general fitting methods based on non-Euclidean norms, is still under development; the normal equations can provide the optimal values of the fitting parameters, while this paper completed the puzzle by improving understanding the derivations and geometric meaning of the errors.
In particular, we showed that the statistical errors of the optimal parameter values are given by the axes of the ellipsoid called uncertainty manifold, that is, the intersection of the paraboloid of the residuals' expansion ∆TD( δp k ) ≡ TD q ( p k = p * k + δp k ) q − TD q ( p * k ) q along the deviations δp k n k=1 , with the hyperplane ∆TD( δp k ) = δTD = const. The constant δTD represents the smallest possible value of a deviation from the TD's minimum, also mentioned as an error of the value of the total deviations.
In summary, the L q -normed fitting involves minimizing: where The normal equations are given by: where we set u = u(x) ≡ V(x; p k * ) − f (x).
Finally, we summarize the concluding relationships of the paper: with special cases: -For n = 1: (50a) -For n = 2: and (50c) Funding: This research was funded by NASA's HGI Program, grant number NNX17AB74G.

Conflicts of Interest:
The author declares no conflict of interest.