Geometric Interpretation of Errors in Multi-Parametrical Fitting Methods Based on Non-Euclidean Norms

Abstract

The paper completes the multi-parametrical fitting methods, which are based on metrics induced by the non-Euclidean L_q-norms, by deriving the errors of the optimal parameter values. This was achieved using the geometric representation of the residuals sum expanded near its minimum, and the geometric interpretation of the errors. Typical fitting methods are mostly developed based on Euclidean norms, leading to the traditional least–square method. On the other hand, the theory of general fitting methods based on non-Euclidean norms is still under development; the normal equations provide implicitly the optimal values of the fitting parameters, while this paper completes the puzzle by improving understanding the derivations and geometric meaning of the optimal errors.

1. Introduction

The keys to evaluating an experimental result—e.g., compare it with the result anticipated by theories—require first the right selection of potential statistical tools and techniques for correctly processing and analyzing this result. This “processing and analyzing” involves two general types of approximation problems: One problem concerns a function fitting to given set of data. The other problem arises when a function is given analytically by an explicit mathematical type but we would like to find an alternative function with simpler form.

Let $V(x;p_1,p_2,\dots ,p_n)$, with $x\in D\subseteq \Re$ and $(p_1,p_2,\dots ,p_n)\in \{D_{p_1}\otimes D_{p_2}\otimes \dots \otimes D_{p_n}\}\subseteq {\Re }^n$, denote a multi-parametrical approximating function [1,2,3,4,5,6], symbolized as $V(x;\{p_k\})$, for short.

The widely used, traditional fitting method of least squares involves minimizing the sum of the squares of the residuals, i.e., the squares of the differences between the function $f(x)$ and the approximating function that represents the statistical model, $V(x)$. However, the least-square method is not unique. For instance, the absolute deviations minimization can also be applied. Generally, as soon as the desired norm of the metric space is given, the respective method of deviations minimization is defined. The least-square method is based on the Euclidean norm, while the alternative absolute deviations method is based on the uniform or Taxicab norm. In general, an infinite number of fitting methods can be defined, based on the metric space induced by the L_q-norm; this case is studied here in detail.

Given the metric induced by the L_q-norm, the functional of the total L_q-normed residuals [7,8,9,10,11,12], noted also as total deviations (TD), between the fixed $f(x)$ and the approximating $V(x;\{p_k\})$ functions in the domain D, is given by:

$$T{D}_q{\left(\left\{p_k\right\}\right)}^q={\displaystyle \underset{x\in D}{\int }{\left|V\left(x;\left\{p_k\right\}\right)-f(x)\right|}^{q}dx}.$$

(1)

The functional of total deviations, $T{D}_q{(\{p_k\})}^q$, is expanded (Taylor series) near its local minimum:

$$T{D}_q{(\{p_k\})}^q=A_0(q)+{\displaystyle \sum _{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot (p_{k_1}-p_{k_1}{}^{*})\cdot (p_{k_2}-p_{k_2}{}^{*})}+O\left({\left|p_k-p_k{}^{*}\right|}^3\right),$$

(2a)

where

$$A_0(q)=T{D}_q{(\{p_k{}^{*}\})}^q,$$

(2b)

is the total deviation function at its global minimum, while

$$A_{2,}{}_{k_1{k}_2}(q)=\frac{1}{2}\cdot {\frac{{\partial }^2}{\partial p_{k_1}\partial p_{k_2}}T{D}_q{(\{p_k\})}^q|}_{\left\{p_k\right\}=\left\{p_k^{*}\right\}},$$

(2c)

is the Hessian matrix at this minimum, where all the components are positive, i.e., $A_0,A_{2,}{}_{k_1{k}_2}\ge 0$, $\forall k_1,k_2=1,2,\dots ,n$.

By expanding the approximating function $V(x;\{p_k\})$ near the TD’s minimum, [7] showed the following equations:

$$A_0(q)={\displaystyle \underset{x\in D}{\int }{\left|u\right|}^{q}dx},$$

(3a)

and

$$\begin{array}{l}{A}_{2,}{}_{k_1{k}_2}(q)={\delta }_{1q}\cdot {\gamma }_{k_1{k}_2}+\\ \frac{q}{2}\cdot {\displaystyle \underset{x\in D}{\int }\left[{\left|u\right|}^{q-1}\cdot \mathrm{sgn}(u)\cdot \frac{{\partial }^{2}V(x;\{p_k{}^{*}\})}{\partial p_{k_1}\partial p_{k_2}}+(q-1)\cdot {\left|u\right|}^{q-2}\cdot \frac{\partial V(x;\{p_k{}^{*}\})}{\partial p_{k_1}}\cdot \frac{\partial V(x;\{p_k{}^{*}\})}{\partial p_{k_2}}\right]}dx,\end{array}$$

(3b)

where

$${\gamma }_{k_1{k}_2}\equiv {\displaystyle \sum _{\begin{array}{c}\forall i:\\ u(x_i)=0\end{array}}\frac{1}{\left|u^{\prime }(x_i)\right|}\cdot \frac{\partial V(x_i;\{p_k{}^{*}\})}{\partial p_{k_1}}\cdot \frac{\partial V(x_i;\{p_k{}^{*}\})}{\partial p_{k_2}}}.$$

(3c)

The normal equations are given by

$${\displaystyle \underset{x\in D}{\int }{\left|u\right|}^{q-1}\cdot \mathrm{sgn}(u)\cdot \frac{\partial V(x;\{p_k{}^{*}\})}{\partial p_l}dx}=0,\forall l=1,2,\dots ,n.$$

(4)

where we set $u=u(x)\equiv V(x;\{p_k{}^{*}\})-f(x)$ for short.

The purpose of this paper is to present the geometric interpretation of the errors of the optimal parameter values, derived from a multi-parametrical fitting, based on a metric induced by the non-Euclidean L_q-norm. In Section 2, we derive the smallest possible value of the variation of the total deviations from its minimum, $\delta TD$, also called, the error of the total deviations value. In Section 3, we describe the geometric interpretation of the errors of the optimal parameter values, while in Section 4, we use this geometry to derive the exact equations that provide these errors. In Section 5, we apply the developed formulation for the 1-dim and 2-dim cases. Finally, Section 6 summarizes the conclusions.

2. The Error of the Total Deviation Values

The total deviations functional, $T{D}_a{(\{p_k\})}^q$, has a minimum value $A_0(q)$. The difference between these functionals cannot be arbitrarily small. Here we derive the smallest possible value of the variation of the total deviations from its minimum, $\delta TD$, also called, the error of the total deviations value.

First, we mention that the transition of the continuous to the discrete way for describing the values of x, can be realized as follows:

$${\displaystyle \underset{a}{\overset{b}{\int }}(\cdots )dx}=\underset{N\to \infty }{\lim }\frac{b-a}{N}\cdot {\displaystyle \sum _{i=1}^N(\cdots )},$$

(5)

while the expression of the total deviations is given by

$$T{D}_q{(\{p_k\})}^q\text{}=\text{}{\displaystyle \underset{x\in D}{\int }{\left|V(x;\text{}\{p_k\})-\text{}f(x)\right|}^q\text{}dx}\approx \frac{L}{N}\cdot \text{}{\displaystyle \sum _{i=1}^N{\left|V(x_i;\text{}\{p_k\})-\text{}f(x_i)\right|}^q}\text{}=x_{res}\text{}\cdot \text{}{\displaystyle \sum _{i=1}^N{\left|V(x_i;\text{}\{p_k\})-\text{}f(x_i)\right|}^q},$$

(6)

for large values of N, where L is the total length of the domain D, and the resolution of x-values is $x_{res}=L/N$. In the discrete case, it is sufficient to express the total deviations simply by

$$T{D}_q{\left(\left\{p_k\right\}\right)}^q={\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q},$$

(7)

where we set $u_i\equiv u(x_i;\{p_k\})\equiv V(x_i;\{p_k\})-f(x_i)$, $V_i\equiv V(x_i;\{p_k\})$ and $y_i\equiv f(x_i)$.

Then, we calculate the error of the total deviations values, δTD, near the local minimum of $T{D}_q{(\{p_k\})}^q$, that is, for $\left\{p_k\right\}=\left\{p_k{}^{*}\right\}$, $\forall k=1,2,\dots ,n$. Thus,

$$\delta TD={\left[\delta T{D}_q{(\{p_k\})}^q\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}={\left[\delta {\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}.$$

(8)

In the case of a large number of sampling elements, we adopt the continuous description, i.e.,

$${\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\approx N\cdot \langle {\left|u\right|}^q\rangle ,\mathrm{with}\langle {\left|u\right|}^q\rangle ={\displaystyle \underset{u\in D_u}{\int }{\left|u\right|}^{q}S(u)du},$$

(9)

and $S(u)$ is the distribution of u-values in their domain $D_u$, that is,

$$\langle {\left|u\right|}^q\rangle =\frac{1}{L}\cdot {\displaystyle \underset{x\in D}{\int }{\left|V(x;\{p_k\})-f(x)\right|}^{q}dx},$$

(10)

since x-values are equidistributed in their domain D. Therefore,

$$\delta {\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\approx \delta N\cdot \langle {\left|u\right|}^q\rangle +N\cdot \delta \langle {\left|u\right|}^q\rangle =\langle {\left|u\right|}^q\rangle +N\cdot \delta \langle {\left|u\right|}^q\rangle \approx \frac{1}{N}{\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}+N\cdot \delta \langle {\left|u\right|}^q\rangle ,$$

(11)

where the number of the sampling elements, N, can be varied by 1, thus δN = 1. Hence,

$${\left[\delta {\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\approx \frac{1}{N}{\left[{\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}+N\cdot {\left[\delta \langle {\left|u\right|}^q\rangle \right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}},$$

(12)

where

$${\left[{\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\text{}=A_0(q).$$

(13)

Moreover, we show that the far right part of Equation (12) is zero. Indeed:

$$\delta \langle {\left|u\right|}^q\rangle =\frac{1}{L}\cdot \delta {\displaystyle \underset{x\in D}{\int }{\left|V(x;\left\{p_k\right\})-f(x)\right|}^{q}dx}=\frac{1}{L}\cdot {\displaystyle \sum _{k=1}^n{A}_{1,}{}_k(q)\delta p_k},$$

(14)

where

$$A_{1,}{}_k(q)=q\cdot \text{}{\displaystyle \underset{x\in D}{\int }{\left|V(x;\{p_k\})-f(x)\right|}^{q-1}sign[V(x;\left\{p_k\right\})-f(x)]\cdot \frac{\partial V(x;\{p_k\})}{\partial p_k}dx},$$

(15)

and thus, we obtain:

$${\left[\delta \langle {\left|u\right|}^q\rangle \right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}=\frac{1}{L}\cdot {\displaystyle \sum _{k=1}^n{A_{1,}{}_k(q)|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\delta p_k{}^{*}}=0,$$

(16)

leading to the set of the following n normal equations:

$${0=A_{1,}{}_k\text{}(q)|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\text{}=q\cdot \text{}{\displaystyle \underset{x\in D}{\int }{\left|V(x;\text{}\{p_k{}^{*}\})-\text{}f\text{}(x)\right|}^{q-1}\text{}sign\left[V(x;\text{}\{p_k{}^{*}\})-\text{}f\text{}(x)\right]\cdot {\frac{\partial V\text{}(x;\text{}\{p_k\})}{\partial p_k}|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\text{}dx}.$$

(17)

Finally,

$${\left[\delta {\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}=\frac{1}{N}{A}_0(q),$$

(18)

hence:

$$\delta TD(q)=\frac{1}{N}{A}_0(q).$$

(19)

Similarly, for the continuous way of x-values, we have:

$$T{D}_q{(\{p_k\})}^q={\displaystyle \underset{x\in D}{\int }{\left|u\right|}^{q}dx}\approx x_{res}\cdot {\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q},$$

(20a)

$$\delta TD\approx x_{res}\cdot {\left[\delta {\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\approx \frac{1}{N}{x}_{res}\cdot {\left[{\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}=\frac{1}{N}{A}_0(q),$$

(20b)

$$A_0(q)={\left[{\displaystyle \underset{x\in D}{\int }{\left|u\right|}^{q}dx}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}}\approx x_{res}\cdot {\left[{\displaystyle \sum _{i=1}^N{\left|u_i\right|}^q}\right]|}_{\left\{p_k\right\}=\left\{p_k{}^{*}\right\}},$$

(20c)

$$\delta TD(q)=\frac{1}{N}{A}_0(q)\approx \frac{x_{res}}{L}{A}_0(q).$$

(20d)

The result of Equation (20d) will be used in Section 4 on the expression of the optimal errors.

3. The Uncertainty Manifold

We define the deviation of the total deviations functional from its minimum, $\Delta TD\equiv T{D}_q{(\{p_k\})}^q-T{D}_q{(\{p_k{}^{*}\})}^q>0$, which is expressed with the quadratic form:

$$\Delta TD\left(\left\{\delta p_k\right\}\right)={\displaystyle \sum _{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot \delta p_{k_1}\delta p_{k_2}},$$

(21)

where we set $\delta p_k\equiv p_k-p_k{}^{*}$, $\forall k=1,\dots ,n$.

Given a particular value of $\Delta TD$, each of these parameter deviations, e.g., the k-th component $\delta p_k$, has a maximum value $\delta p_{k,}{}_{\max }$. This maximum value $\delta p_{k,}{}_{\max }$ of each parameter deviation $\delta p_k$, depends on the value of $\Delta TD$. The smallest possible value of $\delta p_{k,}{}_{\max }$ is deduced when $\Delta TD$ also reaches its smallest value. The smallest possible value of $\delta p_{k,}{}_{\max }$ interprets the error $\delta p_k{}^{*}$ of the optimal parameter values $p_k{}^{*}$, $\forall k=1,2,\dots ,n$; this is achieved when the particular value $\Delta TD$ is given by the smallest possible value of a deviation from the TD’s minimum, $\delta TD$. In Section 2, we showed that $\delta TD$ equals:

$$\delta TD(q)=\frac{1}{N}\cdot A_0(q).$$

(22)

There are cases, where the total deviations value is subject to an experimental, reading, or any other type of a non-statistical error; this is, in general, called the resolution value $T_{res}$. Then, the smallest possible value $\delta TD$ is meaningful only when it stays above the threshold of $T_{res}$; in other words, $\delta TD\ge T_{res}$ or, if $A_0/N\le T_{res}$, then $\delta TD=T_{res}$. Hence,

$$\delta TD(q)=\{\begin{array}{l}\frac{1}{N}\cdot A_0(q),\mathrm{if}\frac{A_0}{N}>T_{res},\\ T_{res},\mathrm{if}\frac{A_0}{N}\le T_{res}.\end{array}$$

(23)

The quadratic form in Equation (21) is positive definite, and thus it defines an n-dimensional paraboloid (hypersurface with a local minimum) immersed into an (n+1)-dimensional space. The corresponding n + 1 axes are given by the n parameter deviations ${\left\{\delta p_k\right\}}_{k=1}^n$ and the deviation $\Delta TD$, describing thus, the (n+1)-dimensional space as

$$(\delta p_1,\delta p_2,\dots ,\delta p_n;\Delta TD)\in \{D_{p_1}\otimes D_{p_2}\otimes \dots \otimes D_{p_n}\}\otimes D_{\Delta TD}\subseteq {\Re }^{n+1},$$

(24)

where $D_{\Delta TD}=\left\{\Delta TD\ge \delta TD>0\backslash \Delta TD\in \Re \right\}$ is the domain of the deviation values, $\Delta TD$.

Given a fixed value of $\Delta TD$, and that can be the value of the smallest deviation, i.e., $\delta TD=\Delta TD(\{\delta p_k\})$, the set of the parameter deviations ${\left\{\delta p_k\right\}}_{k=1}^n$ defines a locus of an n-dimensional ellipsoid, rotated with respect to the axes ${\left\{\delta p_k\right\}}_{k=1}^n$. This n-dimensional ellipsoid is bounded by the (n − 1)-dimensional locus of intersection between the n-dimensional paraboloid $\Delta TD=\Delta TD(\{\delta p_k\})$ and the n-dimensional hyperplane $\Delta TD=\delta TD$.

The n-dimensional ellipsoid is called uncertainty manifold, denoted by ${\mathrm{U}}_n$, for short. This is a manifold with an edge, meaning thus, its boundary, denoted by $\partial {\mathrm{U}}_n$. In general, the edge of an n-dimensional manifold is an (n − 1)-dimensional manifold. Here, the edge $\partial {\mathrm{U}}_n$ involves the (n − 1)-dimensional locus of intersection between the n-dimensional paraboloid $\Delta TD=\Delta TD(\{\delta p_k\})$ and the n-dimensional hyperplane $\Delta TD=\delta TD$. The n-dimensional cuboid, which encloses the uncertainty manifold’s edge $\partial {\mathrm{U}}_n$, is also a manifold with an edge and is denoted by ${\mathrm{Uc}}_n$. Its edge is an (n − 1)-dimensional manifold denoted by $\partial {\mathrm{Uc}}_n$.

For example, consider the case of two-parametrical approximating functions, $V(x;p_1,p_2)$. Then, the quadratic form of Equation (21) defines the two-dimensional paraboloid $\Delta TD=\Delta TD(\delta p_1,\delta p_2)$, immersed into the three-dimensional space with Cartesian axes given by $(x\equiv \delta p_1,y\equiv \delta p_2,z\equiv \Delta TD)$. The two-dimensional ellipsoid is defined by the space bounded by the locus $\delta TD=\Delta TD\left(\delta p_1,\delta p_2\right)$, which is the intersection of the two-dimensional paraboloid $\Delta TD=\Delta TD\left(\delta p_1,\delta p_2\right)$ and the two-dimensional hyperplane $\Delta TD=\delta TD$. For visualizing this example, see Figure 1.

Next, we will use the concept of the hyper-dimensional uncertainty manifold to derive the expressions of the errors of the optimal parameter values.

4. Derivation of the Errors of the Optimal Parameter Values

The expressions of the errors of the optimal parameter values—or simply, optimal errors—are well-known in the case of the least-square and other Euclidean based fitting methods. In [7], we have used the error expression, which is caused by the curvature, in order to have an estimate of the optimal errors (for applications, see [11,12,13,14,15,16,17,18]). Here, we will see the formal geometric derivation of the optimal error expressions.

First, we note that the edge of the uncertainty manifold, $\partial {\mathrm{U}}_n$, has a number of n extrema, denoted by ${\left\{C^{(k)}\right\}}_{k=1}^n$, and they are related to the errors of the parameters optimal values, ${\left\{\delta p_k{}^{*}\right\}}_{k=1}^n$, as follows:

The position vector ${\overrightarrow{\Delta }}^{(\mu )}$ of the corresponding point $C^{(\mu )}$, $\forall \mu =1,2,\dots ,n$, consists of n components each, i.e., ${\overrightarrow{\Delta }}^{(\mu )}=({\Delta }^{(\mu )}{}_1,{\Delta }^{(\mu )}{}_2,\dots ,{\Delta }^{(\mu )}{}_n)$. Thus,

$$\begin{array}{l}{C}^{(1)}:\left({\Delta }^{(1)}{}_1,{\Delta }^{(1)}{}_2,\dots ,{\Delta }^{(1)}{}_n\right)={\overrightarrow{\Delta }}^{(1)}\\ C^{(2)}:\left({\Delta }^{(2)}{}_1,{\Delta }^{(2)}{}_2,\dots ,{\Delta }^{(2)}{}_n\right)={\overrightarrow{\Delta }}^{(2)}\\ \cdots \\ C^{(n)}:\left({\Delta }^{(n)}{}_1,{\Delta }^{(n)}{}_2,\dots ,{\Delta }^{(n)}{}_n\right)={\overrightarrow{\Delta }}^{(n)}.\end{array}$$

(25)

These components are given by the condition:

$${\left\{\frac{\partial \Delta TD\left(\left\{\delta p_k\right\}\right)}{\partial \delta p_{\nu }}\right\}|}_{\left\{\delta p_k\right\}=\left\{{\Delta }^{(\mu )}{}_k\right\}}\text{}={\left\{\frac{\partial }{\partial \delta p_{\nu }}{\displaystyle \sum _{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot \delta p_{k_1}\cdot \delta p_{k_2}}\right\}|}_{\left\{\delta p_k\right\}=\left\{{\Delta }^{(\mu )}{}_k\right\}}=0,$$

(26)

$\forall \nu =1,2,\dots ,n$, with $\nu \ne \mu$.

The above (n − 1) equations, given in Equation (26), together with

$$\delta TD=\Delta TD(\{\delta p_k\}=\{{\Delta }^{(\mu )}{}_k\})={\displaystyle \sum _{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot {\Delta }^{(\mu )}{}_{k_1}{\Delta }^{(\mu )}{}_{k_2}},$$

(27)

are sufficient for the calculation of the n unknown components of ${\overrightarrow{\Delta }}^{(\mu )}=({\Delta }^{(\mu )}{}_1,{\Delta }^{(\mu )}{}_2,\dots ,{\Delta }^{(\mu )}{}_n)$.

The (n − 1) equations, given in Equation (26), arise from the fact that each of the errors of parameters optimal values, i.e., the k-th, $\delta p_k{}^{*}$ is derived from the maximum value of the corresponding component $\delta p_k$, that is, $\delta p_{k,}{}_{\max }$. Hence, the existence of maximum values ${\left\{\delta p_k{}^{*}\right\}}_{k=1}^n$, i.e., $\forall k=1,2,\dots ,n$, $\exists \delta p_k{}^{*}\in {\mathrm{U}}_n$: $\forall \delta p_k\in {\mathrm{U}}_n$, $\delta p_k{}^{*}\ge \delta p_k$, leads to the errors estimation, ${\left\{\delta p_k{}^{*}\right\}}_{k=1}^n={\left\{\delta p_{k,}{}_{\max }\right\}}_{k=1}^n$. These maximum values, are located on the edge of the uncertainty manifold $\partial {\mathrm{U}}_n$, that is the hypersurface $\delta TD={\displaystyle {\sum }_{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot \delta p_{k_1}\delta p_{k_2}}$, i.e., $\exists \delta p_k{}^{*}\in \partial {\mathrm{U}}_n$.

The maximization of the μ-th parameter deviation $\delta p_{\mu }$ within uncertainty manifold is derived as follows: Since $\delta TD=\Delta TD(\{\delta p_k\})$, we can express $\delta p_{\mu }$ in terms of $\delta p_{\nu }$, $\forall \nu =1,2,\dots ,n$ with $\nu \ne \mu$, i.e., $\delta p_{\mu }=\delta p_{\mu }({\left\{\delta p_k\right\}}_{k=1,k\ne \mu }^n)$. Then, the procedure of finding the maximum value of $\delta p_{\mu }$, that is, $\delta p_{\mu }{}^{*}={\delta p_{\mu }|}_{\max }$, involves finding all the derivatives $\partial /\partial \delta p_{\nu }$, $\forall \nu =1,2,\dots ,n$ with $\nu \ne \mu$ of $\delta p_{\mu }=\delta p_{\mu }({\left\{\delta p_k\right\}}_{k=1,k\ne \mu }^n)$, or equivalently, of $\delta TD=\Delta TD(\{\delta p_k\})$ (implicit derivatives). The (n − 1) equations $\partial \delta p_{\mu }({\left\{\delta p_k\right\}}_{k=1,k\ne \mu }^n)/\partial \delta p_{\nu }=0$ lead to the (n − 1) relationships $\delta p_{\nu }=\delta p_{\nu }(\delta p_{\mu })$, $\forall \nu =1,2,\dots ,n$ with $\nu \ne \mu$, which together with $\delta p_{\mu }=\delta p_{\mu }({\left\{\delta p_k\right\}}_{k=1,k\ne \mu }^n)$, leads to the specific values of ${\{{\Delta }^{(\mu )}{}_k=\delta p_k\}}_{k=1}^n$. Yet, only the μ-th component ${\Delta }^{(\mu )}{}_{\mu }=\delta p_{\mu }$ gives the error $\delta p_{\mu }{}^{*}={\delta p_{\mu }|}_{\max }$, i.e.,

$$\delta p_{\mu }{}^{*}={\Delta }^{(\mu )}{}_{\mu }.$$

(28)

In Appendix A, we solve Equations (26) and (27), where, we concluded that

$${\Delta }^{(m_1)}{}_{m_2}=\frac{{\sigma }^2{}_{m_1{m}_2}}{\sqrt{{\sigma }^2{}_{m_1{m}_1}}}=\sqrt{\delta TD}\cdot \frac{{\left(A_2{(q)}^{-1}\right)}_{m_1{m}_2}}{\sqrt{{\left(A_2{(q)}^{-1}\right)}_{m_1{m}_1}^{}}},$$

(29a)

or

$${\Delta }^{(m_1)}{}_{m_2}=\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)}\cdot \frac{{\left(A_2{(q)}^{-1}\right)}_{m_1{m}_2}}{\sqrt{{\left(A_2{(q)}^{-1}\right)}_{m_1{m}_1}^{}}},\mathrm{if}\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}}\cdot \frac{{\left(A_2{(q)}^{-1}\right)}_{m_1{m}_2}}{\sqrt{{\left(A_2{(q)}^{-1}\right)}_{m_1{m}_1}^{}}},\mathrm{if}\frac{A_0}{N}\le T_{res},\end{array}$$

(29b)

and thus, from Equation (28), we finally derive the errors:

$$\delta p_k{}^{*}={\Delta }^{(k)}{}_k=\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}^{}}=\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}},\frac{A_0}{N}\le T_{res},\end{array}$$

(30a)

$\forall k=1,2,\dots ,n$.

Finally, taking into account the resolution of each parameter value, ${\left\{p_{res,}{}_k\right\}}_{k=1}^n$, we have:

$$\delta p_k^{*}=Max\left(p_{res,}{}_k,\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}^{}}\right),$$

(30b)

where

$$\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}}=\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}},\mathrm{if}\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}},\mathrm{if}\frac{A_0}{N}\le T_{res},\end{array}$$

(30c)

$\forall k=1,2,\dots ,n$.

5. Formulation of the Cases of n = 1 and n = 2 Dimensional Uncertainty Manifold

5.1. The Case of n = 1

Let us begin with the case of a one-dimensional paraboloid, given simply by the parabola

$$\Delta TD\left(\delta p\right)=A_2(q)\cdot \delta p^2,$$

(31)

corresponding to uni-parametrical approximating functions. The locus of intersection between this parabola and the line $\Delta TD=\delta TD$ (that is, the one-dimensional hyperplane) are the two points $\delta p_{\pm }=\pm \sqrt{\delta TD/A_2(q)}$. The uncertainty manifold ${\mathrm{U}}_1$ is the one-dimensional ellipsoid, defined by the line segment $\delta p_{-}\le \delta p\le \delta p_{+}$, which is enclosed by the points $\delta p_{\pm }$. In this case, the edge of the uncertainty manifold $\partial {\mathrm{U}}_1$ is restricted to the zero-dimensional space composed only by the two points $\delta p_{\pm }$. The manifolds ${\mathrm{U}}_1$ and ${\mathrm{Uc}}_1$ coincide (similarly with their edges, $\partial {\mathrm{U}}_1$ and $\partial {\mathrm{Uc}}_1$, respectively). Hence,

$$\delta p^{*}=\sqrt{\frac{\delta TD(q)}{A_2(q)}}=\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)/A_2(q)},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}/A_2(q)},\frac{A_0}{N}\le T_{res}.\end{array}$$

(32)

5.2. The Case of n = 2

The case of bi-parametrical approximating functions is characterized by the two-dimensional paraboloid,

$$\Delta TD\left(\delta p_1,\delta p_2\right)=A_{11}(q)\cdot \delta p_1{}^2+2A_{12}(q)\cdot \delta p_1\delta p_2+A_{22}(q)\cdot \delta p_2{}^2,$$

(33)

which is illustrated in Figure 1. The locus of intersection between this paraboloid and the plane $\Delta TD(\delta p_1,\delta p_2)=\delta TD$ is given by the rotated ellipse:

$$\delta TD=A_{11}(q)\cdot \delta p_1{}^2+2A_{12}(q)\cdot \delta p_1\delta p_2+A_{22}(q)\cdot \delta p_2{}^2,$$

(34)

written suitably as

$${\left(\frac{\delta {p^{\prime }}_1}{b_1}\right)}^2+{\left(\frac{\delta {p^{\prime }}_2}{b_2}\right)}^2=1,$$

(35)

after the rotation transformation

$$\left(\begin{array}{c}\delta p_1\\ \delta p_2\end{array}\right)=\left(\begin{array}{cc}\cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{c}\delta {p^{\prime }}_1\\ \delta {p^{\prime }}_2\end{array}\right),\mathrm{or}$$

(36)

$$\overrightarrow{\delta p}=R\overrightarrow{\delta p^{\prime }},$$

(37)

where

$$R=R\left(\theta \right)=\left(\begin{array}{cc}\cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{array}\right),\overrightarrow{\delta p}\equiv \left(\begin{array}{c}\delta p_1\\ \delta p_2\end{array}\right),\overrightarrow{\delta p^{\prime }}\equiv \left(\begin{array}{c}\delta {p^{\prime }}_1\\ \delta {p^{\prime }}_2\end{array}\right).$$

(38)

Then,

$$\delta TD={\overrightarrow{\delta p}}^t{A}_2\overrightarrow{\delta p}=\overrightarrow{\delta p^{\prime }}{}^t(R^t{A}_{2}R)\overrightarrow{\delta p^{\prime }}=\overrightarrow{\delta p^{\prime }}{}^{t}B\overrightarrow{\delta p^{\prime }},$$

(39)

where the diagonal matrix

$$B=B(q)=\left(\begin{array}{cc}{B}_{11}(q)& 0\\ 0& B_{22}(q)\end{array}\right),$$

(40)

has the following elements

$$B_{\pm }=\frac{1}{2}\left[A_{11}+A_{22}\pm \sqrt{{\left(A_{11}-A_{22}\right)}^2+4A_{12}{}^2}\right],B_{11}=B_{+},B_{22}=B_{-},$$

(41)

which are the eigenvalues of the matrix $A_2(q)$. The ellipsis’ major/minor axes in Equation (35) are:

$$b_1=\sqrt{\frac{\delta TD}{B_{11}}},b_2=\sqrt{\frac{\delta TD}{B_{22}}}.$$

(42)

while the rotation angle θ in Equation (36) is given by

$$\tan 2\theta =\frac{2A_{12}}{A_{22}-A_{11}}.$$

(43)

The uncertainty manifold ${\mathrm{U}}_2$ is the rotated 2-dim ellipsoid in the $(\delta p_1,\delta p_2)$ axes, defined by:

$$A_{11}(q)\cdot \delta p_1{}^2+2A_{12}(q)\cdot \delta p_1\delta p_2+A_{22}(q)\cdot \delta p_2{}^2\le \delta TD,$$

(44)

or, in the rotated axes $(\delta {p^{\prime }}_1,\delta {p^{\prime }}_2)$, is simply given by:

$${\left(\frac{\delta {p^{\prime }}_1}{b_1}\right)}^2+{\left(\frac{\delta {p^{\prime }}_2}{b_2}\right)}^2\le 1,$$

(45)

which is enclosed by the ellipse corresponding to the equal sign of Equation (45), that is the edge of the uncertainty manifold, $\partial {\mathrm{U}}_2$.

Finally, the errors are

$$\begin{array}{c}\delta p_1^{*}=\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{11}}=\sqrt{\delta TD(q)\cdot \frac{{(A_2)}_{22}(q)}{D_{A_2}(q)}}\\ =\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)\cdot \frac{{(A_2)}_{22}(q)}{D_{A_2}\left(q\right)}},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}\cdot \frac{{(A_2)}_{22}(q)}{D_{A_2}(q)}},\frac{A_0}{N}\le T_{res},\end{array}\end{array}$$

(46)

and

$$\begin{array}{c}\delta p_2^{*}=\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{22}}=\sqrt{\delta TD(q)\cdot \frac{{(A_2)}_{11}(q)}{D_{A_2}(q)}}\\ =\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)\cdot \frac{{(A_2)}_{11}(q)}{D_{A_2}(q)}},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}\cdot \frac{{(A_2)}_{11}(q)}{D_{A_2}(q)}},\frac{A_0}{N}\le T_{res}.\end{array}\end{array}$$

(47)

6. Conclusions

The paper presented the geometric interpretation of the errors of the optimal parameter values, derived from a multi-parametrical fitting, based on a metric induced by the non-Euclidean L_q-norm. Typical fitting methods are mostly developed based on Euclidean norms, leading to the traditional least–square method. On the other hand, the theory of general fitting methods based on non-Euclidean norms, is still under development; the normal equations can provide the optimal values of the fitting parameters, while this paper completed the puzzle by improving understanding the derivations and geometric meaning of the errors.

In particular, we showed that the statistical errors of the optimal parameter values are given by the axes of the ellipsoid called uncertainty manifold, that is, the intersection of the paraboloid of the residuals’ expansion $\Delta TD(\{\delta p_k\})\equiv T{D}_q{(\{p_k=p_k^{*}+\delta p_k\})}^q-T{D}_q{(\{p_k^{*}\})}^q$ along the deviations ${\left\{\delta p_k\right\}}_{k=1}^n$, with the hyperplane $\Delta TD\left(\left\{\delta p_k\right\}\right)=\delta TD=const.$ The constant $\delta TD$ represents the smallest possible value of a deviation from the TD’s minimum, also mentioned as an error of the value of the total deviations.

In summary, the L_q-normed fitting involves minimizing:

$$T{D}_q{(\{p_k\})}^q=A_0(q)+{\displaystyle \sum _{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot (p_{k_1}-p_{k_1}{}^{*})\cdot (p_{k_2}-p_{k_2}{}^{*})}+O\left({\left|p_k-p_k{}^{*}\right|}^3\right),$$

(48a)

where

$$A_0(q)={\displaystyle \underset{x\in D}{\int }{\left|u\right|}^{q}dx},$$

(48b)

$$\begin{array}{l}{A}_{2,}{}_{k_1{k}_2}(q)={\delta }_{1q}\cdot {\gamma }_{k_1{k}_2}+\\ \frac{q}{2}\cdot {\displaystyle \underset{x\in D}{\int }\left[{\left|u\right|}^{q-1}\cdot \mathrm{sgn}(u)\cdot \frac{{\partial }^{2}V(x;\{p_k{}^{*}\})}{\partial p_{k_1}\partial p_{k_2}}+(q-1)\cdot {\left|u\right|}^{q-2}\cdot \frac{\partial V(x;\{p_k{}^{*}\})}{\partial p_{k_1}}\cdot \frac{\partial V(x;\{p_k{}^{*}\})}{\partial p_{k_2}}\right]}dx,\end{array}$$

(48c)

$${\gamma }_{k_1{k}_2}\equiv {\displaystyle \sum _{\begin{array}{c}\forall i:\\ u(x_i)=0\end{array}}\frac{1}{\left|u^{\prime }(x_i)\right|}\cdot \frac{\partial V(x_i;\{p_k{}^{*}\})}{\partial p_{k_1}}\cdot \frac{\partial V(x_i;\{p_k{}^{*}\})}{\partial p_{k_2}}}.$$

(48d)

The normal equations are given by:

$${\displaystyle \underset{x\in D}{\int }{\left|u\right|}^{q-1}\cdot \mathrm{sgn}(u)\cdot \frac{\partial V(x;\{p_k{}^{*}\})}{\partial p_l}dx}=0,\forall l=1,2,\dots ,n,$$

(48e)

where we set $u=u(x)\equiv V(x;\{p_k{}^{*}\})-f(x)$.

Finally, we summarize the concluding relationships of the paper:

$$\Delta TD\left(\left\{\delta p_k\right\}\right)\equiv T{D}_q{\left(\{p_k=p_k{}^{*}+\delta p_k\}\right)}^q-T{D}_q{\left(\left\{p_k{}^{*}\right\}\right)}^q={\displaystyle \sum _{k_1,k_2=1}^n{A}_{2,}{}_{k_1{k}_2}(q)\cdot \delta p_{k_1}\delta p_{k_2}},$$

(49a)

$$\Delta T{D}_{\min }\equiv Max\left(\delta TD(q),T_{res}\right),\delta TD(q)=\frac{1}{N}\cdot A_0(q),A_0(q)\equiv T{D}_q{\left(\left\{p_k{}^{*}\right\}\right)}^q,$$

(49b)

$$\delta p_k^{*}=Max\left(p_{res,}{}_k,\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{kk}^{}}\right),$$

(49c)

with special cases:

- For n = 1:

$$\delta p^{*}=\sqrt{\frac{\delta TD(q)}{A_2(q)}}=\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)/A_2(q)},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}/A_2(q)},\frac{A_0}{N}\le T_{res}.\end{array}$$

(50a)

- For n = 2:

$$\begin{array}{c}\delta p_1^{*}=\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{11}}=\sqrt{\delta TD(q)\cdot \frac{{(A_2)}_{22}(q)}{D_{A_2}(q)}}\\ =\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)\cdot \frac{{(A_2)}_{22}(q)}{D_{A_2}\left(q\right)}},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}\cdot \frac{{(A_2)}_{22}(q)}{D_{A_2}(q)}},\frac{A_0}{N}\le T_{res},\end{array}\end{array}$$

(50b)

and

$$\begin{array}{c}\delta p_2^{*}=\sqrt{\delta TD(q)\cdot {\left(A_2{(q)}^{-1}\right)}_{22}}=\sqrt{\delta TD(q)\cdot \frac{{(A_2)}_{11}(q)}{D_{A_2}(q)}}\\ =\{\begin{array}{l}\sqrt{\frac{1}{N}\cdot A_0(q)\cdot \frac{{(A_2)}_{11}(q)}{D_{A_2}(q)}},\frac{A_0}{N}>T_{res},\\ \sqrt{T_{res}\cdot \frac{{(A_2)}_{11}(q)}{D_{A_2}(q)}},\frac{A_0}{N}\le T_{res}.\end{array}\end{array}$$

(50c)