Sensitivity Analysis for Gaussian-Associated Features

Abstract

This paper is concerned with the evaluation of the uncertainties associated with Gaussian-associated features following the GUM methodology. We show how sensitivity matrices necessary for a GUM uncertainty evaluation can be calculated and how the variance matrices associated with the feature parameters can be estimated for a range of complete and partial features common in engineering. Example results are given in tables that allow practitioners to estimate, a priori, the uncertainties associated with fitted parameters, given a proposed measurement strategy for the case in which the point-cloud variance matrix is a multiple of the identity matrix. The sensitivity matrices can be used to evaluate the uncertainties for associated features for more general point-cloud variance matrices. All the calculations involved are direct and involve no optimization or Monte Carlo sampling; they can be implemented in spreadsheet software, for example.

1. Introduction

Coordinate metrology is a key technology supporting the quality infrastructure associated with manufacturing; see e.g., [1] and the references therein. Coordinate metrology can be thought of as a two-stage process, the first stage using a coordinate measuring machine (CMM) to gather coordinate data ${\mathit{x}}_{1:m}=\{{\mathit{x}}_i,i=1,\dots ,m\}$, related to a workpiece surface, the second extracting a set of parameters (features, characteristics) $\mathit{a}={(a_1,\dots ,a_n)}^{\top }$ from the data ${\mathit{x}}_{1:m}$ using software implementing mathematical algorithms, e.g., determining the parameters associated with the best-fit cylinder to data. The extracted parameters can then be compared with the workpiece design to assess whether or not the manufactured workpiece conforms to design within a pre-specified tolerance [2,3,4,5,6,7]. The evaluation of the uncertainties associated with geometric features a derived from coordinate data ${\mathit{x}}_{1:m}$ is also a two-stage process, the first in which a $3m\times 3m$ variance matrix $V_X$ associated with the coordinate data is evaluated [8,9,10,11,12,13,14], the second stage in which the uncertainties associated with ${\mathit{x}}_{1:m}$ are propagated through to those for the features a derived from $x_{1:m}$. This paper is concerned with the second stage, the evaluation and analysis of the variance matrix $V_A$ for Gaussian-associated features [15,16,17] determined from a least-squares fit of a geometric element to coordinate data.

Coordinate metrology is different from many other areas of metrology in that the measurands are usually multivariate, for example, a set of point coordinates, or are derived from multivariate quantities, e.g., the radius of a cylinder associated with a set of coordinates. The GUM methodology [18,19,20] involves an input-output model in which the measurand(s) a are described as having a functional relationship $\mathit{a}=\mathit{f}\left(\mathit{x}\right)$ on a set of inputs or influence factors x. Any statistical characterization of the influence factors x defines a corresponding statistical characterization of the outputs a. In particular, if x is associated with a (multivariate) probability distribution with mean $\widehat{\mathit{x}}$ and variance matrix $V_X$, the mean $\widehat{\mathit{a}}$ and variance matrix $V_A$ associated with a are completely defined by the functional relationship $\mathit{a}=\mathit{f}\left(\mathit{x}\right)$. If f is a nonlinear function of x, the mean and variance associated with a may be difficult to compute exactly but can be approximated by linearizing f about $\widehat{\mathit{a}}$. If $G_{A|X}$ is the sensitivity matrix (the symbol $A|X$ can be read as ‘A given X’) of a with respect to x,

$$G_{A|X}(i,j)=\frac{\partial f_i}{\partial x_j},$$

then the law of propagation of uncertainty (LPU, [20,21]) states that $\widehat{\mathit{a}}$ and $V_A$ are approximated by

$$\widehat{\mathit{a}}\approx \mathit{f}\left(\widehat{\mathit{x}}\right),V_A\approx G_{A|X}{V}_X{G}_{A|X}^{\top },$$

(1)

a multivariate version of the well-known formula used in the GUM. The standard uncertainties $\mathit{u}\left(\mathit{a}\right)$ associated $\widehat{\mathit{a}}$ are given by the square roots of the diagonal elements of $V_A$. The key to implementing a GUM approach is the evaluation of the sensitivity matrices $G_{A|X}$, the main subject of this paper. These sensitivity matrices depend on the measurement strategy as represented by the (nominal) points ${\mathit{x}}_{1:m}$ being measured and this paper discusses how $G_{A|X}$ can be calculated for the geometric elements common in engineering. The uncertainties associated with the features depend mostly on the geometry of the patch of geometric surface being measured rather than on the precise details of where the measuring points are located on the patch, so long as the distribution of the points is representative of the patch. In this paper, we use tools borrowed from Monte Carlo integration [22] to estimate uncertainties under the assumption that the proposed measurement strategy is approximately equivalent to a strategy in which a finite set of points is dispersed evenly over the surface being sampled, enabling the variance matrix to be calculated analytically. This allows us to estimate uncertainties without knowing the measurement strategy, only the number of data points and geometry of the area being sampled. The approach is used to examine the effect of sampling only a portion of a geometric element, an arc of a circle, a cap of a sphere, etc., and provide asymptotic results for limiting cases, e.g., as the angle of arc of a circle measured tends to zero. Example uncertainty estimates for different partial features of circles, spheres, planes, cylinders and cones are given in tables.

The methods described in this paper are all based on deriving the sensitivity matrix $G_{A|X}$ that is used to construct the variance matrix $V_A$ for the derived features a from the variance matrix $V_X$ associated with the point cloud ${\mathit{x}}_{1:m}$. For the features involved, the functional relationship $\mathit{a}=\mathit{f}\left({\mathit{x}}_{1:m}\right)$ of a on the point cloud is smooth and almost linear so that the first order approximation of f and the application of the law of propagation of uncertainty (1) is very effective in estimating uncertainties associated with the derived features. For features derived according to Chebyshev/minimum zone and related criteria, the functional relationship is not smooth and the first order approximation of f might not be fit for purpose. An alternative approach is to use a Monte Carlo sampling approach [19] generating point-cloud data sets ${\mathit{x}}_{1:m,q}$ and derived features ${\mathit{a}}_q={\mathit{a}}_q\left({\mathit{x}}_{1:m,q}\right)$ for each data set, $q=1,\dots ,M$. The variance matrix associated with the sample ${\mathit{a}}_{1:M}$ is an approximation to $V_A$. The Monte Carlo approach is the basis of the virtual CMM approach to uncertainty evaluation [14,23]. Our focus here on features derived according to the least-squares criterion and avoids the requirement for Monte Carlo sampling approaches.

The remainder of this paper is organized as follows. Section 2 provides some preliminary calculations of sensitivity matrices associated with an axis, e.g., the axis of a cylinder. Section 3 describes in general how uncertainties associated with point clouds are propagated through to Gaussian-associated features. Section 4, Section 5, Section 6, Section 7 and Section 8 applies the general approach to different geometric elements. Each section describes how sensitivity matrices for least-squares element fitting can be evaluated and how the variance matrices for the associated features can be approximated, for the case in which the point-cloud variance matrix $V_X$ is a multiple of the identity matrix. Each section also provides an analysis of how uncertainties associated with partial features behave. A numerical example relating to establishing a datum location from the measurement of a reference sphere is given in Section 9. Our concluding remarks are given in Section 10.

2. Calculations Associated with Axes

Many of the calculations involve the distance to an axis or a plane and we consider those first.

2.1. Point on an Axis

If $\mathit{x}={\mathit{x}}_A+t_A{\mathit{v}}_A$, is a point on the axis specified by locating point ${\mathit{x}}_A={(x_A,y_A,z_A)}^{\top }$ and direction vector ${\mathit{v}}_A={(u_A,v_A,w_A)}^{\top }$, then the sensitivity matrix $G_{Z|B}$ of x with respect to ${\mathit{b}}_A^{\top }=({\mathit{x}}_A^{\top },{\mathit{v}}_A^{\top })={(x_A,y_A,z_A,u_A,v_A,w_A)}^{\top }$ is the $3\times 6$ matrix

$$G_{X|B_A}=\left[\begin{array}{cccccc}1& 0& 0& t_A& 0& 0\\ 0& 1& 0& 0& t_A& 0\\ 0& 0& 1& 0& 0& t_A\end{array}\right].$$

2.2. Distance from a Point to an Axis

For an axis specified by ${\mathit{x}}_A$ and ${\mathit{v}}_A={(u_A,v_A,w_A)}^{\top }$ with $\parallel {\mathit{v}}_A\parallel =1$, the distance from a point x to the axis is given by

$$d_A(\mathit{x},{\mathit{b}}_A)=\parallel (\mathit{x}-{\mathit{x}}_A)\times {\mathit{v}}_A\parallel ,$$

(2)

with

$$(\mathit{x}-{\mathit{x}}_A)\times {\mathit{v}}_A=\left[\begin{array}{c}\xi \\ \eta \\ \zeta \end{array}\right]=\left[\begin{array}{c}(y-y_A)w_A-(z-z_A)v_A\\ (z-z_A)u_A-(x-x_A)w_A\\ (x-x_A)v_A-(y-y_A)u_A\end{array}\right].$$

The $1\times 6$ sensitivity matrix $G_{D_A|B_A}$ of $d_A=d_A(\mathit{x},\mathit{b})$ with respect to ${\mathit{b}}_A^{\top }=({\mathit{x}}_A^{\top },{\mathit{v}}_A^{\top })$ is given by

$$G_{D|B}^{\top }=\frac{1}{d_A}\left[\begin{array}{c}\eta w_A-\zeta v_A\\ \zeta u_A-\xi w_A\\ \xi v_A-\eta u_A\\ \eta (z-z_A)-\zeta (y-y_A)\\ \zeta (x-x_A)-\xi (z-z_A)\\ \xi (y-y_A)-\eta (x-x_A)\end{array}\right].$$

(3)

2.3. Distance from a Point to a Plane Orthogonal to an Axis

The distance $d_P(\mathit{x},{\mathit{b}}_A)$ from a point x to the plane ${(\mathit{x}-{\mathit{x}}_A)}^{\top }{\mathit{v}}_A=0$ specified by ${\mathit{x}}_A$ and ${\mathit{v}}_A={(u_A,v_A,w_A)}^{\top }$ with $\parallel {\mathit{v}}_A\parallel =1$, is given by

$$d_P(\mathit{x},{\mathit{b}}_A)={(\mathit{x}-{\mathit{x}}_A)}^{\top }{\mathit{v}}_A.$$

(4)

The $1\times 6$ sensitivity matrix $G_{D_P|B_A}$ of $d_P=d_P(\mathit{x},\mathit{b})$ with respect to ${\mathit{b}}^{\top }=({\mathit{x}}_A^{\top },{\mathit{v}}_A^{\top })$ is given by

$$G_{D_P|B_A}^{\top }=\left[\begin{array}{c}-u_A\\ -v_A\\ -w_A\\ x-x_A\\ y-y_A\\ z-z_A\end{array}\right].$$

(5)

3. Least-Squares (LS) Feature Assessment

See also, e.g., [24,25,26,27]. Suppose $\mathit{u}\mapsto \mathit{s}(\mathit{u},\mathit{a})$ defines a parametric curve or surface. The parameters u determine the position of a point on the surface and the parameters a determine the shape and position of the surface. We assume that set of measured coordinates, ${\mathit{x}}_{1:m}$ nominally represent points on such a surface, so that

$${\mathit{x}}_i\approx \mathit{s}({\mathit{u}}_i,\mathit{a}),$$

(6)

for some ${\mathit{u}}_i$ and some a. The least-squares (LS) estimates ${\widehat{\mathit{u}}}_{1:m}$ and $\widehat{\mathit{a}}$ of ${\mathit{u}}_{1:m}$ and a, respectively, can be found by minimizing

$$\sum _{i=1}^m{d}^2({\mathit{x}}_i,\mathit{a}),d({\mathit{x}}_i,\mathit{a})={({\mathit{x}}_i-\mathit{s}({\mathit{u}}_i^{*},\mathit{a}))}^{\top }{\mathit{n}}_i,$$

(7)

where ${\mathit{u}}_i^{*}$ specifies the point ${\mathit{s}}_i^{*}=\mathit{s}({\mathit{u}}_i^{*},\mathit{a})$ on the surface closest to ${\mathit{x}}_i$ and ${\mathit{n}}_i$ is the normal vector at ${\mathit{s}}_i^{*}$. The term least-squares orthogonal distance regression (LSODR) is also used for this type of optimization problem [28,29] as $d({\mathit{x}}_i,\mathit{a})$ is the (signed) distance of ${\mathit{x}}_i$ from the surface $\mathit{s}(\mathit{u},\mathit{a})$ measured orthogonally to the surface. For standard geometric elements, $d(\mathit{x},\mathit{a})$ can be evaluated analytically [24]. The geometric elements considered in this paper are: circle in the $xy$-plane, plane, sphere, cylinder and cone [15,24]. For more general surfaces, numerical methods are required [27]. Let $J=J({\mathit{x}}_{1:m},\mathit{a})$ be the Jacobian matrix defined by

$$J_{ij}=\frac{\partial d}{\partial a_j}({\mathit{x}}_i,\mathit{a}).$$

The optimality conditions for a to minimize the sum of squares in (7) are of the form

$$J^{\top }\mathit{d}=\mathbf{0},J=J(x_{1:m},\mathit{a}),d_i=d({\mathit{x}}_i,\mathit{a}).$$

These optimality conditions implicitly define the solution a as function of the data points ${\mathit{x}}_{1:m}$ and allow us to evaluate the sensitivity matrix $G_{A|X}$ of a with respect to the data Section 4.2.4 [30]. If J is the Jacobian matrix and ${\mathit{n}}_i$ are the corresponding surface normals at the solution ${\widehat{\mathit{u}}}_{1:m}$ and $\widehat{\mathit{a}}$ then

$$G_{A|X}=G_{A|D}{N}^{\top },G_{A|D}=-{\left(J^{\top }J\right)}^{-1}{J}^{\top },$$

(8)

where N is the $3m\times m$ block diagonal matrix storing the normal vectors ${\mathit{n}}_i$ in the $3\times 1$ diagonal blocks. The matrix $n\times m$$G_{A|D}$ is the sensitivity matrix of the parameters a with respect to changes in $x_i$ in the direction orthogonal to the fitted surface.

If QR factorization [31] of J is given by

$$J=QR=\left[Q_1{Q}_2\right]\left[\begin{array}{c}{R}_1\\ \mathbf{0}\end{array}\right],$$

(9)

where Q is orthogonal and $R_1$ is upper-triangular, then

$$G_{A|D}=-R_1^{-1}{Q}_1^{\top }.$$

If $V_X$ is the variance matrix associated with ${\mathit{x}}_{1:m}$, then the variance matrix associated with the features a is given by

$$V_A=G_{A|X}{V}_X{G}_{A|X}^{\top }.$$

If $V_X$ is the diagonal matrix ${\sigma }_R^{2}I$, then

$$V_A={\sigma }_R^2{\left(J^{\top }J\right)}^{-1}={\sigma }_R^2{\left(R_1^{\top }{R}_1\right)}^{-1},$$

(10)

using the fact that $N^{\top }N=I$.

Weighted Least-Squares Orthogonal Distance Regression

It is sometimes useful to incorporate weights $w_i\ge 0$ into the orthogonal distance regression scheme so that the counterpart of (7) is

$$\sum _{i=1}^m{w}_i^2{d}^2({\mathit{x}}_i,\mathit{a}),d({\mathit{x}}_i,\mathit{a})={({\mathit{x}}_i-\mathit{s}({\mathit{u}}_i^{*},\mathit{a}))}^{\top }{\mathit{n}}_i.$$

(11)

Let W be the diagonal matrix with $w_i^2$ in the ith diagonal element. Then the counterpart of (8) is

$$G_{A|X,W}=G_{A|D,W}{N}^{\top },G_{A|D,W}=-{\left(J^{\top }WJ\right)}^{-1}{J}^{\top }W.$$

(12)

The least-squares estimates derived from solving the weighted LSODR problem correspond to maximum likelihood estimates [32] of the parameters for the statistical model

$${\mathit{x}}_i\in \mathcal{N}({\mathit{s}}_i,{\sigma }_i^{2}I),w_i=1/{\sigma }_i,$$

in which the measured coordinates ${\mathit{x}}_i$ are perturbed from the true point ${\mathit{s}}_i$ on the surface by independent random effects drawn from a multivariate Gaussian (normal) distribution with variance matrix ${\sigma }_i^{2}I$ where, here, I the $3\times 3$ identity matrix. From a statistical point of view, the weights $w_i$ should be assigned to be $1/{\sigma }_i$ if ${\sigma }_i$ is known or estimated. If ${\sigma }_i=\sigma$ is the same for all points, then the weights $w_i$ can be set to 1. In practice, the weights can be used to reflect knowledge about the uncertainties associated with the measurement system. For example, measurements with a longer probe offset can be assigned a smaller weight than those with a shorter probe offset. Another practical use of the weights is to de-weight or effectively remove points (setting $w_i=0$) from the analysis that are suspected of being outliers. Setting $w_i=0$ can also be used to assess the contribution of the ith measurement point to the estimate of the fitted parameters.

4. Sensitivity Matrix Associated with a Least-Squares Circle Fit to Data in a Plane

If a circle is parametrized by $\mathit{a}={(x_0,y_0,r_0)}^{\top }$ specifying its center coordinates ${\mathit{x}}_0={(x_0,y_0)}^{\top }$ and radius $r_0$, the signed distance $d({\mathit{x}}_i,\mathit{a})$ from a data point $x_i$ to the circle specified by a is

$$d({\mathit{x}}_i,\mathit{a})=r_i-r_0,$$

where $r_i^2={(x_i-x_0)}^2+{(y_i-y_0)}^2$. The ith row Jacobian matrix J of partial derivatives of $d({\mathit{x}}_i,\mathit{a})$ with respect to ${\mathit{a}}^{\top }$ is given by

$$J(i,:)=-\frac{1}{r_i}[x_i-x_0,y_i-y_0,r_i]=-[{\mathit{n}}_i^{\top },1],{\mathit{n}}_i=({\mathit{x}}_i-{\mathit{x}}_0)/r_i$$

The $3\times 3$ matrix $H=J^{\top }J$ is given by

$$H=\left[\begin{array}{cc}\sum _i{\mathit{n}}_i{\mathit{n}}_i^{\top }& \sum _i{\mathit{n}}_i\\ \sum _i{\mathit{n}}_i^{\top }& m\end{array}\right].$$

The sensitivity matrix $G_{A|X}=H^{-1}{J}^{\top }{N}^{\top }$ where

$$J^{\top }{N}^{\top }=-\left[\begin{array}{cccc}{\mathit{n}}_1{\mathit{n}}_1^{\top }& {\mathit{n}}_2{\mathit{n}}_2^{\top }& \cdots & {\mathit{n}}_m{\mathit{n}}_m^{\top }\\ {\mathit{n}}_1^{\top }& {\mathit{n}}_2^{\top }& \cdots & {\mathit{n}}_m^{\top }\end{array}\right].$$

(13)

Thus, perturbing ${\mathit{x}}_i$ by $\Delta {\mathit{x}}_i$ causes ${\mathit{x}}_0$ and $r_0$ to be perturbed by an amount that depends on extend to which $\Delta {\mathit{x}}_i$ is aligned with the normal ${\mathit{n}}_i$.

For points ${\mathit{x}}_i$ approximately uniformly distributed around the circle, H is approximately diagonal with

$$H\approx m\left[\begin{array}{ccc}1/2& 0& 0\\ 0& 1/2& 0\\ 0& 0& 1\end{array}\right],H^{-1}\approx \frac{1}{m}\left[\begin{array}{ccc}2& 0& 0\\ 0& 2& 0\\ 0& 0& 1\end{array}\right].$$

(14)

Thus, if $V_X={\sigma }_R^{2}I$ then, from (10), $V_A={\sigma }_R^2{H}^{-1}$, showing that the variances associated with a vary with $1/m$ and that the variance associated with ${\mathit{x}}_0$ is twice that associated with $r_0$, for a uniform distribution of points.

Analytical Approximations for (An Arc Of) a Circle

The calculation of H in (14) is for the case of points uniformly distributed around a complete circle. Similar but more complicated calculations can be made for points on a partial circle, including the often-problematic case of a small arc of a circle. For these calculations, it is often convenient to work in polar coordinates. For points ${(x_i,y_i)}^{\top }=r_0{(cos{\theta }_i,sin{\theta }_i)}^{\top }$ on a circle, the corresponding contribution to the Jacobian matrix is the row $(-cos{\theta }_i,-sin{\theta }_i,-1)$ and the matrix $H=J^{\top }J$ is given by

$$H=\left[\begin{array}{ccc}\sum _i{cos}^2{\theta }_i& \sum _{i}cos{\theta }_{i}sin{\theta }_i& \sum _{i}cos{\theta }_i\\ \sum _{i}cos{\theta }_{i}sin{\theta }_i& \sum _i{sin}^2{\theta }_i& \sum _{i}sin{\theta }_i\\ \sum _{i}cos{\theta }_i& \sum _{i}sin{\theta }_i& m\end{array}\right].$$

The principle of Monte Carlo integration [22] Section 7.7 states that for a function $f\left(\theta \right)$ defined over a region A the integral of the function over the region can be approximated according to

$$\frac{1}{\left|A\right|}{\int }_{A}f\left(\theta \right)\mathrm{d}\theta \approx \frac{1}{m}\sum _{i=1}^{m}f\left({\theta }_i\right),$$

(15)

where ${\theta }_{1:m}$ is a sample of points uniformly distributed over the region A and $\left|A\right|$ is the area/volume of the region. We can use this approximation in the other direction to approximate H derived from a discrete set of points from analytically derived integrals. For example, suppose points ${\mathit{x}}_{1:m}$ are approximately uniformly distributed of the arc of the circle defined by $-\alpha \le {\theta }_i\le \alpha$. Then

$$\sum _{i=1}^m{cos}^2{\theta }_i\approx \frac{m}{2\alpha }{\int }_{-\alpha }^{\alpha }{cos}^2\theta \mathrm{d}\theta =m\left(\frac{1}{2}+\frac{sin2\alpha }{4\alpha }\right);$$

see Appendix A. Continuing in this way, let

$$H_{\alpha }=\left[\begin{array}{ccc}1/2+(sin2\alpha )/4\alpha & 0& (sin\alpha )/\alpha \\ 0& 1/2-(sin2\alpha )/4\alpha & 0\\ (sin\alpha )/\alpha & 0& 1\end{array}\right],$$

(16)

so that $V_{\alpha }=H_{\alpha }^{-1}$ is given by

$$V_{\alpha }=\frac{1}{D_{13}}\left[\begin{array}{ccc}1& 0& -(sin\alpha )/\alpha \\ 0& D_{13}/(1/2-(sin2\alpha )/4\alpha )& 0\\ -(sin\alpha )/\alpha & 0& (1/2+(sin2\alpha )/4\alpha )\end{array}\right],$$

where

$$D_{13}=1/2+(sin2\alpha )/4\alpha -{((sin\alpha )/\alpha )}^2,$$

is the determinant of the $2\times 2$ submatrix of $H_{\alpha }$ constructed from its first and third rows and columns. Then

$${\left(J^{\top }J\right)}^{-1}\approx \frac{1}{m}{V}_{\alpha }.$$

If the variance matrix associated ${\mathit{x}}_{1_m}$ can be approximated by ${\sigma }_R^{2}I$, then the variance matrix $V_A$ associated with the fitted circle parameters is approximated by

$$V_A\approx \frac{{\sigma }_R^2}{m}{V}_{\alpha },$$

and the standard uncertainties associated with a are given by ${\sigma }_R\sqrt{v_{jj}/m}$, where $v_{jj}$ is the jth diagonal element of $V_{\alpha }$. The quantities $s\left(a_j\right)=\sqrt{v_{jj}}$ for selected values of $\alpha$ are given in

Table 1. Arc of a circle. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{\alpha }$. For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed on the arc of the circle defined by $-\alpha \le {\theta }_i\le \alpha$ and for point-cloud variance matrix ${\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

2$\mathit{\alpha }$/deg	$\mathit{s}\left({\mathit{x}}_0\right)$	$\mathit{s}\left({\mathit{y}}_0\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$
360	1.41	1.41	1.00
270	1.81	1.28	1.14
180	3.25	1.41	2.30
160	3.96	1.51	2.97
140	5.00	1.65	3.98
120	6.62	1.85	5.56
100	9.30	2.14	8.23
80	14.25	2.61	13.16
60	24.95	3.40	23.85
40	55.54	5.02	54.42
20	220.70	9.95	219.58
$\alpha \le 5$ deg
$\alpha$/rad	≈$\sqrt{45}/{\alpha }^2$	≈$\sqrt{3}/\alpha$	≈$\sqrt{45}/{\alpha }^2$

. For $\alpha$ less than $10\pi /180$, i.e., less than 10 degrees, then $D_{13}$ is approximated by ${\alpha }^4/45$ and the diagonal elements of $V_{\alpha }$ are approximated by $45/{\alpha }^4$, $3/{\alpha }^2$ and $45(1-{\alpha }^2/3)/{\alpha }^4$; see also Table 1. The uncertainty associated with the y-coordinate of the circle center scales with $1/\alpha$ while the uncertainties associated with the x-coordinate and radius scale with $1/{\alpha }^2$, for small $\alpha$. The estimate of the x-coordinate of circle center is almost perfectly negatively correlated with the estimate of the radius.

5. Sensitivity Matrix Associated with a Least-Squares Sphere Fit to Data

If a sphere is parametrized by $\mathit{a}={(x_0,y_0,z_0,r_0)}^{\top }$ specifying its center coordinates ${\mathit{x}}_0={(x_0,y_0,z_0)}^{\top }$ and radius $r_0$, the signed distance $d({\mathit{x}}_i,\mathit{a})$ from a data point ${\mathit{x}}_i$ to the sphere given by a is $d({\mathit{x}}_i,\mathit{a})=r_i-r_0$, where $r_i^2={(x_i-x_0)}^2+{(y_i-y_0)}^2+{(z_i-z_0)}^2$.

The ith row Jacobian matrix J of partial derivatives of $d({\mathit{x}}_i,\mathit{a})$ with respect to ${\mathit{a}}^{\top }$ is given by

$$J(i,:)=-\frac{1}{r_i}[x_i-x_0,y_i-y_0,z_i-z_0,r_i]=-[{\mathit{n}}_i^{\top },1],{\mathit{n}}_i=({\mathit{x}}_i-{\mathit{x}}_0)/r_i$$

The $4\times 4$ matrix $H=J^{\top }J$ is given by

$$H=\left[\begin{array}{cc}\sum _i{\mathit{n}}_i{\mathit{n}}_i^{\top }& \sum _i{\mathit{n}}_i\\ \sum _i{\mathit{n}}_i^{\top }& m\end{array}\right].$$

For points ${\mathit{x}}_i$ approximately uniformly distributed around the sphere, H is approximately diagonal with

$$H\approx m\left[\begin{array}{cccc}1/3& 0& 0& 0\\ 0& 1/3& 0& 0\\ 0& 0& 1/3& 0\\ 0& 0& 0& 1\end{array}\right],H^{-1}\approx \frac{1}{m}\left[\begin{array}{cccc}3& 0& 0& 0\\ 0& 3& 0& 0\\ 0& 0& 3& 0\\ 0& 0& 0& 1\end{array}\right].$$

Thus, if $V_X={\sigma }_R^{2}I$ then, from (10), $V_A={\sigma }_R^2{H}^{-1}$, showing that the variances associated with a vary with $1/m$ and that the variances associated with $x_0$ are three times that associated with $r_0$, for a uniform distribution of points around the complete sphere.

The sensitivity matrix of a with respect to ${\mathit{x}}_{1:m}$ is given by $G_{A|X}=H^{-1}{J}^{\top }{N}^{\top }$ where

$$J^{\top }{N}^{\top }=-\left[\begin{array}{cccc}{\mathit{n}}_1{\mathit{n}}_1^{\top }& {\mathit{n}}_2{\mathit{n}}_2^{\top }& \cdots & {\mathit{n}}_m{\mathit{n}}_m^{\top }\\ {\mathit{n}}_1^{\top }& {\mathit{n}}_2^{\top }& \cdots & {\mathit{n}}_m^{\top }\end{array}\right].$$

(17)

Perturbing ${\mathit{x}}_i$ by $\Delta {\mathit{x}}_i$ causes ${\mathit{x}}_0$ and $r_0$ to be perturbed by an amount that depends on extend to which $\Delta {\mathit{x}}_i$ is aligned with the normal ${\mathit{n}}_i$.

5.1. Analytical Approximations for a Patch of a Sphere

We can use the principle of Monte Carlo integration (15) to determine analytical approximations for matrices used to construct the sensitivity matrices. It is convenient to work in spherical coordinates $(x,y,z)=(cos\theta cos\varphi ,sin\theta cos\varphi ,sin\varphi )$, where $\theta$ is the azimuth angle about the z-axis and $\varphi$ is the angle of elevation above the $xy$-plane. For points approximately uniformly distributed on the sphere on the patch determined by $-\pi \le {\alpha }_1\le \theta \le {\alpha }_2\le \pi$ and $-\pi /2\le {\beta }_1\le \varphi \le {\beta }_2\le \pi /2$ we have

$$\frac{1}{m}{J}^{\top }J\approx \frac{1}{A}{H}_A,$$

where $H_A$ is the symmetric matrix with

$$\begin{array}{cc}\hfill H_A(1,1)& ={\int }_{{\alpha }_1}^{{\alpha }_2}{cos}^2\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}{cos}^3\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(1,2)& ={\int }_{{\alpha }_1}^{{\alpha }_2}sin\theta cos\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}{cos}^3\varphi \mathrm{d}\varphi \hfill \\ \hfill H_A(1,3)& ={\int }_{{\alpha }_1}^{{\alpha }_2}cos\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}sin\varphi {cos}^2\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(1,4)& ={\int }_{{\alpha }_1}^{{\alpha }_2}cos\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}{cos}^2\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(2,2)& ={\int }_{{\alpha }_1}^{{\alpha }_2}{sin}^2\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}{cos}^3\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(2,3)& ={\int }_{{\alpha }_1}^{{\alpha }_2}sin\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}sin\varphi {cos}^2\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(2,4)& ={\int }_{{\alpha }_1}^{{\alpha }_2}sin\theta \mathrm{d}\theta {\int }_{{\beta }_1}^{{\beta }_2}{cos}^2\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(3,3)& =({\alpha }_2-{\alpha }_1){\int }_{{\beta }_1}^{{\beta }_2}{sin}^2\varphi cos\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(3,4)& =({\alpha }_2-{\alpha }_1){\int }_{{\beta }_1}^{{\beta }_2}sin\varphi cos\varphi \mathrm{d}\varphi ,\hfill \\ \hfill H_A(4,4)& =({\alpha }_2-{\alpha }_1){\int }_{{\beta }_1}^{{\beta }_2}cos\varphi \mathrm{d}\varphi .\hfill \end{array}$$

These integrals can be evaluated according to the formulæ in Appendix A. The elements of $H_A$ take into account the change of variables from Cartesian to spherical coordinates and involve an additional $cos\varphi$ term.

5.2. Cap of a Sphere

We consider here the case $-\pi \le \theta \le \pi$ and $-\pi /2\le {\beta }_1\le \varphi \le \pi /2$. For this case, the nonzero elements of $H_{\gamma }=\frac{1}{\left|A\right|}{H}_A$ are determined by

$$\begin{array}{cc}\hfill H_{\gamma }(1,1)& =\frac{1}{2(1-cos\gamma )}(2/3-cos\gamma -{cos}^3\gamma /3),\hfill \\ \hfill H_{\gamma }(2,2)& =\frac{1}{2(1-cos\gamma )}(2/3-cos\gamma -{cos}^3\gamma /3),\hfill \\ \hfill H_{\gamma }(3,3)& =\frac{1}{1-cos\gamma }(1-{cos}^3\gamma )/3,\hfill \\ \hfill H_{\gamma }(3,4)& =\frac{1}{1-cos\gamma }(1-cos2\gamma )/4,\hfill \\ \hfill H_{\gamma }(4,4)& =1,\hfill \end{array}$$

where $\gamma =\pi /2-{\beta }_1$. Thus, $\gamma$ is the angle between the lower edge of the spherical cap and the North pole. The nonzero elements of $V_{\gamma }=H_{\gamma }^{-1}$ are determined by

$$\begin{array}{cc}\hfill V_{\gamma }(1,1)& =1/H_{\gamma }(1,1),\hfill \\ \hfill V_{\gamma }(2,2)& =1/H_{\gamma }(2,2),\hfill \\ \hfill V_{\gamma }(3,3)& =\frac{1}{D_{34}}\hfill \\ \hfill V_{\gamma }(3,4)& =-\frac{1}{D_{34}}{H}_{\gamma }(3,4),\hfill \\ \hfill V_{\gamma }(4,4)& =\frac{1}{D_{34}}{H}_{\gamma }(3,3),\hfill \end{array}$$

where

$$D_{34}=H_{\gamma }(3,3)-H_{\gamma }^2(3,4),$$

the determinant of the bottom right $2\times 2$ submatrix of $H_{\gamma }$. If the point-cloud data are associated with variance matrix ${\sigma }_R^{2}I$, then the variance matrix $V_A$ associated with the fitted sphere parameters a is approximated by

$$V_A\approx \frac{{\sigma }_R^2}{m}{V}_{\gamma }.$$

For $\gamma$ approaching zero, corresponding to measurements on a cap of a sphere, $D_{34}\approx {\gamma }^4/48$, and

$$V_{\gamma }\approx \left[\begin{array}{cccc}4/{\gamma }^2& 0& 0& 0\\ 0& 4/{\gamma }^2& 0& 0\\ 0& 0& 48/{\gamma }^4& -48(1-{\gamma }^2/4)/{\gamma }^4\\ 0& 0& -48(1-{\gamma }^2/4)/{\gamma }^4& 48(1-{\gamma }^2/2)/{\gamma }^4\end{array}\right].$$

(18)

The quantities $s\left(a_j\right)=\sqrt{V_{\gamma }(j,j)}$ for selected values of $\gamma$ are given in

Table 2. Cap of a sphere. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{\gamma }$ in (18). For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed on the sphere with elevation angle satisfying $\pi /2-\gamma \le {\varphi }_i\le \pi /2$ and for point-cloud variance matrix ${\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

$\mathit{\gamma }$/deg	$\mathit{s}\left({\mathit{x}}_0\right)$	$\mathit{s}\left({\mathit{y}}_0\right)$	$\mathit{s}\left({\mathit{z}}_0\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$
180	1.73	1.73	1.73	1.00
135	1.65	1.65	2.03	1.04
90	1.73	1.73	3.46	2.00
80	1.83	1.83	4.19	2.66
70	1.97	1.97	5.26	3.67
60	2.19	2.19	6.93	5.29
50	2.52	2.52	9.70	8.03
40	3.04	3.04	14.81	13.11
30	3.95	3.95	25.86	24.15
20	5.82	5.82	57.44	55.72
10	11.50	11.50	228.02	226.29
$\gamma \le 5$ deg
$\gamma$/rad	≈$2/\gamma$	≈$2/\gamma$	≈$\sqrt{48}/{\gamma }^2$	≈$\sqrt{48}/{\gamma }^2$

. The uncertainty associated with the x and y-coordinates of the sphere center scale with $1/\gamma$ while the uncertainties associated with the z-coordinate and radius scale with $1/{\gamma }^2$, for small $\gamma$. The estimate of the z-coordinate of sphere center is almost perfectly negatively correlated with the estimate of the radius. These results are in line with results associated with an arc of a circle discussed in Section 4. These calculations are also relevant to determining the radius of curvature for other surfaces such as paraboloids and aspherics that have low curvature.

5.3. Equatorial Band of a Sphere

The calculations in Section 5.1 can be used to estimate sensitivities associated with measurements distributed along an equatorial band of a sphere defined by $-\pi \le \theta \le \pi$ and $-\beta \le \varphi \le \beta \le \pi /2$. Here $\beta$ is the angle between top and bottom of the band and the equatorial plane. These calculations are also relevant to measurements using ball bar or machine checking gauge that rotates about a fixed point and defines points on a virtual sphere. The area over which the integration is performed is $\left|A\right|=4\pi sin\beta$. For points approximately uniformly distributed in an equatorial band, if the point-cloud data are associated with variance matrix ${\sigma }_R^{2}I$, then the variance matrix $V_A$ associated with the fitted sphere parameters a is approximated by

$$V_A\approx \frac{{\sigma }_R^2}{m}{V}_{\beta }$$

where

$$V_{\beta }=\left[\begin{array}{cccc}2/(1-{sin}^2\beta /3)& 0& 0& 0\\ 0& 2/(1-{sin}^2\beta /3)& 0& 0\\ 0& 0& 3/{sin}^2\beta & 0\\ 0& 0& 0& 1\end{array}\right].$$

(19)

Table 3. Equatorial band of a sphere. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{\beta }$ in (19). For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed on an equatorial band of the sphere with elevation angle satisfying $-\beta \le {\varphi }_i\le \beta \le \pi /2$ and for point-cloud variance matrix ${\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

$\mathit{\beta }$/deg	$\mathit{s}\left({\mathit{x}}_0\right)$	$\mathit{s}\left({\mathit{y}}_0\right)$	$\mathit{s}\left({\mathit{z}}_0\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$
90	1.73	1.73	1.73	1.00
80	1.72	1.72	1.76	1.00
70	1.68	1.68	1.84	1.00
60	1.63	1.63	2.00	1.00
50	1.58	1.58	2.26	1.00
40	1.52	1.52	2.69	1.00
30	1.48	1.48	3.46	1.00
20	1.44	1.44	5.06	1.00
10	1.42	1.42	9.97	1.00
$\beta \le 5$ deg
$\beta$/rad	≈$\sqrt{2}$	≈$\sqrt{2}$	≈$\sqrt{3}/\beta$	1

gives the square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{\beta }$ in (19) for selected values of the parameter $\beta$. For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed on an equatorial band of the sphere with elevation angle satisfying $-\beta \le {\varphi }_i\le \beta \le \pi /2$ and for point-cloud variance matrix ${\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$. The parameters $x_0$, $y_0$ and $r_0$ remain well-defined as $\beta$ approaches zero while the uncertainty associated with $z_0$ scales with $1/\beta$.

5.4. Points on a Longitudinal Segment of a Sphere

The calculations in Section 5.1 can also be used to estimate sensitivities associated with measurements distributed in a longitudinal segment of a sphere defined by $-\pi \le -\alpha \le \theta \le \alpha \le \pi$ and $-\pi /2\le \varphi \le \pi /2$ ( the curved surface of a segment of an orange). The area over which the integration is performed is $\left|A\right|=4\alpha$. For points approximately uniformly distributed over the segment, if the point-cloud data are associated with variance matrix ${\sigma }_R^{2}I$, then the variance matrix $V_A$ associated with the fitted sphere parameters a is approximated by

$$V_A\approx \frac{{\sigma }_R^2}{m}{V}_{\alpha }$$

(20)

where the nonzero elements of $V_{\alpha }$ are specified by

$$\begin{array}{cc}\hfill V_{\alpha }(1,1)& =1/D_{14},\hfill \\ \hfill V_{\alpha }(1,4)& =-\pi sin\alpha /\left(4\alpha D_{14}\right),\hfill \\ \hfill V_{\alpha }(2,2)& =3\alpha /(\alpha -(sin2\alpha )/2),\hfill \\ \hfill V_{\alpha }(3,3)& =3,\hfill \\ \hfill V_{\alpha }(4,4)& =3\alpha /\left(D_{14}(\alpha +(sin2\alpha )/2)\right),\hfill \end{array}$$

with

$$D_{14}=\frac{\alpha +(sin2\alpha )/2}{3\alpha }-{\left(\frac{\pi sin\alpha }{4\alpha }\right)}^2.$$

the determinant of the $2\times 2$ submatrix of $H_{\alpha }=V_{\alpha }^{-1}$ comprising of rows and columns 1 and 4. For $\alpha$ near zero, $D_{14}\approx 2/3-{(\pi /4)}^2\approx 0.05$,

$$V_{\alpha }\approx \left[\begin{array}{cccc}20& 0& 0& -5\pi \\ 0& 9/\left(2{\alpha }^2\right)& 0& 0\\ 0& 0& 3& 0\\ -5\pi & 0& 0& 40/3\end{array}\right].$$

(21)

The quantities $s\left(a_j\right)=\sqrt{V_{\alpha }(j,j)}$ for selected values of $\alpha$ are given in

Table 4. Longitudinal segment of a sphere. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{\alpha }$ in (20). For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed on a segment of the sphere with $-\alpha \le {\theta }_i\le \alpha \le \pi$ and for point-cloud variance matrix ${\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

$2\mathit{\alpha }$/deg	$\mathit{s}\left({\mathit{x}}_0\right)$	$\mathit{s}\left({\mathit{y}}_0\right)$	$\mathit{s}\left({\mathit{z}}_0\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$
360	1.73	1.73	1.73	1.00
270	2.20	1.57	1.73	1.13
180	3.46	1.73	1.73	2.00
160	3.85	1.85	1.73	2.36
140	4.22	2.02	1.73	2.74
120	4.50	2.26	1.73	3.09
100	4.66	2.62	1.73	3.36
80	4.69	3.19	1.73	3.53
60	4.64	4.16	1.73	3.62
40	4.56	6.15	1.73	3.65
20	4.50	12.19	1.73	3.66
$\alpha \le 5$ deg
$\alpha$/rad	≈$\sqrt{20}$	≈$\sqrt{4.5}/\alpha$	≈$\sqrt{3}$	≈$\sqrt{40/3}$

. The uncertainty associated with the y-coordinate of the sphere center scales with $1/\alpha$, while all other parameters remain well defined. These calculations depend on measuring over the complete segment, including the two poles. Corresponding calculations for measurements reduced to an equatorial band of the segment can be made using the results in Section 5.1.

6. Sensitivity Matrix Associated with a Least-Squares Plane Fit to Data

The calculations associated with an axis given above in Section 2 allow us to evaluate the sensitivity matrix associated with a least-squares plane fit to data. Given a location point ${\mathit{x}}_A$ and unit direction vector ${\mathit{v}}_A$, the equation of the associated plane can be written as

$${(\mathit{x}-{\mathit{x}}_A)}^{\top }{\mathit{v}}_A=0.$$

The calculations involve a parametrization of the plane in terms of three parameters a in which the kth coordinate of ${\mathit{v}}_A$ is held fixed and only the kth coordinate of ${\mathit{x}}_A$ is free.

Analytical Approximation for Measuring a Rectangular Area

Suppose data points ${\mathit{x}}_{1:m}$ are distributed approximately uniformly on the plane $z=0$ with $-a\le x_i\le a$ and $-b\le y_i\le b$ and J is the $m\times 3$ Jacobian matrix associated with fitting a plane to the data with $J(i,1:3)=(-1,x_i,y_i)$. This Jacobian matrix corresponds to parametrization in terms of the z-coordinate of ${\mathit{x}}_A$ and the x- and y-coordinates of ${\mathit{v}}_A$, $\mathit{a}={(z_A,u_A,v_A)}^{\top }$, holding $x_A$, $y_A$ and $w_A$ constant. Then, using the principle of Monte Carlo integration (15),

$$\frac{1}{m}{J}^{\top }J\approx H_{ab},$$

where

$$H_{ab}=\left[\begin{array}{ccc}1& 0& 0\\ 0& a^2/3& 0\\ 0& 0& b^2/3\end{array}\right].$$

(22)

Set

$$V_{ab}=H_{ab}^{-1}=\left[\begin{array}{ccc}1& 0& 0\\ 0& 3/a^2& 0\\ 0& 0& 3/b^2\end{array}\right].$$

(23)

If the variance matrix associated with ${\mathit{x}}_{1:m}$ is approximated by ${\sigma }_R^{2}I$, then the standard uncertainties $u\left(\mathit{a}\right)$ associated with the parameters is given by

$$u\left(\mathit{a}\right)=\frac{{\sigma }_R}{\sqrt{m}}{(1,\sqrt{3}/a,\sqrt{3}/b)}^{\top }.$$

(24)

Thus, the uncertainty in the angle of rotation about the x-axis scales with $1/b$ and that associated with rotation about the y-axis scales with $1/a$.

7. Sensitivity Matrix Associated with a Least-Squares Cylinder Fit to Data

The calculations associated with an axis given above in Section 2 allow us to evaluate the sensitivity matrix associated with a least-squares cylinder fit to data.

Analytical Approximation for Measuring a Cylindrical Patch

For this section, it is convenient to work in cylindrical coordinates. Suppose data points ${\mathit{x}}_i={(r_{0}cos{\theta }_i,r_{0}sin{\theta }_i,z_i)}^{\top }$ are distributed approximately uniformly on a cylinder $x^2+y^2=r_0^2$ with $-\pi \le -\alpha \le {\theta }_i\le \alpha \le \pi$ and $-a\le z_i\le a$. In words, the points on the cylindrical patch are limited to a circumferential arc of length $2\alpha$. Let ${\mathit{x}}_A={(x_A,y_A,z_A)}^{\top }$ and ${\mathit{v}}_A={(u_A,v_A,w_A)}^{\top }$ specify the locating point and direction vector of the cylinder axis. Parametrizing the cylinder in terms of $\mathit{a}={(x_A,y_A,u_A,v_A,r_0)}^{\top }$ (holding $z_A$ and $w_A$ fixed), the associated $m\times 5$ Jacobian matrix has ith row given by

$$J(i,1:5)=-\left[cos{\theta }_{i}sin{\theta }_i-z_{i}sin{\theta }_i{z}_{i}cos{\theta }_{i}1\right].$$

Then, using the principle of Monte Carlo integration (15),

$$\frac{1}{m}{J}^{\top }J\approx H_{a\alpha },$$

where the nonzero elements of $H_{a\alpha }$ are specified by

$$\begin{array}{cc}\hfill H_{a\alpha }(1,1)& =\frac{1}{2\alpha }\left(\alpha +(sin2\alpha )/2\right),\hfill \\ \hfill H_{a\alpha }(1,5)& =(sin\alpha )/\alpha \hfill \\ \hfill H_{a\alpha }(2,2)& =\frac{1}{2\alpha }\left(\alpha -(sin2\alpha )/2\right),\hfill \\ \hfill H_{a\alpha }(3,3)& =\frac{a^2}{6\alpha }\left(\alpha -(sin2\alpha )/2\right),\hfill \\ \hfill H_{a\alpha }(4,4)& =\frac{a^2}{6\alpha }\left(\alpha +(sin2\alpha )/2\right),\hfill \\ \hfill H_{a\alpha }(5,5)& =1.\hfill \end{array}$$

For $\alpha =\pi$, corresponding to data approximately uniformly distributed on the cylindrical surface, we have

$$V_{a\pi }=H_{a\pi }^{-1}=\left[\begin{array}{ccccc}2& 0& 0& 0& 0\\ 0& 2& 0& 0& 0\\ 0& 0& 6/a^2& 0& 0\\ 0& 0& 0& 6/a^2& 0\\ 0& 0& 0& 0& 1\end{array}\right],$$

showing that the uncertainties associated with the direction vector $v_A$ scale with $1/a$ (but are independent of the radius $r_0$). For general $\alpha$, the elements in the first, second and fifth rows and columns of $H_{a\alpha }$ above are exactly the same as the elements of $H_{\alpha }$ in (16) associated with the analysis of measurements of an arc of a circle. In particular, the behavior for measurements of a section of a cylinder subtending a small angle can be derived from the analysis on an arc of a circle. Let $V_{a\alpha }=H_{a\alpha }^{-1}$. The matrix $V_{\alpha }$ given by (20) is a submatrix of $V_{a\alpha }$. For points approximately uniformly distributed in $-\pi \le -\alpha \le {\theta }_i\le \alpha \le \pi$, $-a\le z_i\le a$ and if the point-cloud data are associated with variance matrix $V_X={\sigma }_R^{2}I$, then the variance matrix $V_A$ associated with the fitted cylinder parameters a is approximated by

$$V_A\approx \frac{{\sigma }_R^2}{m}{V}_{a\alpha }.$$

(25)

Table 5. Cylindrical patch. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{a\alpha }$ in (25). For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed over a segment of a cylinder with $-\alpha \le {\theta }_i\le \alpha \le \pi$ and $-a\le z_i\le a$, and point-cloud variance matrix $V_X={\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

$2\mathit{\alpha }$/deg	$\mathit{s}\left({\mathit{x}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{y}}_{\mathit{A}}\right)$	$\mathit{as}\left({\mathit{u}}_{\mathit{A}}\right)$	$\mathit{as}\left({\mathit{v}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$
360	1.41	1.41	2.45	2.45	1.00
270	1.81	1.28	2.22	2.76	1.14
180	3.25	1.41	2.45	2.45	2.30
160	3.96	1.51	2.61	2.31	2.97
140	5.00	1.65	2.85	2.18	3.98
120	6.62	1.85	3.20	2.06	5.56
100	9.30	2.14	3.71	1.96	8.23
80	14.25	2.61	4.51	1.88	13.16
60	24.95	3.40	5.89	1.81	23.85
40	55.54	5.02	8.70	1.77	54.42
20	220.70	9.95	17.24	1.74	219.58
$\alpha \le 5$ deg
$\alpha$/rad	≈$\sqrt{45}/{\alpha }^2$	≈$\sqrt{3}/\alpha$	≈$3/\alpha$	≈$\sqrt{3}$	≈$\sqrt{45}/{\alpha }^2$

shows the square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{a\alpha }$ as a function of a (the height of the cylinder is $2a$) and $\alpha$. For $V_X={\sigma }_R^2$, $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$. For a small arc of a cylinder, $\alpha$ near zero, the uncertainties in the x-coordinate of the axis locating point and the radius scale with $1/{\alpha }^2$, the y-coordinate of the axis locating point and the angle of rotation about the x-axis scale with $1/\alpha$ while the angle of rotation of about the y-axis is well defined. The uncertainties associated with the angles of rotation scale with $1/a$.

8. Sensitivity Matrix Associated with a Least-Squares Cone Fit to Data

The calculations associated with an axis given above in Section 2 and a cylinder fit can be extended to evaluate the sensitivity matrix associated with a least-squares cone fit to data. The calculations below are based on specifying the cone in terms of an axis locating point ${\mathit{x}}_A$, an axis unit direction vector ${\mathit{v}}_A$, cone radius $r_0$, and cone angle $\varphi$. The radius parameter is the radius of the circle defined by the intersection of the cone with the plane passing through ${\mathit{x}}_A$ and orthogonal to ${\mathit{v}}_A$, i.e., the set of points x on the cone satisfying ${(\mathit{x}-{\mathit{x}}_A)}^{\top }{\mathit{v}}_A=0$. The cone angle is the angle the cone generator makes with cone axis, i.e., half the vertex angle, with the convention that if $\varphi >0$, then the vertex of the cone lies at ${\mathit{x}}_A+t{\mathit{v}}_A$ with $t>0$. Although it may be natural to use the cone vertex as the axis locating point, the parametrization in terms of a radius remains stable for cone angles near zero.

The distance d from a point x to a cone specified by ${\mathit{b}}^{\top }={({\mathit{x}}_A^{\top },{\mathit{v}}_A^{\top },r_0,\varphi )}^{\top }$ is given by

$$d=d(\mathit{x},\mathit{b})=(cos\varphi )d_C(\mathit{x},\mathit{b})+(sin\varphi )d_P(\mathit{x},\mathit{a}),$$

(26)

where $d_C(\mathit{x},\mathit{b})$ is the distance of x to the cylinder specified by ${\mathit{x}}_A$, ${\mathit{v}}_A$ and $r_0$ and $d_P(\mathit{x},\mathit{b})$ is the distance of x to the plane specified by ${\mathit{x}}_A$ and ${\mathit{v}}_A$.

Analytical Approximation for Measuring a Patch of a Cone

Suppose data points ${\mathit{x}}_i={(r_{i}cos{\theta }_i,r_{i}sin{\theta }_i,z_i)}^{\top }$, $r_i=r_0-tan\varphi z_i$, are distributed approximately uniformly on a cone with ${\mathit{x}}_A=\mathbf{0}$, $v_A={(0,0,1)}^{\top }$, with $-\alpha \le {\theta }_i\le \alpha \le \pi$ and $-a\le z_i\le a$. Parametrizing the cone in terms of $\mathit{a}={(x_A,y_A,u_A,v_A,r_0,\varphi )}^{\top }$ (holding $z_A$ and $w_A$ constant), the associated $m\times 6$ Jacobian matrix has ith row given by

$$J(i,1:6)=\left[\begin{array}{c}-cos\varphi cos{\theta }_i\\ -cos\varphi sin{\theta }_i\\ w_{i}sin{\theta }_i\\ -w_{i}cos{\theta }_i\\ -cos\varphi \\ z_i/cos\varphi \end{array}\right],w_i=z_i/cos\varphi -r_{0}sin\varphi .$$

Then, using the principle of Monte Carlo integration (15),

$$\frac{1}{m}{J}^{\top }J\approx H_{a\alpha \varphi },$$

where the nonzero elements of $H_{a\alpha \varphi }$ are given by the integrals of functions of $\theta$, z and $\varphi$ determined from the form of the Jacobian matrix above. The integrals are somewhat more complicated than the other cases already considered but can be easily evaluated using one-dimensional quadrature routines [22]. Here we give some example calculations.

Table 6. Cone frustum of varying cone angle. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{a\alpha \varphi }$ as a function of $\varphi$. For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed over cone with cone angle $\varphi$, $-\pi \le {\theta }_i\le \pi$ and $-a\le z_i\le a$, $a=100$ and point-cloud variance matrix $V_X={\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

$\mathit{\varphi }$/deg	$\mathit{s}\left({\mathit{x}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{y}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{u}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{v}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$	$\mathit{s}\left(\mathit{\varphi }\right)$
0	1.41	1.41	0.05	0.05	1.00	0.03
10	1.71	1.71	0.05	0.05	1.02	0.03
20	2.38	2.38	0.05	0.05	1.07	0.03
30	3.21	3.21	0.04	0.04	1.17	0.03
40	4.14	4.14	0.04	0.04	1.35	0.03
50	5.28	5.28	0.03	0.03	1.66	0.02
60	7.12	7.12	0.03	0.03	2.31	0.02
70	13.59	13.59	0.03	0.03	4.80	0.02

shows the square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{a\alpha \varphi }=H_{a\alpha \varphi }^1$ as a function of $\varphi$ for the case $\alpha =\pi$, $r_0=50$ and $a=100$. For $V_X={\sigma }_R^2$, $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$. As $\varphi$ approaches 90 degrees, the uncertainties associated with $x_A$, $y_A$ and $r_0$ increase markedly.

Table 7. Arc segment of a cone. Square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{a\alpha \varphi }$ as a function of $\alpha$. For points ${\mathit{x}}_{1:m}$ approximately uniformly distributed over a segment of a cone with cone angle $\varphi =45$ degrees, $-\alpha \le {\theta }_i\le \alpha \le \pi$ and $-a\le z_i\le a$, and point-cloud variance matrix $V_X={\sigma }_R^{2}I$, the uncertainties $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$.

$2\mathit{\alpha }$/deg	$\mathit{s}\left({\mathit{x}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{y}}_{\mathit{A}}\right)$	$\mathit{as}\left({\mathit{u}}_{\mathit{A}}\right)$	$\mathit{as}\left({\mathit{v}}_{\mathit{A}}\right)$	$\mathit{s}\left({\mathit{r}}_0\right)$	$\mathit{s}\left(\mathit{\varphi }\right)$
360	4.06	4.06	0.04	0.04	1.73	0.03
270	5.21	3.69	0.04	0.05	1.97	0.03
180	9.33	4.06	0.04	0.10	3.98	0.07
160	11.37	4.34	0.05	0.12	5.14	0.09
140	14.37	4.73	0.05	0.15	6.89	0.12
120	19.01	5.30	0.06	0.20	9.64	0.17
100	26.72	6.15	0.06	0.28	14.25	0.25
80	40.94	7.48	0.08	0.43	22.79	0.39
60	71.67	9.77	0.10	0.75	41.31	0.72
40	159.51	14.43	0.15	1.67	94.27	1.63
20	633.90	28.59	0.30	6.62	380.32	6.59

shows the square roots $s\left(\mathit{a}\right)$ of the diagonal elements of $V_{a\alpha \varphi }$ as a function of $\alpha$. For $V_X={\sigma }_R^2$, $u\left(\mathit{a}\right)={\sigma }_{R}s\left(\mathit{a}\right)/\sqrt{m}$. For a small arc of a cone, $\alpha$ near zero, the uncertainties associated with $x_A$, $u_A$, $v_A$, $r_0$ and $\varphi$ scale with $1/{\alpha }^2$ while those associated with the $y_A$ and $u_A$ scale with $1/\alpha$. No parameter is well defined.

9. Numerical Example: Measuring a Sphere to Determine Datum Location

In several applications a high-quality sphere or tooling ball is used to determine a location in specifying a frame of reference. Here we give example calculations for measurement strategies involving:

• Six points:four equally spaced around the equator, and one at each pole.
• Five points: as for six points but missing the point at the South pole
• Nine points: four equally spaced around the equator and another four equally spaced at elevation angle $\varphi >0$ and additional point at the North pole.

We assume that the associated point-cloud variance matrix is given by $V_X={\sigma }_R^{2}I$.

We are interested in strategies such that all coordinates of the sphere center have approximately the same uncertainty. This is achieved by the six-point strategy which is symmetrical in each axis. However, due constraints associated mounting the reference sphere, it is quite unlikely that such a six-point strategy is feasible. The other point distributions all involve points on a hemisphere and are usually possible to implement.

For the nine-point strategy, the measurement strategy depends on the elevation angle $\varphi$. If $J_{\varphi }$ is the Jacobian matrix associated with the nine points, and $H_{\varphi }=J_{\varphi }^{\top }{J}_{\varphi }$, then

$$H_{\varphi }=\left[\begin{array}{cccc}4-2{sin}^2\varphi & 0& 0& 0\\ 0& 4-2{sin}^2\varphi & 0& 0\\ 0& 0& 1+4{sin}^2\varphi & 1+4sin\varphi \\ 0& 0& 1+4sin\varphi & 9\end{array}\right].$$

The variance matrix associated with a depends on matrix $V_{\varphi }=H_{\varphi }^{-1}$ and the requirement that the uncertainties in the sphere center coordinates are equal leads to the equation

$$38{sin}^2\varphi -8sin\varphi -28=0,$$

a quadratic equation in $sin\varphi$, from which we deduce

$$sin\varphi =(8+\sqrt{4320})/76,$$

i.e., $\varphi =75.{95}^{\circ }$. For this value of $\varphi$, the four points on the upper circle are at height $0.97r_0$ and radius approximately $r_0/4$, so quite near the North pole.

Table 8. Sphere datum location. Estimates of $s\left(a\right)={\sigma }_{R}u\left(a\right)$ associated with sphere parameters for three measurement strategies. The first column of two gives the uncertainties calculated using an analytic approximation based on assuming a uniform distribution of points on the sphere or hemisphere while the second column gives the uncertainties based on the actual distribution of points.

	$\mathit{s}|6$	$\mathit{s}|{\mathit{X}}_6$	$\mathit{s}|5$	$\mathit{s}|{\mathit{X}}_5$	$\mathit{s}|9$	$\mathit{s}|{\mathit{X}}_9$
$x_0$	0.71	0.71	0.77	0.71	0.58	0.69
$y_0$	0.71	0.71	0.77	0.71	0.58	0.69
$z_0$	0.71	0.71	1.55	1.12	1.15	0.69
$r_0$	0.41	0.41	0.89	0.50	0.67	0.50

gives estimates $s\left(a\right)={\sigma }_{R}u\left(a\right)$ for the three measurement strategies discussed above. The first column in the pairs of columns gives the uncertainties calculated using an analytic approximation based on assuming a uniform distribution of points on the sphere or hemisphere while the second column gives the uncertainties based on the actual distribution of points. For the six-point strategy, the uncertainties in the sphere center coordinates are seen to be the same and that the analytical approximation to the uncertainties is exact. For the five-point strategy, the uncertainty associated with $z_0$ is larger than those associated with $x_0$ and $y_0$ by a factor of two. For the nine-point strategy, $\varphi$ is set to value that delivers equal uncertainties associated with the sphere center coordinates. We note the analytical approximations given in columns labelled $s|5$ and $s|9$ in Table 8 are related to the elements of the row of Table 2 corresponding to $\gamma ={90}^{\circ }$ (hemisphere) by dividing through by $\sqrt{5}$ and $3=\sqrt{9}$, respectively.

10. Concluding Remarks

This paper has been concerned with the evaluation of the uncertainties associated with Gaussian-associated features following the GUM methodology. We have shown how sensitivity matrices necessary for a GUM uncertainty evaluation can be calculated and how the variance matrices associated with the feature parameters can be estimated for a range of complete and partial features common in engineering. Example results are given in tables that allow practitioners to estimate, a priori, the uncertainties associated with fitted parameters, given a proposed measurement strategy for the case in which the point-cloud variance matrix is a multiple of the identity matrix. The sensitivity matrices can be used to evaluate the uncertainties for associated features for more general point-cloud variance matrices. All the calculations are straightforward and have been implemented in a spreadsheet supporting basic matrix algebra, for example.