The Storage within Digital Calibration Certificates of Uncertainty Information Obtained Using a Monte Carlo Method

Abstract

Supplement 1 to the ‘Guide to the expression of uncertainty of measurement’ describes a Monte Carlo method as a general numerical approach to uncertainty evaluation. Application of the approach typically delivers a large number of values of the output quantity of interest from which summary information such as an estimate of the quantity, its associated standard uncertainty, and a coverage interval for the quantity can be obtained and reported. This paper considers the use of a Monte Carlo method for uncertainty evaluation in calibration, using two examples to demonstrate how so-called ‘digital calibration certificates’ can allow the complete set of results of a Monte Carlo calculation to be reported.

1. Introduction

Technological advancements within the last few decades have served to digitalise many aspects of metrology. For example, instruments can be programmed to undertake time-consuming measurements with little or no need for human interaction, while the availability of greater computer processing power allows complex systems to be modelled increasingly accurately. There is one aspect of metrology, viz. the provision of calibration services, to which digital transformation has been applied in a much more modest way. Many calibration service providers continue to disseminate calibration information using paper-based certificates. Some organisations have moved to providing certificates in electronic form, for example, in archiveable Portable Document Format (PDF-A) [1]. While the provision of electronic certificates brings obvious benefits such as decreased use of paper and the potential for storage within dedicated document management systems, one undesirable property persists—a lack of machine-readability, i.e., information is not presented in a form that can be processed by computer. Currently, information on a paper-based or electronic calibration certificate can only be used if it is transcribed manually. Such a process is inevitably prone to error.

Recent initiatives have looked at how paper-based or electronic calibration certificates can be replaced by fully machine-readable certificates. The European Metrology Programme for Innovation and Research (EMPIR) [2] has funded the Joint Research Project ‘Communication and validation of smart data in IoT-networks’ (short name ‘SmartCom’) [3,4]. One objective of the SmartCom project has been to develop a framework for what are referred to as ‘digital calibration certificates’, abbreviated hereafter in this paper to ‘DCCs’. From the perspective of the SmartCom project, the critical property of DCCs is that they are fully machine-readable. It is noted that the term ‘digital calibration certificate’ has been and is used by other authors to refer to calibration certificates that take the form of electronic files but are not machine-readable.

When presenting the outcome of calibration, measurement data must be accompanied by associated uncertainty information. On a calibration certificate, it is common for a quantity value to be provided along with an associated expanded uncertainty (defined in the International Vocabulary of Metrology (VIM) [5], clause 2.35) and coverage factor (VIM [5], clause 2.38), or a coverage interval (VIM [5], clause 2.36), corresponding to a specified coverage probability (VIM [5], clause 2.37). A standard uncertainty may also be provided.

The focus of this paper is on the storage of uncertainty information obtained using the numerical approach described in the supporting document to the ‘Guide to the expression of uncertainty in measurement’ (GUM) [6] known as Supplement 1 to the GUM (GUMS1) [7]. The approach is a Monte Carlo method (MCM) for the propagation of probability distributions and is based on repeated random sampling. A key aspect underpinning the approach is the provision of a measurement model that describes mathematically how a quantity of interest (the measurand or output quantity) depends on other quantities (input quantities) to which probability distributions can be assigned. The output of an implementation of MCM (a ‘Monte Carlo calculation’) provides rich information in the form of (often hundreds of thousands of) sampled values of the measurand. The sampled values can be used to define an approximation to the probability distribution for the measurand. Summary information can be calculated using those sampled values. For example, the expectation and the standard deviation provide, respectively, an estimate of the measurand and its associated standard uncertainty, while a coverage interval for the measurand corresponding to a specified coverage probability can also be determined. The provision of additional summary information has been considered, e.g., in [8].

For calibration services where MCM is used to undertake uncertainty evaluation, it is common for only summary information to be provided on calibration certificates. The reasons for not including the sampled values of the measurand on the certificate are understandable, e.g., the number of pages could increase significantly, and the effort required to transcribe the sampled values would make it highly unlikely that they would ever be used in practice. It is possible for the sampled values to be made available in an electronic file. When doing so, consideration must be given to aspects including the provision of additional information such as units of measurement and appropriate metadata, while the electronic file must also be transmitted using a suitably secure means that ensures the file cannot be corrupted.

Should information about the measurand be required as input to a subsequent calculation, it is common, in the absence of any other information, for a Gaussian (normal) distribution, with expectation and standard deviation given, respectively, e.g., by the estimate and standard uncertainty quoted on the calibration certificate, to be assigned to the measurand. Such an assignment is often made even though the true probability distribution may be significantly different. The quality of the result of the subsequent uncertainty calculation may be significantly influenced by the assumption of normality. Were the sampled values generated by MCM available, one could instead implement MCM for the subsequent calculation by drawing randomly from those values. The SmartCom project has developed a data model that allows measurement data and associated uncertainty information to be stored in digital form. The model builds upon the International System of Units (SI) [9], the globally-agreed system of measurement units that has at its heart the seven base units of kilogram, metre, second, ampere, kelvin, mole and candela. The data model, referred to as the ‘Digital SI’ (frequently shortened simply to ‘D-SI’) [10], allows the representation of quantities that are real or complex, and univariate or multivariate.

This paper focuses on how the D-SI allows uncertainty information, including the complete set of results of a Monte Carlo calculation, to be provided within a DCC. While the GUMS1 approach to uncertainty evaluation is well-established, DCCs are a much more recent development and the potential overlap between GUMS1 and DCCs has not previously been discussed. Consideration is given to the cases where the measurand is real and univariate, i.e., a single real quantity, and real and multivariate, i.e., comprises more than one real quantity. Section 2 provides a brief summary of uncertainty evaluation undertaken using MCM for both cases. Section 3 introduces the main components of the DCC and outlines how measurement data and associated uncertainty information for real quantities can be encapsulated in the D-SI. Section 4 describes two examples, the first relating to the measurement of a univariate real quantity and based on an example in GUMS1, the second relating to the measurement of a multivariate real quantity and based on an example in Supplement 2 to the GUM (GUMS2) [11]. Concluding remarks are presented in Section 5.

Note that this paper does not discuss technical and legal aspects associated with the generation, delivery and use of DCCs. Such aspects are considered in, e.g., [12].

2. The Monte Carlo Method

Uncertainty evaluation is generally considered to comprise two stages, formulation and calculation. The stages are summarised below for the cases where the measurand is real and univariate, and real and multivariate.

2.1. Univariate Real Quantity

• The formulation stage involves the following steps:
  • Identification of the measurand Y and the input quantities $\mathbf{X}={\left(X_1,\dots ,X_N\right)}^{\top }$ on which the measurand depends.
  • Assignment of the mathematical relationship between the measurand and the input quantities, e.g., 

    $$Y=f\left(\mathbf{X}\right).$$
  • Assignment of probability distributions for the input quantities. The quantities may all be independent, in which case each quantity is assigned a probability distribution, or there may be correlation between some of the quantities, meaning that a joint probability distribution is assigned to those quantities.
• The calculation stage, when implementing MCM, involves the following steps:
  • Assign a number M of trials.
  • For $k=1,\dots ,M$, sample values $x_{1,k},\dots ,x_{N,k}$ from the probability distributions for the input quantities and evaluate

    $$y_k=f(x_{1,k},\dots ,x_{N,k}).$$
  • Calculate the estimate y of the measurand and its associated standard uncertainty $u\left(y\right)$ given, respectively, by the expectation and standard deviation of the values $y_k,k=1,\dots ,M$.
  • Use the approximation to the distribution function for the measurand to determine a coverage interval corresponding to a specified coverage probability.

Note that, both for simplicity and to reflect the choice of examples in Section 4, this section considers only the case of a measurement model that can be classified as explicit, i.e., the measurand can be expressed as an explicit mathematical function of the input quantities. Variants of MCM are available for the case where the relationship between the measurand and input quantities cannot be expressed explicitly. The storage within DCCs of uncertainty information obtained using MCM is equally applicable to explicit and implicit measurement models. In addition, again for simplicity, this section considers a particular implementation of MCM where a fixed number M of trials is assigned in the first step. Alternative applications of MCM can, and possibly should, be implemented, e.g., an adaptive approach as described in clause 7.9 of GUMS1 [7] where an increasing number of trials are carried out until results are deemed to have stabilised sufficiently according to predetermined criteria.

2.2. Multivariate Real Quantity

• The formulation stage involves the following steps:
  • Identification of the measurand $\mathbf{Y}={\left(Y_1,\dots ,Y_m\right)}^{\top }$ and the input quantities $\mathbf{X}={\left(X_1,\dots ,X_N\right)}^{\top }$ on which the measurand depends.
  • Assignment of the mathematical relationship between the measurand and the input quantities, e.g., 

    $$\mathbf{Y}=\mathbf{f}(X_1,\dots ,X_N)\equiv {\left(f_1\left(\mathbf{X}\right),\dots ,f_m\left(\mathbf{X}\right)\right)}^{\top }.$$
  • Assignment of probability distributions for the input quantities.
• The calculation stage, when implementing MCM, involves the following steps:
  • Assign a number M of trials.
  • For $k=1,\dots ,M$, sample values $x_{1,k},\dots ,x_{N,k}$ from the probability distributions for the input quantities and evaluate

    $${\mathbf{y}}_k\equiv {\left(y_{1,k},\dots ,y_{m,k}\right)}^{\top }=\mathbf{f}(x_{1,k},\dots ,x_{N,k}).$$
  • From the values ${\mathbf{y}}_k,k=1,\dots ,M$, calculate an estimate $\mathbf{y}={\left(y_1,\dots ,y_m\right)}^{\top }$ of the measurand and its associated covariance matrix

    $$V_{\mathbf{y}}=\left[\begin{array}{ccccc}{u}^2\left(y_1\right)& u(y_1,y_2)& \dots & u(y_1,y_{m-1})& u(y_1,y_m)\\ u(y_2,y_1)& u^2\left(y_2\right)& \dots & u(y_2,y_{m-1})& u(y_2,y_m)\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ u(y_{m-1},y_1)& u(y_{m-1},y_2)& \dots & u^2\left(y_{m-1}\right)& u(y_{m-1},y_m)\\ u(y_m,y_1)& u(y_m,y_2)& \dots & u(y_m,y_{m-1})& u^2\left(y_m\right)\end{array}\right],$$

    where $u\left(y_i\right)$ is the standard uncertainty associated with $y_i$ and $u(y_i,y_j)\equiv u(y_j,y_i)$ is the covariance associated with $y_i$ and $y_j$.
  • Use the approximation to the distribution function for the measurand to determine a coverage region corresponding to a specified coverage probability.

3. Digital Calibration Certificates

3.1. Overview

A DCC [13] provides all information relating to a calibration in machine-readable form. Its structure mirrors the information that is required by ISO/IEC 17025 [14] for reporting the results of calibration. Consequently, a DCC is divided into four main sections:

• Administrative data (compulsory, regulated)—this section contains information that is typically displayed on the front page of a paper-based certificate. For example, identification of the calibration laboratory, the calibration object and the calibration service customer.
• Measurement results (compulsory, partially regulated)—this section allows measurement results, including uncertainty information, from different metrology domains and of different types to be presented. Currently, only measurement results that rely on the International System of Units (SI) can be provided in this section.
• Comments (optional, not regulated)—this section contains non-regulated information that is specifically intended for humans, e.g., proprietary data such as calibration-specific data sheets, formatting information, etc., and that cannot be used by computer without the need for human interpretation. The section may include graphical, video or audio information.
• Document (optional)—this section allows a human-readable version of the calibration certificate to be stored and allows users to view an electronic version of the certificate more akin to the traditional paper-based certificate.

The focus of this paper is on uncertainty-related aspects of the measurement results section of a DCC.

3.2. Measurement Results Section

A number of internationally recognised documents provide the foundation for the representation of measurement data within the D-SI: the BIPM SI brochure [9], the GUM [6], the VIM [5] and ISO 80000-1 [15]. The D-SI allows several types of measurement data—real, complex, univariate, multivariate, etc.—to be represented.

3.2.1. Univariate Real Quantity

Table 1 lists the components of the D-SI for a univariate real quantity. The information marked in bold shows the minimal information required and comprises the numerical value of the quantity and the unit of measurement. Additional information may optionally be provided: a label providing descriptive information for the quantity, e.g., to provide metadata related to the measurement, and the date and time of the measurement.

Table 1. Component structure for a univariate real quantity. Components in bold are mandatory. The notation ‘>’ indicates that the component on the right is a subcomponent of the component on the left.

real	>	value
	>	unit
	>	label
	>	dateTime

In practice, a measurement result is generally considered to be incomplete if it is not accompanied by information regarding its uncertainty. The D-SI allows for uncertainty information by extending the basic concept in Table 1 to allow the provision of an expanded uncertainty or a probabilistically symmetric coverage interval.

Table 2 and Table 3 list the components of the D-SI that can be used to represent a univariate real quantity with uncertainty information provided in the form of an expanded uncertainty and a coverage interval, respectively. For each case, uncertainty information is provided using an additional, optional, component which itself comprises a number of mandatory and optional components. Therefore, for example, uncertainty is a subcomponent of expandedUnc which is itself a subcomponent of real. Note that the unit of measurement is not explicitly provided for the uncertainty information but is implicitly inherited from the unit component.

Table 2. Component structure for a univariate real quantity with expanded uncertainty.

real	>	value
	>	unit
	>	expandedUnc	>	uncertainty
			>	coverageFactor
			>	coverageProbability
			>	distribution
	>	label
	>	dateTime

Table 3. Component structure for a univariate real quantity with coverage interval.

real	>	value
	>	unit
	>	coverageInterval	>	standardUnc
			>	intervalMin
			>	intervalMax
			>	coverageProbability
			>	distribution
	>	label
	>	dateTime

For expanded uncertainty, in addition to the numerical value and the unit of measurement, the mandatory information comprises the coverage probability and the expanded uncertainty and coverage factor corresponding to that coverage probability. The expanded uncertainty and coverage factor allow the standard uncertainty to be determined if required. Information about the probability distribution, e.g., distribution type, parameter values, may optionally be provided.

For a coverage interval, the additional mandatory information comprises the standard uncertainty, the coverage probability and the lower and upper limits of the coverage interval corresponding to the coverage probability. Again, information about the probability distribution may optionally be provided.

3.2.2. Multivariate Real Quantity

Frequently in metrology, it is necessary to consider a multivariate real quantity, i.e., a vector of real quantities. The D-SI allows multivariate quantities to be treated by employing a ‘list’ structure. In its most general form, the list structure allows a multivariate real quantity to be represented as shown in Table 4, where each element is of a type specified in Table 1, Table 2 and Table 3.

Table 4. Component structure for a multivariate quantity comprising a series of real quantities.

list	>	real
	>	real
		⋮
	>	real

GUMS2 [11] describes how both hyper-ellipsoidal and hyper-rectangular coverage regions can be defined for multivariate quantities. Table 5 and Table 6 list the components of the D-SI that can be used to represent a multivariate real quantity with uncertainty information provided in the form of a hyper-ellipsoidal coverage region and hyper-rectangular coverage region, respectively. For each case, uncertainty and covariance information is provided in the form of an additional component which itself comprises a number of components.

Table 5. Component structure for a multivariate real quantity with a hyper-ellipsoidal coverage region.

list	>	real
	>	real
		⋮
	>	real
	>	ellipsoidalRegion	>	covarianceMatrix
			>	coverageFactor
			>	coverageProbability
			>	distribution

Table 6. Component structure for a multivariate real quantity with hyper-rectangular coverage region.

list	>	real
	>	real
		⋮
	>	real
	>	rectangularRegion	>	covarianceMatrix
			>	coverageFactor
			>	coverageProbability
			>	distribution

Consider the covariance matrix

$$\begin{array}{c}\hfill V_{\mathbf{y}}=\left[\begin{array}{cc}{u}^2\left(y_1\right)& u(y_1,y_2)\\ u(y_2,y_1)& u^2\left(y_2\right)\end{array}\right]\end{array}$$

of size $2\times 2$. Table 7 lists the components of the D-SI that can be used to represent $V_{\mathbf{y}}$. Information is provided one column at a time, starting at column one, and within each column information is presented one row at a time, starting at row one. Therefore, in Table 7, information is presented in the order $u(y_1,y_1)\equiv u^2\left(y_1\right)$, $u(y_2,y_1)$, $u(y_1,y_2)$ and $u(y_2,y_2)\equiv u^2\left(y_2\right)$. The approach generalises straightforwardly for covariance matrices of larger size.

Table 7. Component structure for a covariance matrix of size $2\times 2$.

covarianceMatrix	>	column	>	covariance	>	value
					>	unit
			>	covariance	>	value
					>	unit
	>	column	>	covariance	>	value
					>	unit
			>	covariance	>	value
					>	unit

A multivariate quantity may consist of multiple measurements of quantities of the same type, e.g., measurements of temperature at a particular location taken at regular time intervals, or quantities of different types, e.g., measurements of different environmental factors within a laboratory. In the former case, if all quantities have the same unit of measurement, the list structure as presented leads to unnecessary repetition of information. The D-SI has been adapted to allow for more efficient representation in such cases. Table 8 shows how the same unit of measurement may be assigned to all individual quantities in a vector of real quantities. (For ease of reading, the optional label and dateTime components have been omitted.) When the listUnit component is used, there is no longer the mandatory requirement to provide a unit of measurement for each quantity (c.f. Table 1).

Table 8. Component structure for a multivariate quantity comprising a series of real quantities with the same unit of measurement.

list	>	listUnit
	>	real	>	value
	>	real	>	value
		⋮
	>	real	>	value
	>	real	>	value

The D-SI also allows for the same expanded uncertainty or coverage interval to be associated with all the real quantities of a multivariate quantity but this functionality is not discussed further in this paper. A complex quantity is treated as a special case of a multivariate quantity and is considered in [16].

3.2.3. Matrices and Tensors

The D-SI allows data of dimension higher than that of vectors to be treated using a ‘list of lists’ structure. This structure allows, e.g., a matrix to be represented as shown in Table 9. Depending on the application and the preference of the user, a matrix may be represented as a list of rows or a list of columns.

Table 9. Component structure for a matrix comprising a list of lists.

list	>	list
	>	list
		⋮
	>	list

3.3. Implementation of the Data Model

The D-SI may be implemented in the language of choice of the user. Within the SmartCom project, Extensible Markup Language (XML) [17] has been used, but the use of alternative languages such as JavaScript Object Notation (JSON) [18] is equally valid. The following features have been imposed:

• The expression of numerical values is compatible with decimal floating-point numbers in the ANSI/IEEE 754 double precision format [19].
• Date and time information is presented relative to Universal Coordinated Time (UTC) and complies with the format described in ISO 8601 [20] for legal local date and time with a difference to UTC.
• The bases for the expression of units of measurement are the BIPM SI brochure [9], the siunitx package for LaTeX [21] and IEC TS 62720 [22].
• Standard Unicode Transfer Format 8-bit (UFT-8) is to be used for all character strings including those that indicate numerical values.

Within the XML implementation of the D-SI [23], to reflect the dependence on the SI, all structural elements have the prefix ‘si’, e.g., si:real.

4. Examples

4.1. Univariate Real Quantity

To illustrate the ability of DCCs to capture uncertainty information obtained from MCM, consider the example of gauge block calibration from clause 9.5 of GUMS1 [7]. In this example, the length of a gauge block of nominal length $50\mathrm{mm}$ is determined by comparing it with a known reference standard that has the same nominal length. The measurand is the deviation from the nominal length.

The measurand is expressed as an explicit function of nine input quantities. The probability distributions assigned to the input quantities comprise scaled and shifted t-distributions, a rectangular distribution, a normal distribution, an arc sine distribution and rectangular distributions with inexactly prescribed limits.

For this paper, the calculation stage has been implemented using a fixed number $M={10}^6$ of trials (c.f. the implementation in GUMS1 where an adaptive approach was used). Using the component structure of Table 3, summary information from the calculation stage can be encapsulated as follows:

    <!-- MCM, 1e6 samples - Summary information -->

    <si:real>

      <si:label>Deviation from nominal length</si:label>

      <si:value>838</si:value>

      <si:unit>\nano\metre</si:unit>

      <si:coverageInterval>

        <si:standardUnc>36</si:standardUnc>

        <si:intervalMin>745</si:intervalMin>

        <si:intervalMax>932</si:intervalMax>

        <si:coverageProbability>0.99</si:coverageProbability>

      </si:coverageInterval>

    </si:real>

Using the component structure of Table 8, the full set of values $y_k,k=1,\dots ,M$, of the output quantity returned by the Monte Carlo calculation can be encapsulated as follows, showing only the first three ($k=1,2,3$) and final three ($k=M-2,M-1,M$) values:

    <!-- MCM, 1e6 samples - Output quantity values -->

    <si:list>

      <si:listUnit>\nano\metre</si:listUnit>

      <si:real>

        <si:value>829.5221</si:value>

      </si:real>

      <si:real>

        <si:value>873.3864</si:value>

      </si:real>

      <si:real>

        <si:value>822.9225</si:value>

      </si:real>

        …

      <si:real>

        <si:value>825.8857</si:value>

      </si:real>

      <si:real>

        <si:value>862.1964</si:value>

      </si:real>

      <si:real>

        <si:value>798.6789</si:value>

      </si:real>

    </si:list>

As discussed in Section 1, it would be impractical to generate a paper certificate that contains such a large number of numerical values. When written to file (without any spaces or indentation), the full set of values in the format above takes up approximately 48.6 MB (and requires $3M+3$ lines). For comparison, were the representation that allows the same unit of measurement to be assigned to all quantities not available, using the component structure of Table 4 would lead to a file of approximate size 78.2 MB ($4M+2$ lines).

4.2. Multivariate Real Quantity

To illustrate the ability of DCCs to present information for a multivariate real quantity, consider the example of simultaneous measurement of resistance and reactance from clause 9.4 of GUMS2 [11]. In this example, the resistance and reactance of a circuit element are determined by measuring the amplitude of a sinusoidally-alternating potential difference across its terminals, the amplitude of the alternating current passing through it, and the phase angle of the alternating potential difference relative to the alternating current. There are three output quantities: the resistance, the reactance and the impedance.

For this paper, the calculation stage has been implemented using a fixed number $M={10}^6$ of trials. Using the component structure of Table 8, summary information from the calculation stage can be encapsulated as follows:

    <!-- MCM, 1e6 samples - Summary information -->

    <si:list>

      <si:listUnit>\ohm</si:listUnit>

      <si:real>

        <si:value>127.732</si:value>

      </si:real>

      <si:real>

        <si:value>219.847</si:value>

      </si:real>

      <si:real>

        <si:value>254.260</si:value>

      </si:real>

      <si:ellipsoidalRegion>

        <si:covarianceMatrix>

          <si:column>

            <si:value>0.003364</si:value>

            <si:unit>\ohm\ohm</si:unit>

            <si:value>-0.04090216</si:value>

            <si:unit>\ohm\ohm</si:unit>

            <si:value>-0.02688114</si:value>

            <si:unit>\ohm\ohm</si:unit>

          </si:column>

          <si:column>

            <si:value>-0.04090216</si:value>

            <si:unit>\ohm\ohm</si:unit>

            <si:value>0.058081</si:value>

            <si:unit>\ohm\ohm</si:unit>

            <si:value>0.2281734312</si:value>

            <si:unit>\ohm\ohm</si:unit>

          </si:column>

          <si:column>

            <si:value>-0.02688114</si:value>

            <si:unit>\ohm\ohm</si:unit>

            <si:value>0.2281734312</si:value>

            <si:unit>\ohm\ohm</si:unit>

            <si:value>0.037249</si:value>

            <si:unit>\ohm\ohm</si:unit>

          </si:column>

        </si:covarianceMatrix>

        <si:coverageFactor>2.80</si:coverageFactor>

        <si:coverageProbability>0.95</si:coverageProbability>

      </si:ellipsoidalRegion>

    </si:list>

Using the component structure of Table 9, the full set of values ${\mathbf{y}}_k,k=1,\dots ,M$, of the output quantity returned by the Monte Carlo calculation can be encapsulated as follows, showing only the first ($k=1$) and final ($k=M$) values:

    <!-- MCM, 1e6 samples - Output quantity values -->

    <si:list>

      <si:list>

        <si:listUnit>\ohm</si:listUnit>

        <si:real>

          <si:value>127.763</si:value>

        </si:real>

        <si:real>

          <si:value>219.303</si:value>

        </si:real>

        <si:real>

          <si:value>254.397</si:value>

        </si:real>

      </si:list>

        …

      <si:list>

        <si:listUnit>\ohm</si:listUnit>

        <si:real>

          <si:value>127.838</si:value>

        </si:real>

        <si:real>

          <si:value>220.055</si:value>

        </si:real>

        <si:real>

          <si:value>253.699</si:value>

        </si:real>

      </si:list>

    </si:list>

When written to file (without any spaces or indentation), the full set of values in the format above takes up approximately 194 MB (and requires $12M+2$ lines).

5. Discussion

The GUM recommends that uncertainty information be presented in such a way that permits it to be used in a subsequent uncertainty calculation. On a calibration certificate, information about a quantity is frequently provided in the form of an estimate of the quantity and either an associated standard uncertainty or expanded uncertainty corresponding to a specified coverage probability. If the quantity is to be used in another calculation, it is common, in the absence of any additional information, to assign a Gaussian (or normal) probability distribution to the quantity. This assignment may be made even if the true probability distribution for the quantity is significantly different. Consequently, the results of the subsequent calculation may be unreliable.

Even if uncertainty evaluation has been undertaken using a Monte Carlo method, it is common for only the summary information mentioned above to be presented on the calibration certificate. The reasons are understandable, e.g., limiting the number of pages in the certificate to a reasonable value, the difficulty in putting to practical use the full set of values returned by the Monte Carlo method.

Digital calibration certificates (DCCs) provide two key benefits that immediately aid the reporting and use of a complete set of Monte Carlo results. First, the presentation of information in a fully machine-readable form. Second, through the application of the Digital SI (D-SI) data model, the potential to include much greater volumes of data than is currently practical in a paper-based or electronic (e.g., PDF-A) certificate.

For a subsequent uncertainty calculation, a second Monte Carlo approach can be implemented. The sample values for the first quantity can be read from the DCC and samples from (an approximation to) the probability distribution for the first quantity can be obtained by randomly sampling from that set of values. The storage and use in calculations of a large number (e.g., ${10}^6$) of sample values is facilitated in modern personal computers and mathematical software packages.

DCCs therefore provide the means to transfer uncertainty information that is encapsulated in a set of Monte Carlo samples. However, one should be aware of the circumstances under which the use of a Monte Carlo approach does not support transferability of results, e.g., as discussed in [24].

Although the D-SI was not designed with a Monte Carlo approach to uncertainty evaluation explicitly in mind, the data components in the D-SI are readily suitable for reporting the full set of results from a Monte Carlo calculation. As its use becomes more widespread, it is anticipated that the D-SI will be subject to updates, e.g., to take account of feedback from users from various metrology domains. One potential update could be the development of a specific component in the D-SI that allows all information from a Monte Carlo calculation to be encapsulated. That information could include details of the approach (standard or adaptive) and the number of Monte Carlo trials implemented. It may also be useful to provide all relevant information on the input quantities and their probability distributions as well as the measurement model (in an appropriate format).