^{1}

^{*}

^{1}

^{2}

^{2}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

The performance of 2D digital imaging systems depends on several factors related with both optical and electronic processing. These concepts have originated standards, which have been conceived for photographic equipment and bi-dimensional scanning systems, and which have been aimed at estimating different parameters such as resolution, noise or dynamic range. Conversely, no standard test protocols currently exist for evaluating the corresponding performances of 3D imaging systems such as laser scanners or pattern projection range cameras. This paper is focused on investigating experimental processes for evaluating some critical parameters of 3D equipment, by extending the concepts defined by the ISO standards to the 3D domain. The experimental part of this work concerns the characterization of different range sensors through the extraction of their resolution, accuracy and uncertainty from sets of 3D data acquisitions of specifically designed test objects whose geometrical characteristics are known in advance. The major objective of this contribution is to suggest an easy characterization process for generating a reliable comparison between the performances of different range sensors and to check if a specific piece of equipment is compliant with the expected characteristics.

It is well known that the performances of an imaging system are associated with several parameters, which depend on the kind of sensor that is considered. In 2D imaging, for example, various standards have been defined for photographic equipment [

In the last two decades, due to the rapid growth in performances of digital technologies, 3D imaging has seen significant advances with a corresponding decrease in costs, which has made three-dimensional acquisition and modeling more and more widespread in several application areas, ranging from industrial manufacturing to cultural heritage documentation [

Therefore, the testing procedures adopted by the 3D imaging industry are not univocally defined and the official technical features for end-users are not often easily comparable among them. For this reason, in order to take full advantage of 3D imaging systems some metrological laboratories have been set up in the last few years, pointing out not only the advantages, but also the limitations of these instruments. Besides the creation of specific laboratories, research studies are being conducted in various research centers aiming at defining a characterization processes for assessing 3D camera features.

This paper lies in this stream, and is focused on investigating an experimental process for creating a 3D equipment datasheet with quantitative and comparable data. The creation of standards and the definition of neutral and coherent test methodologies are critical for increasing user awareness in the application of 3D acquisition technologies. Although these standards should address the whole measuring process from acquisition to data processing [

The paper presents a short overview of previous works in the field of 3D equipment characterization in section 2. Section 3 is devoted to a general description of the proposed methodology, Section 4 analyzes the equipment tested and their nominal operating characteristics, Section 5 describes material and geometry of the custom target objects used in this experimentation, with a particular attention to their certification, Section 6 shows the experimental results organized according to the parameter under analysis. Conclusions and perspectives about this characterization approach are reported in the last section.

In the 3D imaging domain, the definition of best practices that led to standards started less than 15 years ago [

A few applications of the concept of Modulation Transfer Function (MTF) have been developed for determining the resolution level of a 3D imaging system and applied in the biomedical imaging area for X-Ray devices [

Metrological definitions are not yet shared and accepted in the 3D field, and this has led to the creation of specific working groups aiming at creating a metrological standard for 3D optical devices [

The methodology presented in this paper is based on the production of 3D images related to specifically developed 3D test objects, whose size and shape are known in advance, with a level of accuracy far better than the parameters we intend to evaluate. Such accuracy has been obtained by modeling each object with a CAD tool, manufacturing it with a suitable technology as, for example, Computer Numerically Controlled (CNC) machine tools and, independently of the specific manufacturing process, measuring each physical prototype with a Coordinate Measurement Machine (CMM) capable of giving an accuracy of below 4 micrometers in the xyz directions, while the parameters we intend to evaluate range from 20 to 300 micrometers (

The accuracy of each piece of equipment has been estimated by comparing the actual dimensions of each certified object with the measures attainable through their range maps, acquired with the non-contact range devices under test. Since some parameters were too dependent on the sampling pattern generated in any acquired range image, we decided to make wide use of primitive fitting in order to get each parameter as the result of an averaging process rather than a single measured value.

A clear example is the measurement of a cone height, which can be surely wrong unless the tip of the cone corresponds exactly with one of the 3D points acquired by the range device. Conversely, if a set of points representing a cone is fitted with a conical primitive, the cone height is determined with a high level of accuracy, independently of the sampling grid positioning, by measuring the height of the corresponding primitive. The deviation of such value from the cone height determined by CMM measurements will be the accuracy range device estimation for that particular parameter. This approach has been applied for each geometrical shape used.

Uncertainty estimation has been referred to both planar and curved surfaces. It has been obtained by calculating the standard deviation of each acquired range map with respect to the corresponding ideal geometries. We have used this approach since 3D devices may give different response when the optical axis is inclined with different angles to the explored area. For example, let us consider a set of points representing a cylinder acquired through a device oriented at right angles to the cylinder axis; the deviation of the actual 3D points from the corresponding primitive will be different in the areas where the device’s optical axis is perpendicular to the cylinder surface, while it will be progressively different as the angle changes. For this reason, both standard deviations given by a plane and by a curved surface are separately reported in the results.

Finally, the resolution has been estimated differently along the optical axis of each range device (z) and along the other two directions (xy). These two resolution parameters will be indicated hereafter as z resolution and horizontal resolution respectively. One key point of the proposed approach for resolution estimation consists in representing the range images associated with each 3D device as grey level images on which the analyses previously developed for the 2D imaging area can be applied. For example, if a black to white transition has to be replicated, the corresponding 3D target will be characterized by an abrupt jump of its 3D surface along the z direction, and the range device will be oriented in order to have its optical axis parallel to that direction, centered on the 3D feature. The range map will be thus transformed in a grey level image by assigning white and black to the minimal and maximal sensor-to-target distance respectively, with points of distances inside this range linearly grey coded. The resulting 2D image can be processed by means of ISO processes as those defined in ISO 12233 and ISO 16067. With such approaches the resolution can be estimated in two ways:

In our experiments, the horizontal resolution has been estimated in the latter way, while z resolution has been explored through a specific 3D object characterized by several steps of progressively smaller values, evaluating up to which level these were detectable on the range image.

Specific range maps have been acquired for each test object by using homogeneous nominal settings, in order to characterize the aforementioned parameters in comparable operating conditions.

The software packages used for extracting the parameter of interest from the range maps consist of specifically developed MATLAB procedures, public domain MATLAB scripts (sfrmat2), and commercial packages for both 3D data processing (Innovmetric Polyworks) and for checking resolution estimation (QuickMTF).

Differently from TOF range sensors, where each point of a range map is generated by measuring the time needed by a light pulse for going from the emitting sensor back to a photodetector, in triangulation range sensors the geometrical information is generated thanks to a scan head made by a light source generating one or multiple sheets of light, and projecting them at known angles on the target. A video camera, positioned at a fixed baseline from the source, acquire the corresponding light profiles and generates 3D data of the points belonging to each profile by estimating their parallax (

The instruments used for our experiment are shown in

The instruments are listed below with their main characteristics:

- ShapeGrabber® SG100: this laser line triangulation scanner is equipped with a high precision linear rail system that permits a 60 cm horizontal translation of the sensor side by side. The minimum working distance (Standoff) is 90 mm while the maximum is 290 mm. The focal length is fixed and each profile is made by 1,280 3D points.

- ShapeGrabber® SG1000: this is provided with a mechanical rotating head that permits a 330° rotation along its axis. The standoff is 250 mm, and the maximum working distance is 1,150 mm. The focal length is fixed and each profile is made by 1,280 3D points.

- Minolta® Vivid 910: includes three lens with different focal length: Tele (f = 25 mm), Middle (f = 14 mm) and Wide (f = 8 mm). It has a working distance ranging from 600 mm to 2,500 mm and is provided with a bi-dimensional CCD (640 × 480 pixel), that defines a fixed sampling grid on the imaged surface.

- Minolta® Vivid VI-9i: this is a model derived from the Vivid 910, provided with updated opto-electronic equipment but with the same 640 × 480 sensor size.

- Nextengine®: this is based on the Multistripe Laser Triangulation (MLT) technology. The 3D camera is equipped with a twin array of 4 solid state lasers and two 3 Mpixel CMOS RGB array sensors. The system makes the acquisitions in two different modes that correspond to two different baselines: wide mode, which requires the object to be 45 cm far from the front of the scanner, and the macro mode, where the object is positioned 16 cm far away. We have used the instrument in the Wide mode configuration and Fine modality, to assure a nominal resolution level compliant with the other devices.

- GOM® Atos II: is a pattern projection range sensor based on the triangulation principle. It is equipped with one pattern projector and two cameras with fixed focal lens (f = 17 mm) observing the object to be measured from different points of view in order to reduce the effects of occlusions.

- Minolta® Range7: \is a laser stripe device provided with two lenses: Tele (f = 25 mm) and Wide (f = 8 mm). The reflected light is received by a CMOS sensor that evaluate 3D information by acquiring 1.31 megapixels (1,280 × 1,024) images of the laser light profiles. The working distance ranges from 450 to 800 mm.

The instruments set-up of each device was chosen in order to have approximately the same sample spacing along the horizontal plane (0.3 mm). Sometimes this was not possible, as for example for the SG100, made for high resolution measurements at close range, where the minimal sampling step along the scan-line is 0.1 mm. In this case the choice was to set up the only parameter changeable (

In order to reduce measuring time and expenses, we have defined some targets with different geometries using standard materials, which we were able to produce using the facilities available in our laboratories (see

The first object is a “set of steps” (

The second target is a set of solids mechanically connected to a thick plate, and painted matte white to avoid reflections. As shown in

We have investigated the accuracy and uncertainty of the 3D data generated from these characteristic features, such as the cone slope, the cylinder diameter, and the angle between the two intersecting planes. The other parts of the object have not been used because they presented geometrical features similar to other test objects but less suitable for our tests.

This target is a parallelepiped block, made of rectified iron painted matte white to avoid reflections. It has dimension 100 mm × 60 mm × 270 mm and its main feature is to become a 60 mm vertical step, once laying over an horizontal planar reference with its larger face. Such step shape has been used as large bandwidth stimulus to the systems under test in order to investigate their xy resolution performances with a frequency domain analysis described in section 6.3.

In order to guarantee small deviations from planarity and low cost, we have decided to use a thick (11 mm) piece of glass. The particular manufacturing process of this material allows one to obtain a planar target with a peak deviation from the theoretical plane in the order of few micrometers. Geometrically, this suitable for testing our devices, characterized by measurement uncertainties and accuracies in the range of 20–400 micrometers. Optically, this material is not as suitable as it is geometrically, due to its transparency. This property makes it not compliant with an active range sensor, which works only of diffusive reflecting surfaces. We solved that by applying matte white paint on one side of the target. It was dried with an oven treatment, with the same process used for painting cars, to avoid as much as possible the generation of geometrical irregularities on the paint layer. This solution permitted to obtain, at low cost, a good planar and optically cooperative surface. The area of this planar reference (700 mm × 528 mm) was large enough for covering the acquisition field of all the instruments tested.

In this section we present the experimental approaches used to obtain accuracy, uncertainty and resolution from some range images taken in pre-defined conditions with all the devices under test, applied on the test objects mentioned above, starting from the “a priory” knowledge of their actual geometrical data.

This parameter is defined as the dispersion of the z coordinate of each point (target-to-sensor distance) around its average value. In other words, it represents the random component of the measurement error. This particular parameter can be estimated by calculating the distance of each measured point from its theoretical value, which is here given by the point on the corresponding fitted primitive. The standard deviation extracted from the statistical distribution of this sequence of distances gives an evaluation of the instrument uncertainty. The targets used for this test are both the planar reference and the set of solids.

Range maps generated by acquiring the planar reference through each range device, have been processed numerically in order to find the equation of the corresponding best fitting planes. The residual deviations between the actual measured points and the fitting plane, which was assumed to be the mathematical representation of the true physical reference, were first imaged and then statistically processed calculating the error histogram and the corresponding standard deviations [

An image like that in

In the histogram of

In addition, a pseudo-random component might be added on the laser based devices for the speckle effect. It is produced by the reflection of the coherent laser light on diffusive surfaces that, for their intrinsic roughness, give to any reflected photon a different phase contribution. This generates a destructive or constructing interference on the backscattered light when it reaches the recording sensor, depending on the actual position over the sensor area. Such effect involves a Gaussianly-distributed random parallax alteration between points at the same sensor-to-target distance that once added to the electronic noise effect, maintains the same statistical distribution. Seven different range maps corresponding to each analyzed 3D scanner have been analyzed, obtaining the uncertainty evaluation reported in

The deviation from the Gaussian behavior of the error histograms in _{1}, evaluated by fitting a plane on the whole framed area and calculating the statistics of residual 3D points-to-fitting plane deviations, and σ_{2}, evaluated on a 50 mm × 50 mm subarea in the center of the range image. In addition the same table reports the peak positive and negative deviation of each data set from the fitting plane, once outliers were manually eliminated (e.g., wrong 3D measurements due to reflection or clearly recognizable artifacts). This might also be considered indicator of the systematic error influencing σ_{1}, which after all can be considered as a cumulative index of measurement uncertainty and accuracy. On the other hand systematic alterations in most acquisition appear to be characterized by a very low spatial frequency. As shown in _{2}, being evaluated on a 50 mm × 50 mm central spot of each range map can be considered as an uncertainty index, purely due to random effects.

The object described in

A second range map was generated for acquiring the cone superimposed on the cylinder, keeping the camera axis approximately perpendicular to the cone axis. For both acquisitions the standard deviation (1σ) between the actual data and the fitting primitives (cone, parallelepiped, planes,

In any measurement device the concept of accuracy refers to the systematic component of the measurement error with respect to the real data.

We define here as

Similarly, we define as

Both aforementioned parameters are represented by single values even if originated by a cloud of 3D points, because of the extraction of these data is based on a fitting primitive, that operates an averaging of several raw 3D points for producing the actual single value (e.g., a cylinder fitted on a set of 3D points is a way for extracting one clean diameter value from several 3D noisy points).

Finally we define here _{max} and Δz_{min} are the maximum and minimum deviation of the actual range data from the fitted primitive expressed in micrometers and d is the diagonal of the framed area expressed in millimeters. Differently from the two previous parameters, given by an averaging process, this one is evaluated directly on the raw data, taking into account the worst samples, after the elimination of outliers. This parameter represents in practical terms the average accuracy error along z (in micrometers), for each millimeter explored by the range map along the xy plane.

All accuracy parameters have been extracted from range maps, using as test objects both the set of solids and the set of steps, as described in the next two sections.

From the peak positive and negative errors reported in _{max} and Δz_{min.} and dividing the result for the diagonal size of each analyzed image. The values reported in

It can be noticed that linear accuracy is always better with short focal lengths for both scanners usable with different focal lengths.

For checking the capability to catch the actual value of an angle between surfaces inclined each other, the range map of the set of solids has been acquired and processed.

The angular accuracy has been measured in two different ways. First we have evaluated the angle α between the two intersecting planes shown in the left part of _{m}. The value of the same parameter assumed as “true”, and indicated as α_{t}, is the one generated through the CMM measurements. The deviation Δα has been evaluated as difference between the measured and the true values, as shown in

Secondarily, the angle defined by the tip of the cone, corresponding to twice the angle between the generator and the axis of the cone has been evaluated. Similarly to the previous angle, the set of data acquired on the cone surface has been used for fitting a conical primitive. The resulting cone slope, indicated in _{m}, was considered as the angle originated by the measurement. Its deviation from the value assumed as true, given by the CMM, has been defined as Δβ.

Finally, the linear accuracy in this situation has been estimated by comparing the cylinder radius and the cone height extracted from the conical primitives associated with the acquired 3D data, and the corresponding CMM values.

As we can see from the values reported in

The linear accuracy is extremely good for GOM Athos II in both cone height and cylinder radius estimation. The first parameter has in general larger deviations, anyway below half millimeter for NextEngine, VI-910 with the 14mm lens and for the Minolta Range 7. Regarding the cylinder radius the best results were obtained with Minolta Range 7. An absolute error below 50 micrometers was obtained with all the other devices, with the exception of SG1000, that evidenced an error more than three times larger of the second worst device.

From the range map of this kind of test object we have been able to evaluate the linear accuracy along the z direction. At first a reference planar primitive has been created starting from the wider ring (higher sensor-to-object range) and used as zero reference. Then, for each step, a plane parallel to the zero reference has been created and located at a distance defined by the CMM measurements reported in section 5.1. The set of 3D points belonging to the specific set were then selected and the average distance between those points and the related plane was evaluated (

Since this selection was made by hand, this process made evident the need of distinguishability of points belonging to a specific step, which was not the same for the different range devices (see 6.3).

Two different resolution aspects have been examined: limitation in z-resolution due to the opto-geometric configuration (distance-to-baseline ratio), and horizontal resolution capability mainly due to optical performances.

In the first case, the “set of steps” object has allowed us to test the z resolution. Range maps of the sequence of steps have been acquired with different sensors. The z resolution capacity of every instrument has been detected by means of the user’s capability of identifying and selecting the set of points belonging to each step of the object. This activity has been necessary for calculating the root mean square distance between the selected points and the primitive defined by CMM measures (see accuracy analysis). When the points that should lie on the same plane are not distinguishable from those belonging to the adjacent step, that point is classified as an undistinguishable step; the last distinguishable step is defined as the z resolution limit.

The xy resolution has been estimated by acquiring a sharp edge of the metallic parallelepiped. The edge of the range map has been transformed into a B/W image by coding the distance (z) as a gray level. The ISO slanted edge analysis has been applied on the resulting image [

The z resolution test involved a check on the range maps originated by acquiring the set of steps test object with all the different devices under test. The same set of data were used to verify the linear accuracy along optical axis.

The selection of circular sets of 3D points corresponding to the different steps was made according to their distance from the object axis. However, the influence of uncertainty and accuracy on the actual optical resolution of each device imposed a well defined criteria for deciding if a set of 3D points is distinguishable from the adjacent ones.

The relationship between the average given by a certain set and the value assumed as true seen in the previous section, would suggest the possibility to detect instrumentally a distinction between sets even when characterized by very small z differences.

For example deviations below 10 micrometers in the detection of all steps have been found, an amount lower than the smaller step to be detected (16 μm). Although this instrumental possibility, since this method is the 3D counterpart of the direct ISO resolution detection, we decided to leave to an operator the final decision about the distinguishability of each subset of data, depending on the color-coded deviation map of the acquired data compared to a plane located in correspondence of the first ring of data (zero reference).

The same test was performed with different color maps and range of analysis, basically with the same result. This is probably due to the randomness of 3D data, expressed by the standard deviations reported in

In order to exemplify this concept, a simulation of a set of random data Gaussianly-distributed around a predefined z level, have been considered, with a standard deviation α of 30 micrometers. In this way 10,000 points have been generated, half around z = 0, and half around z = step, with three different step height, given by σ, 2σ, and 3σ respectively, and statistically processed as a whole set of data. As highlighted by

Using the range data processing described above for checking z-accuracy it is possible to see directly which is the minimal z-deviation that appears detectable to the average user. With reference to the symbols reported in

In addition to the “set of steps” target, used here for a direct estimation of resolution along the z axis, the xy resolution of a 3D system can be tested indirectly by analyzing its output in the frequency domain.

The concept of frequency is easily understandable in acoustics, where a deep voice is associated to a low frequency, while a high note corresponds to a high frequency. The frequency, in that case, represents the number of cycles of an acoustic wave in the time lag of one second, and therefore the measurement unit is the inverse of time (1 Hz = 1 /sec).

In the optical case we consider spatial frequency, with reference to the cyclic repetition of image details rather than sound waves. If we consider for example an image pattern made of a sequence of vertical black and white strips, we can evaluate the maximum number of transitions in a fixed space actually acquirable by an optical digital system, as the measurement of its capability to digitize the complex iconographic content of an image.

This capability is influenced both by the optical resolution of the lens, and by the bandwidth of the electronic system that will operate the amplification and the A/D conversion of the sensor signal. As in an electro-acoustical chain (

The practical way for evaluating the MTF of an optical system consists in estimating the capability to reproduce an abrupt luminance variation, by analyzing an acquired image containing a black-to-white transition, in the frequency domain.

Given that the image sensor is digital, the black-to-white transition may involve a different number of pixels depending on the relative positioning between the pattern and the sampling grid. This is why the edge is not chosen for example exactly vertical. In this case the sampling would in fact remain the same in all the image lines. A slanted edge ensure a different relative positioning of the black-to-white transition respect to the sampling pattern, as shown in

Since the spectral analysis for estimating the MTF, is made on the transition, in order to average the effect of the relative grid-pattern positioning on several pattern rows, a pattern made with a straight slanted edge is used, attenuating in this way possible artifacts induced by any specific sampling.

In order to make evident the relationship between the amount of blurring and the related MTF, in the

As shown in

In addition to this qualitative explanation of the relationship between MTF and image sharpness, a quantitative parameter can be conventionally defined as the number of cycles per pixel (or line pairs per millimeter), corresponding to 50% of the spectral peak (MTF50).

The evaluation here described can be done with public domain software, as the MATLAB script sfrmat2, developed by Peter Burns (Kodak) [

This concept can be transferred to 3D sensors, by converting a range map in a gray level image, where each camera-to-target distance is coded with light gray levels for points close to the camera and darker gray levels, as the detected points become farer.

The slanted edge can be generated with a parallelepiped, as the one shown in

The oblique straight line generated through the parallelepiped object is useful for ensuring a multiplicity of sampling grid positioning over the black-white transition, shown for the 2D image in

It is important to notice that the image generated from the range map has a 1:1 correspondence between 3D points and image pixels, with no re-sampling. In order to have the best possible correspondence between device performances and image indexes, any post-processing on range maps, such as smoothing, cleaning,

In

In this paper we have presented a low cost method for characterizing 3D cameras. Specifically, the use of ISO derived approaches have allowed us to easily obtain results by using methods that have been already accepted in other domains. In the experimentation, seven different pieces of equipment with nine configurations, nominally similar in performance, have given very different results, thus evidencing the need of standard parameters for characterizing a 3D camera.

Accuracy, uncertainty and resolution parameters have been evaluated during an experimental phase that involved specific certified test objects. Angular accuracy appears to be good for each range sensor (below 7% errors in the worst case), while linear accuracy seems more influenced by the range sensor cost. In particular the parameter Relative Linear Accuracy, easily evaluable in a lab with a reference plane, gives a quality classification of the devices under test that has a correspondence with the quality “perceived” by expert users and with the equipment cost.

The measurement uncertainty evaluated on planes ranges from 5 to 60 micrometers and is roughly coherent with what declared by each manufacturer. The same parameter, when evaluated on a cylinder reveals an amazing difference between different range devices. This is due to cameras capability in maintaining a good capture quality with the camera axis very angled respect to the surface to be captured; for example, a very good uncertainty for Minolta Range 7 and GOM on the plane (5 and 8 μm respectively), on the cylinder becomes 18 μm for the Range 7 (2×) and 159 μm for the GOM (32×).

At last the obtained parameters show a qualitative and quantitative evaluation of horizontal and z resolution. The cheapest system that we have tested (NextEngine) cannot give a readable result up to the fifth step of the “set of steps” test object, while two of the most expensive (Minolta Range 7 and GOM Atos II), have deviations smaller than 20 micrometers for the whole range of 10 steps. The evaluation of xy resolution seems to be clearly discriminated thanks to the transposition of the MTF based approach to 3D imaging.

These kinds of tests represent a preliminary approach for defining a process for range camera characterization. Future work for generating 3D versions of other 2D ISO targets has to be performed, in order to enhance the number of quantitative data obtainable with simple test objects and procedures. At the moment, this methodology seems to give usable results for verifying instruments performances in standard laboratory activities.

The authors would like to acknowledge the support given by the Department of Mechanical Engineering, and in particular: Giovanni Moroni for the CMM measurements, and Ing. Pasquale Aquilino for the accurate realization of some test samples.

Standard targets for 2D imaging equipment characterization:

Set of 3D laser scanner technology used in our experiment, from left to right:

Test objects used in the experiments:

Comparison between the range data originated by a laser scan of the planar reference made with the Minolta V9i scanner (25 mm lens) and the corresponding best fitting plane:

Deviations from the fitting plane of the range devices under test, color coded as in

Components under analysis for the “set of solids” test object:

“Set of steps” experiment:

Color coded deviation of the range map originated by the acquisition of the set-of-steps with a Minolta Vi with 14 mm lens respect to the reference plane obtained by best-fitting the first plane (zero reference). It appears clear that with this scanner the first two steps are not recognizable.

Simulated 3D acquisition of a step 30, 60 and 90 μm high (from left to right), with a measurement uncertainty of 30 μm. Raw data are in the upper row, corresponding frequency histograms in the lower one. Groups of data are statistically undistinguishable in the first two cases.

Pixel values (0 = black, 1 = white) originated on an horizontal line by different sampling grid positioning (gray line) respect to an abrupt black to white transition.

Example of slanted-edge with a blurring level progressively stronger:

Plot of MTF related to the four simulated cases described in

Plot of MTF related to the seven combinations of 3D scanner/lens analyzed in the paper.

Nominal characteristics of the instruments used for the tests, at the chosen working distances, available from the producers data sheets (all values are in mm).

NextEngine | Not Avail. | 430 | 0.4 × 0.4 | 0.38 | Not avail. |

GOM Atos II | 17 | 750 | 0.3 × 0.3 | 0.03 | Not avail. |

Minolta VI-9i | 14 | 750 | 0.4 × 0.4 | 0.13 | 0.02 |

Minolta VI-9i | 25 | 1200 | 0.3 × 0.3 | 0.12 | 0.03 |

Minolta VI-910 | 14 | 750 | 0.4 × 0.4 | 0.30 | 0.07 |

Minolta VI-910 | 25 | 1200 | 0.4 × 0.4 | 0.30 | 0.08 |

ShapeGrabber SG 1000 | Not Avail. | 350 | 0.2 × 0.3 | 0.16 | 0.03 |

ShapeGrabber SG 100 | Not Avail. | 200 | 0.1 × 0.3 | 0.05 | 0.02 |

Minolta Range 7 | 8 | 760 | 0.1 × 0.1 | 0.06 | 0.01 |

Nominal and actual step size of the “set of steps” test object shown in

15 | 30 | 60 | 120 | 240 | 480 | 960 | 1920 | 3840 | 7680 | |

16 | 32 | 60 | 120 | 239 | 480 | 960 | 1920 | 3838 | 7672 |

Plane fitting parameters obtained from each analyzed range sensors; σ_{1} is the standard deviation evaluated on the whole range map while σ _{2} on the 50 mm × 50 mm central area.

_{1} (μm) |
_{max} (μm) |
_{min} (μm) |
_{2} (μm) | ||
---|---|---|---|---|---|

NextEngine | 137 × 125 | 77 | 390 | −485 | 35.2 |

GOM Atos II | 360 × 290 | 18 | 55 | −146 | 4.6 |

Minolta VI-9i (14 mm) | 247 × 191 | 45 | 294 | −106 | 18.5 |

Minolta VI-9i (25 mm) | 208 × 165 | 61 | 366 | −297 | 35.9 |

Minolta VI-910 (14 mm) | 251 × 188 | 66 | 427 | −443 | 46.4 |

Minolta VI-910 (25 mm) | 220 × 167 | 133 | 354 | −532 | 59.9 |

ShapeGrabber SG1000 | 527 × 238 | 164 | 734 | −490 | 27.3 |

ShapeGrabber SG100 | 435 × 117 | 53 | 194 | −348 | 28.8 |

Minolta Range7 | 320 × 255 | 28 | 191 | −197 | 8.4 |

Uncertainty related to the different geometries analyzed in the set of solids.

NextEngine | 33 | 29 | 114 | 53 |

GOM Atos II | 15 | 18 | 159 | 74 |

Minolta VI-9i (14 mm) | 39 | 33 | 97 | 36 |

Minolta VI-9i (25 mm) | 90 | 76 | 95 | 134 |

Minolta VI-910 (14 mm) | 47 | 49 | 68 | 97 |

Minolta VI-910 (25 mm) | 64 | 56 | 97 | 327 |

ShapeGrabber SG1000 | 83 | 65 | 116 | 52 |

ShapeGrabber SG100 | 26 | 25 | 85 | 50 |

Minolta Range7 | 16 | 21 | 18 | 18 |

Relative linear accuracy evaluated on the range devices under test.

_{LR} (μm/mm) | |||
---|---|---|---|

NextEngine | 875 | 185.5 | 4.72 |

GOM Atos II | 201 | 462.3 | 0.31 |

Minolta VI-9i (14 mm) | 400 | 312.2 | 1.28 |

Minolta VI-9i (25 mm) | 663 | 265.5 | 2.50 |

Minolta VI-910 (14 mm) | 870 | 313.6 | 2.77 |

Minolta VI-910 (25 mm) | 886 | 276.2 | 3.21 |

ShapeGrabber SG1000 | 1.224 | 578.2 | 2.12 |

ShapeGrabber SG100 | 542 | 450.5 | 1.20 |

Minolta Range7 | 388 | 409.2 | 0.95 |

Angular and linear accuracy report. Deviation between angles and linear dimensions (Radius and Height) evaluated by means of the CMM equipment, and corresponding angles and dimensions evaluated from planes fitted onto the primitives or extracted directly from the range maps.

NextEngine | 0.11 | 0.02 | −221 | −12 |

GOM Atos II | 0.11 | 0.01 | 16 | 44 |

Minolta VI-9i (14 mm) | −0.04 | −0.06 | 761 | 54 |

Minolta VI-9i (25 mm) | −0.39 | −0.07 | 547 | −32 |

Minolta VI-910 (14 mm) | 0.24 | −0.01 | 152 | 11 |

Minolta VI-910 (25 mm) | 0.09 | 0.03 | −462 | −49 |

ShapeGrabber SG1000 | −0.06 | −0.02 | 738 | 165 |

ShapeGrabber SG100 | 0.01 | 0.07 | −530 | 41 |

Minolta Range7 | −0.03 | 0.02 | −156 | 1 |

Step-to-reference distance deviation. RMS distance between the plane located at the distance from the bottom given by the CMM measurement and the corresponding set of data originated by the range camera. Some of the cells in the table have not been filled due to the undistinguishbility between adjacent sets of data according to the definition given in section 6.2. Step 1 is the smaller (16 μm).

NextEngine | / | / | / | / | 3 | 6 | 30 | 16 | 49 | −35 | 12 |

GOM Atos II | / | / | 18 | 1 | −13 | 2 | −3 | −14 | 9 | 4 | 1 |

Minolta VI-9i (14 mm) | / | / | 24 | 11 | 2 | 6 | 11 | 24 | −6 | −2 | 9 |

Minolta VI-9i (25 mm) | / | / | 14 | −2 | 4 | 3 | 28 | 3 | 14 | 26 | 11 |

Minolta VI-910 (14 mm) | / | / | 13 | 14 | 27 | 28 | 35 | 49 | 56 | 63 | 36 |

Minolta VI-910 (25 mm) | / | / | 37 | 33 | 47 | 67 | 89 | 9 | 2 | −28 | 32 |

ShapeGrabber SG1000 | / | / | 0 | −17 | −6 | −7 | −13 | −17 | −27 | −51 | −17 |

ShapeGrabber SG100 | −31 | −23 | −10 | −12 | −8 | 13 | 15 | 11 | 24 | −1 | −2 |

Minolta Range7 | 7 | 6 | 6 | 3 | 4 | 3 | 3 | 7 | 10 | 10 | 6 |

Values of MTF50 for the spectra shown in

| |
---|---|

1.063 | |

0.247 | |

0.155 | |

0.087 |

Values of MTF50 for the different scanners analyzed in the paper, whose spectra are reported in

NextEngine (Wide) | NA |

GOM Atos II | 1.65 |

Minolta VI-9i (Middle) | 0.64 |

Minolta VI-9i (Tele) | 1.35 |

Minolta VI-910 (Middle) | 0.65 |

Minolta VI-910 (Tele) | 1.63 |

ShapeGrabber SG1000 | 0.19 |

ShapeGrabber SG100 | 0.78 |

Minolta Range7 | NA |