A Critical Comparison of 3D Digitization Techniques for Heritage Objects ^†

Abstract

Techniques for the three-dimensional digitization of tangible heritage are continuously updated, as regards active and passive sensors, data acquisition approaches, implemented algorithms and employed computational systems. These developments enable higher automation and processing velocities, increased accuracy, and precision for digitizing heritage assets. For large-scale applications, as for investigations on ancient remains, heritage objects, or architectural details, scanning and image-based modeling approaches have prevailed, due to reduced costs and processing durations, fast acquisition, and the reproducibility of workflows. This paper presents an updated metric comparison of common heritage digitization approaches, providing a thorough examination of sensors, capturing workflows, processing parameters involved, metric and radiometric results produced. A variety of photogrammetric software were evaluated (both commercial and open sourced), as well as photo-capturing equipment of various characteristics and prices, and scanners employing different technologies. The experimentations were performed on case studies of different geometrical and surface characteristics to thoroughly assess the implemented three-dimensional modeling pipelines.

1. Introduction

The importance attributed by the scientific community to documenting heritage objects is causally related to the needs for protection, conservation, and valorization. In accordance with the international literature, some of its areas of application include conservation, digital restoration, digital archiving, augmented or virtual reality, the 3D printing of replicas, real-time documentation of archaeological excavations, and monitoring [1].

Heritage case studies related to detailed digitization and visualization, such as surveys of utilitarian, decorative and ritual objects, historical architecture details, paintings, murals, rock art, engravings, and in situ archaeological documentation of fragmented objects, require 3D models of high visual fidelity and accuracy [2]. The continuous advancements on new active and passive sensors for reality capture, data acquisition techniques, processing algorithms, computational systems, and moreover the constant updates on existing hardware and software for the reconstruction of complex geometries, constitute a significant factor that makes the high-resolution recording, processing, and visualization of detailed heritage data more feasible. These developments have enabled more automation, higher velocities, increased accuracy, and precision. Especially, improvements in new instruments and digital tools, such as handheld scanners and photogrammetric automated or semi-automated software, provide powerful 3D digitization solutions for the experts and inexperienced users both [3,4,5,6].

Accurate and high-resolution heritage 3D modeling has been investigated with various passive and active sensors aiming to reconstruct surface characteristics accurately. Triangulation-based laser scanning [7], structured light scanning [8], modeling with range imaging (RGB-D) cameras [9], and image-based photogrammetric modeling [10,11,12] constitute some of the techniques explored for the 3D digitization of heritage objects. Although triangulation scanning, structured light scanning, and image-based modeling systems implementing structure-from-motion (SfM) and multi-view stereo (MVS) algorithms [13] have prevailed for large-scale heritage applications, mainly due to the combined: lower costs, faster data acquisition and processing, high precision, and ability to capture high-resolution texture [8,14,15,16]. Many contemporary workflows combine more than one technique to optimize digital 3D results [17,18]. The advantages of the SfM–MVS combination are also evident for complex case studies, such as low-feature or featureless artifacts [19,20], and metric modeling from beyond-visible spectrum imagery [21].

Metric evaluations have been carried out regarding heritage objects, by Remondino et al. [22], Nabil and Saleh [23], Galizia et al. [24], Bianconi et al. [25] for SfM software, by Evgenikou and Georgopoulos [26] and Menna et al. [27] on scanning versus SfM photogrammetric software approaches and by Kersten et al. [28,29] and Morena et al. [30] for portable/handheld scanners. Additionally, due to recent significant improvements in mobile phone camera technology, regarding sensors’ quality and camera software performance, smartphone cameras have been evaluated for the post-texturing of models from active sensors and for the direct image-based metric modeling of heritage objects [31,32,33]. Despite frequent assessments concerning the aforementioned digitization techniques being found in bibliography, they quickly become outdated due to the rapid technological developments. Furthermore, considering that the productive documentation of heritage assets not only means digitization but also the ability to effectively communicate its results with heritage experts and the public alike, metric comparisons of digitization techniques need to be updated regularly. These assessments should not only seek to evaluate cost and time-effectiveness, but additionally, to have solidly defined the heritage-related results and metadata, so that the limits of their use for dissemination can also be well defined.

Metric heritage modeling consists of three factors: the object (size, geometry, surface characteristics, materials), data acquisition (sensors, conditions, referencing/scaling techniques, and workflows), and processing (hardware, software, algorithms, outputs). For metric assessments to be performed for any 3D digitization technique, two out of these three parameters must remain constant. In this context, the presented research aims to provide a comprehensive comparison on different SfM-based photogrammetric, and scanning workflows for small-sized heritage objects’ digitization, by keeping constant most of the essential parameters of the data acquisition procedures and by altering the passive or active sensors utilized or the software involved in the image-based modeling approaches implemented. In addition, due to the specific interest in the metric testing of the results produced, their comparability was taken into consideration, for planning the acquisition of datasets.

Therefore, in the second section of this paper, a thorough comparison between different commercial and free photogrammetric software implementations based on the SfM image-based modeling approach was accomplished for the imagery datasets collected with three cameras of different characteristics. The third section describes the tests performed with different phase-based laser and structured light portable scanners. The fourth section refers to further evaluations that were performed on final meshes and the fifth to comparisons between the meshes produced by scanning and image-based 3D modeling. The last section is devoted to offering some concluding remarks and research aims for further investigation.

Furthermore, it should be highlighted, that in the presented research, significant importance is attributed not only to the metric comparison and visual assessment of the 3D reconstruction results, but also to the prices of the sensors and software and to the durations of the processing workflows, as they are crucial factors for the selection of instrumentation and software for heritage modeling, especially in cases of in situ, mass, or rapid digitization.

The test objects for this study (presented in Figure 1) were:

• A copy of Early Cycladic II Spedos-variety marble figurine, dimensions: 4 cm × 4 cm × 16 cm;
• A Roman column capital replica, dimensions: 45 cm × 45 cm × 45 cm;
• A bust of Francis Joseph I of Austria from the Accademia Carrara di Bergamo (Province of Bergamo, Lombardy, Italy), dimensions: 40 cm × 50 cm × 90 cm; and
• A small 19th century religious stone sculpture of Christ Crucified from Castello di Casotto (Province of Cuneo, Piedmont, Italy), dimensions: 31 cm × 22 cm × 5 cm.

2. Image-Based 3D Modeling

2.1. Data Acquisition

The instrumentation used for the collection of imagery (specifications in

Table 1. Specifications of the employed imaging sensors.

Type	Mid-Size DSLR	Compact DSLR	Huawei P30 Phone Camera
Brand	Canon	Canon	Sony
Model	EOS 5DS R	EOS 1200D	IMX 650
Resolution	52 MP	18 MP	40 MP
Sensor Size	full frame	APS-C	1/1.7”
Pixel Size	4.14 mm	4.31 mm	0.93 mm
Sensor Type	CMOS	CMOS	CMOS BSI
Lens used	Canon EF 24–105 mm f/4L IS USM	Canon EF-S 18–55 mm IS II	5.6 mm (integrated)

) consisted of a full-frame Canon EOS 5DS R digital single-lens reflex (DSLR) camera with a Canon EF 24–105 mm f/4L IS USM lens (USD 4800), a Canon EOS 1200D DSLR camera (APS-C image sensor) with Canon EF-S 18–55 mm f/3.5–5.6 IS II lens (USD 380) and a Huawei P30 smartphone (USD 485) with 5.6 mm (27 mm equivalent) f/1.7 lens camera (Sony IMX 650 Exmor RS sensor; Leica optics).

Imagery for the figurine copy and the small sculpture was acquired using a turntable; artificial targets were placed around the objects to scale the scenes. A tripod was utilized to stabilize the cameras in order to prevent micro-blur. Regarding the case studies of larger dimensions, images were acquired obliquely with large overlaps; an invar scale bar was also photographed in the scene for scaling. Despite the different resolutions of the imaging sensors, it was attempted to maintain similar object sampling distances by considering the distance from the object and the available focal lengths to acquire comparable data. Depth-of-field (DoF) was calculated during acquisition, as sharpness was also considered. The characteristics of the datasets are summarized in

Table 2. Characteristics of imagery datasets.

Dataset	Object	Camera Model	Mega- Pixels	f (mm)	Distance (m)	No. of Images	f-Stop	Exposure (s)	ISO
1	Figurine copy	EOS 5DS R	52	24	0.25	50	f/7.1	1/20	200
2	Figurine copy	EOS 1200D	18	18	0.20	50	f/8	1/20	200
3	Figurine copy	Exmor RS IMX650	40	5.6	0.25	50	f/8	1/20	200
4	Capital replica	EOS 5DS R	52	35	0.88 *	50	f/7.1	1/40	200
5	Capital replica	EOS 1200D	18	18	0.69 *	50	f/8	1/40	200
6	Stone bust	EOS 1200D	18	18	0.90 *	50	f/16	1/60	100
7	Small sculpture	EOS 1200D	18	18	0.38	142	f/16	1/15	100
8	Small sculpture	EOS 1200D	18	55	0.27	60	f/16	1/15	100
9	Small sculpture	Exmor RS IMX650	40	5.6	0.12	60	f/1.8	1/50	100

Note: * signifies average values.

. The last two datasets captured only the upper part of the small sculpture.

2.2. Processing Software and Parameters

The photogrammetric software solutions employed for the SfM–MVS approach image-based modeling included:

• Agisoft Metashape Professional 1.5.1 (USD 3499);
• 3DFlow Zephyr Aerial 4519 (USD 4329) [34];
• Pix4Dmodel 4.5.3 (USD 49/month);
• Autodesk ReCap Photo 19.3.1.4 (web-based; ReCap Pro USD 54/month);
• Regard3D 1.0.0. (free and open sourced) which employs a K-GRraph matching algorithm and implements the Multi-View Environment [35] for dense scene reconstruction;
• A pipeline combining VisualSFM [36,37]—a GPU-based bundler for SfM, CMVS [38] for MVS dense scene reconstruction, and MeshLab for Screened Poisson Surface Reconstruction [39] and mesh color-texturing [40].

The image-based processing solutions are herein referred to with the abbreviations AMP, FZA, P4D, ARP, R3D and VCM, respectively. For the datasets’ photogrammetric processing, a customized laptop was used, with a 6-core Intel i7-8750H CPU at 2.2 GHz (Max 4.1 GHz), 32 GB RAM, and NVIDIA GeForce RTX 2070 GPU.

To effectively evaluate the performance of the software implemented and the effects of utilizing different imaging sensors, similar parameters, when applicable, were selected for all the image-based modeling workflows, as summarized in

Table 3. Processing parameters of image-based photogrammetric modeling.

Reconstruction Step	Parameter	Value
Feature detection and matching alignment	Key point density	High (50K)
	Tie point density	High (50K)
	Pair preselection	Higher matches
	Camera model fitting	Refine
Dense matching	Point density	High
	Depth filtering	Moderate
Mesh generation	Max number of faces	5M (10M for capital replica)
	Surface interpolation	Limited
Texture generation	Texture size	8192 × 8192 pixels
	Color balancing	Disabled

. Standard semi-automatic digitization approaches were implemented in all cases, starting with the reconstruction of a sparse cloud, then densifying it after estimating depth maps, creating a mesh with the use of triangulation algorithms, and finally texturing the generated mesh with ortho-photo adaptive approaches. No manual removal of noise was performed, apart from deleting the scene and other components unconnected to the object, selected by component size.

Concerning the implemented algorithms, it should be highlighted that the free software VisualSFM utilizes scale-invariant feature transform (SIFT)-based detection and description [41], while Regard3D uses A-KAZE [42] and Local Intensity Order Patterns (LIOP) for this purpose [43]; in contrast, 3DFlow Zephyr uses a modified Difference-of-Gaussian (DoG) detector. Furthermore, both Metashape and Zephyr perform global bundle adjustment, however, VisualSFM and R3D implement Incremental SfM. Moreover, while all other software utilized Poisson Surface Reconstruction to generate the triangulated mesh, in FZA, an edge-preserving algorithmic approach was selected to compare the results.

The masking of the background on the images was performed in all commercial software, barring ARP that does not allow the user to intervene in any processing step. In P4D, the application of annotations takes place only after an initial full scene reconstruction, if images are correctly aligned, meaning that for the case study of the capital an extra 1:03:56 was required for the EOS 5DS R dataset and an extra 0:37:41 was required for the EOS 1200D dataset, for the first dense cloud and textured mesh reconstruction, additionally of what is reported below in results (Section 4).

The dense point clouds deriving from free reconstruction pipelines—where there was no option for masking unwanted areas of the imagery—were cleaned automatically using Statistical Outlier Removal (SOR) to efficiently remove noise before mesh generation. A maximum octree depth of 13, limited surface interpolation, and specific limits for the number of triangular faces were selected in those solutions that allowed the customization of the parameters for the 3D mesh generation step. Mesh texturing was performed without color or exposure-balancing the imagery, and without averaging values from multiple images, creating single-image file textures.

2.3. Results

For the figurine case study, only AMP and FZA were able to fully reconstruct the scene correctly from datasets 1 (EOS 5DS R), 2 (EOS 1200D), and 3 (Exmor RS IMX650). FZA required significantly more processing time, producing generally sparser results. Similar re-projection errors were observed. P4D was not successful in reconstructing the object from any dataset, and ARP results included partial reconstructions with noise and fictitious surfaces (Figure 2). The VCM pipeline entirely reconstructed the scene from dataset 1 (figurine, EOS 5DS R), but a small amount of noise remained after triangulation, affecting the final mesh, and the texturing results were problematic. Furthermore, the VCM pipeline was not able to reconstruct the object from dataset 2 (figurine, EOS 1200D) and resulted in an incomplete point cloud with a lot of noise for dataset 3 (figurine, Exmor RS IMX650; see Figure 2). R3D resulted in dense point clouds that included a big percentage of noise for dataset 1 (figurine, EOS 5DS R) and 3 (figurine, Exmor RS IMX650; see Figure 2), that was not possible to be removed automatically (or manually) and therefore were not exploited further to construct 3D meshes. Results from SfM–MVS photogrammetric processing are listed in

Table 4. Photogrammetric results, datasets 1–3.

		Dataset 1		Dataset 2		Dataset 3
	Software	AMP	FZA	AMP	FZA	AMP	FZA
Sparse Cloud	Images Aligned	50	50	50	50	50	42
	Matching time (hh:mm:ss)	00:00:40	00:02:48	00:00:18	00:01:40	00:00:41	00:05:34
	Alignment time (hh:mm:ss)	00:00:19	00:01:11	00:00:06	00:00:20	00:00:10	00:00:34
	Tie points (1000 points)	98	24	29	19	77	27
	Projections (1000 points)	321	136	92	91	212	118
	Adjustment error (pixels)	0.49	0.79	0.54	0.46	0.65	0.72
	Resolution (mm/pixel)	0.05	0.05	0.06	0.06	0.04	0.04
Dense Cloud	Processing time (hh:mm:ss)	00:10:31	01:16:39	00:04:31	00:24:03	00:09:09	00:46:40
	Point count (1000 points)	1832	591	1169	370	2414	3920
Triangle Mesh	Processing time (hh:mm:ss)	00:00:21	00:00:08	00:00:16	00:00:47	00:00:30	00:00:10
	Faces (1000 triangles)	4482	1168	2846	737	5000	1551
	Vertices (1000 points)	2246	589	1427	369	2514	783
Texture	Processing time (hh:mm:ss)	00:04:07	00:04:01	00:02:46	00:01:25	00:05:49	00:02:32
	Total time (hh:mm:ss)	00:15:58	01:24:47	00:07:57	00:28:15	00:16:19	00:55:30

.

For the capital case study, all software solutions could fully reconstruct the scene correctly (

Table 5. Photogrammetric results, datasets 4 and 5.

		Dataset 4			Dataset 5
	Software	AMP	FZA	P4D	AMP	FZA	P4D
Sparse Cloud	Images Aligned	50	50	50	50	50	50
	Matching time (hh:mm:ss)	00:01:05	00:10:14	0:00:51	00:01:04	00:09:23	00:00:49
	Alignment time (hh:mm:ss)	00:00:33	00:01:02	0:02:53	00:00:21	00:00:27	0:01:50
	Tie points (1000 points)	197	78	1262	102	52	547
	Projections (1000 points)	535	361	2697	258	247	1126
	Adjustment error (pixels)	0.98	1.44	0.17	0.69	0.94	0.11
	Resolution (mm/pixel)	0.08	0.09	0.08	0.16	0.16	0.16
Dense Cloud	Processing time (hh:mm:ss)	00:23:15	01:44:35	00:11:35	00:07:51	00:31:01	00:03:15
	Point count (1000)	43,611	2168	12,032	10,941	1811	3742
	Manual denoizing	no	no	no	no	no	no
Triangle Mesh	Processing time (hh:mm:ss)	00:36:40	00:00:27	00:07:20	00:03:44	00:00:21	00:00:44
	Faces (1000 triangles)	10,000	4245	10,000	9995	3587	10,000
	Vertices (1000 points)	7739	2935	7445	5507	2293	6773
Texture	Processing time (hh:mm:ss)	00:36:16	00:07:00	00:35:40	00:11:35	00:04:36	00:10:02
	Total time (hh:mm:ss)	01:37:49	02:03:18	0:58:19	00:24:35	00:45:48	00:16:40

). Processing times were comparable between each commercial software. P4D produced the densest sparse cloud results, and AMP the densest dense point cloud results. Notably, the camera auto-calibration parameters, extracted from the commercial software, were interchangeable upon changing their format.

Similarly, from the obliquely acquired dataset 6 (bust; EOS 1200D) the object was reconstructed with all photogrammetric solutions. However, ARP seemingly created a few double surfaces and misaligned different planes of the object’s surface. Processing times between free software and AMP were comparable while FZA required significantly more time to process the dataset in this case. The meshes created with the open source software had some holes at the lower and upper parts. R3D generated a very dense point cloud, but there was a high number of duplicate points, resulting in a low-resolution mesh (

Table 6. Photogrammetric results, dataset 6.

		Dataset 6
	Software	VCM	R3D	ARP	AMP	FZA
Sparse Cloud	Images aligned	50	48	50	50	50
	Matching time (hh:mm:ss)	00:02:19	00:03:36		00:00:36	00:00:59
	Alignment time (hh:mm:ss)	00:01:03	00:00:30		00:00:13	00:17:34
	Tie points (1000 points)	23	143		59	48
	Projections (1000 points)	75	498		156	205
	Adjustment error (pixels)	1.30	0.17		0.52	0.60
Dense Cloud	Processing time (hh:mm:ss)	00:11:39	00:23:01		00:05:37	00:22:22
	Point count (1000 points)	1582	11,786		9880	2666
Triangle Mesh	Processing time (hh:mm:ss)	00:06:05	00:01:02		00:06:31	00:02:01
	Faces (1000 triangles)	1451	252	1003	5000	3737
	Vertices (1000 points)	726	127	1848	2500	1873
Texture	Processing time (hh:mm:ss)	00:01:52	00:00:48		00:03:10	00:03:54
	Total time (hh:mm:ss)	0:22:58	0:28:57		0:16:07	0:46:50

). Texturing results were similar.

The object in dataset 7 (small sculpture, EOS 1200D, f = 18 mm) was fully digitized using AMP, while FZA produced sparser results because not all the images of the scene were oriented, despite the significantly longer processing time required. Each software produced dense results from dataset 8 (small sculpture, EOS 1200D, f = 55 mm) fully reconstructing the scene; AMP generated the densest 3D point cloud with the lowest reconstruction error, and FZA required the most time for processing. From dataset 9 (small sculpture, Exmor RS IMX650), we were able to retrieve the complete scene only by using VisualSFM. Both ARP and FZA generated partial models (Figure 3), while AMP produced a very noisy surface. Regard3D failed to generate any mesh from datasets 7–9. Results from the SfM–MVS photogrammetric processing of these datasets are listed in

Table 7. Photogrammetric results, datasets 7–9.

		Dataset 7		Dataset 8			Dataset 9
	Software	AMP	FZA	AMP	FZA	VCM	AMP	FZA
Sparse Cloud	Images aligned	142	69	60	60	60	60	23
	Matching time (hh:mm:ss)	00:01:05	00:03:40	00:01:55	0:01:39	00:01:22	00:01:38	0:02:58
	Alignment time (hh:mm:ss)	0:00:24	00:09:05	00:01:19	00:46:48	00:01:28	00:00:55	0:28:02
	Tie points (1000 points)	89	36	420	132	54	88	34
	Projections (1000 points)	273	154	1270	803	253	242	127
	Adjustment error (pixels)	0.52	0.52	0.35	0.47	1.02	1.15	1.43
	Resolution (mm/pixel)	0.09	0.09	0.02	0.02	0.02	0.02	0.2
Dense Cloud	Processing time (hh:mm:ss)	00:23:10	00:55:20	00:14:27	00:42:16	00:15:43	00:25:30	00:25:06
	Point count (1000 points)	2058	4211	9980	3958	1764	11,325	1720
Triangle Mesh	Processing time (hh:mm:ss)	00:01:23	00:02:30	00:02:58	00:03:34	00:06:03	00:05:32	00:01:13
	Faces (1000 triangles)	4846	4605	5000	4839	3864	5000	2061
	Vertices (1000 points)	2424	2312	2563	2500	1935	2504	1055
Texture	Processing time (hh:mm:ss)	00:09:25	00:18:25	00:03:59	00:09:50	00:09:47	00:04:09	00:02:51
	Total time (hh:mm:ss)	0:35:27	1:29:00	0:24:38	1:44:07	0:34:23	0:37:44	1:00:10

.

3. Scanning

3.1. Data Acquisition

For the objects’ scanning sessions, a FARO Focus3D X 330 was used along with two portable handheld near-infrared structure light scanners: the FARO Freestyle3D, and the STONEX F6 SR, recently evaluated for the digitization of heritage objects [44,45]. The characteristics of the scanning instrumentation are summarized in

Table 8. Specifications of the employed scanners.

Type	Phase-Based Laser Scanner	Handheld Structured Light Scanner	Handheld Structured Light Scanner
Brand	FARO	FARO	STONEX
model	Focus 3D X 330	Freestyle3D	F6 SR
Accuracy	2 mm	1.5 mm	0.09 mm
Point density	0.2 mm	0.2 mm	0.4 mm
Depth of field	0.6–130 m	0.3–0.8 m	0.25–0.5 m
Acquisition speed	up to 976,000 points/s	up to 88,000 points/s	up to 640,000 points/s
Noise level	0.3 mm	0.7 mm	0.5 mm
Approx. price	EUR 25,000	EUR 13,000	EUR 10,000

.

Scans were performed under homogeneous light conditions in circular patterns, planning to cover the complete geometry of the objects and eliminate occlusions as much as possible. The scanning distances were approximately 0.4–1 m, translating to 0.2–0.4 mm resolution point cloud densities according to the manufacturers’ specifications for all scanners. The case study of the capital replica required eight separate scans to be performed to fully capture the object’s 3D surface with the Focus3D X 330 and to ascertain the registration of all partial scans in one scene.

3.2. Processing

Scanned point cloud manipulation was performed with the software provided or suggested by manufacturers. Registration, denoising, and decimation were performed with Autodesk ReCap Pro 5.0.4.17 for raw scans from FARO scanners and with Mantis Vision Echo 2.3.1 for raw scans from the STONEX scanner. The 3D meshing was performed in MeshLab with similar parameters as used for the photogrammetric point clouds.

3.3. Results

For the figurine copy case study, no model was constructed, since the Focus 3D X 330 scanner did not provide results with enough density and the handheld scanners resulted in point clouds not correctly registered, with large amounts of noise which could not be removed either manually or automatically. For the capital replica case study, FARO Focus 3D X 330 and SF6 SR produced dense results, with some holes remaining in the first case due to occlusions. Freestyle 3D produced a very noisy and sparse cloud. These scanning results are listed in detail in

Table 9. Scanning results, capital replica.

	STONEX F6 SR	FARO Focus3D X 330	FARO Freestyle
Acquisition duration (mm:ss)	02:16	90:56	10:40
Registration duration (mm:ss)	05:08	14:35	---
Denoising duration (mm:ss)	24:15	2:26	00:02
Meshing duration (mm:ss)	01:23	04:01	01:27
Cloud points (1000 points)	20,928	1289	435
Mesh triangles (1000 triangles)	6350	6488	1951

. Scanning-based models were also produced for the stone bust after merging eight partial overlapping surface models captured with the F6 SR (14 million points), and for one side of the small sculpture (383 thousand points); all other digitization by scanning failed, because partially scanned scenes could not be registered.

4. Evaluation of the Results

The assessment of the quality of produced meshes considered the completeness, preservation of surface detail, noise, roughness, and additionally visual fidelity of texture for photogrammetric models. Models from the scanners had no observable noise. However, the surface produced with F6 SR was oversimplified, proving that the Mantis Echo Vision eliminated some of the surface details, despite low values for noise filtering had been selected; distances ranged below 2 mm (Figure 4).

For the photogrammetric datasets 1, 2, and 3, AMP and FZA produced very consistent results of similar detail and roughness. The models produced from dataset 1 (EOS 5DS R, figurine copy) were of remarkably high texture quality (Figure 5). The models from dataset 3 (Exmor RS IMS 650, figurine copy) contained an amount of noise (Figure 6). The calculated absolute distances between all models for those datasets were smaller than 0.5 mm (one standard deviation), which is roughly 0.3% of the objects’ size, with mean distances being lower than 0.3 mm (

Table 10. Hausdorff distances between the photogrammetric models for the figurine copy case study—datasets 1–3 (distances in mm).

	Dataset 2 AMP		Dataset 3 AMP		Dataset 1 FZA		Dataset 2 FZA		Dataset 3 FZA
Dataset 1 AMP	0.15	0.11	0.17	0.14	0.21	0.29	0.16	0.15	0.19	0.18
Dataset 2 AMP			0.19	0.16	0.23	0.28	0.21	0.18	0.18	0.14
Dataset 3 AMP					0.18	0.17	0.17	0.14	0.15	0.10
Dataset 1 FZA							0.20	0.20	0.17	0.16
Dataset 2 FZA									0.16	0.10
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

).

For dataset 4 (capital replica, EOS 5DS R), models from AMP and VCM had more holes. Models from P4D and ARP were smoother. The other models contained a small amount of noise on flatter surfaces and P4D seemed to oversimplify the surface details. Furthermore, the free software models had a small amount of remaining noise at the edges (Figure 7). All commercial software produced similar textures. The calculated absolute distances between the photogrammetric models for dataset 4 were smaller than 1.5 mm (one standard deviation), roughly 0.4 % of the objects’ dimensions, except for the model from P4D, for which the calculated distances to other models were larger than 2 mm (one standard deviation), as displayed in

Table 11. Hausdorff distances between the photogrammetric models for the capital replica case study—dataset 4 (distances in mm).

	FZA		P4D		ARP		VCM		R3D
AMP	0.66	0.45	0.75	1.30	0.76	0.59	0.69	0.54	0.80	0.57
FZA			0.80	1.50	0.72	0.78	0.72	0.71	0.79	0.68
P4D					0.95	2.14	0.94	1.06	0.96	1.07
ARP							0.80	0.67	0.82	0.64
VCM									0.60	0.59
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

. A mapping of the geometric differences is also presented with Figure 8. Regarding dataset 5 (capital replica, EOS 1200D), all models contained an amount of surface noise, ranging from low to medium levels, with the VCM pipeline being the one that resulted in a higher level of preservation of surface detail and lower roughness levels. P4D and R3D meshes were the noisiest (Figure 9). Overall, the texture quality between the commercial software was similar to but better than that produced by free reconstruction pipelines (Figure 10). The calculated absolute distances between photogrammetric models (vertices of final mesh) for dataset 5 were smaller than 2.5 mm (one standard deviation—σ), roughly 0.6% of the objects’ dimensions, except for the model from P4D, where higher values were observed (

Table 12. Hausdorff distances between the photogrammetric models for the capital replica case study—dataset 5 (distances in mm).

	FZA		P4D		ARP		VCM		R3D
AMP	0.60	0.45	0.68	0.81	0.72	0.99	0.50	0.50	5.45	3.33
FZA			0.94	1.06	1.04	1.23	0.73	0.68	5.37	3.23
P4D					1.07	1.51	0.90	0.95	5.48	3.22
ARP							1.05	1.51	5.55	3.23
VCM									5.37	3.26
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

).

Concerning dataset 6 (stone bust, EOS 1200D), FZA, ARP, and VCM-generated meshes were those more similar to the scanned mesh, and between them, in terms of surface detail and surface roughness (Figure 11 and Figure 12). However, as mentioned above, the ARP mesh had duplicate surfaces, and the FZA-produced mesh had a few holes on the top of the head where there was a smaller overlap between images. The AMP-produced mesh was also similar to the scanned one on the flatter surfaces, but on edges and fabric folds, a significant amount of noise remained. The textures were similar and differentiated only because of the small surface anomalies present at the web-based and open source software-produced meshes. Calculated surface differences are presented in Figure 13. Differences between the AMP-generated model and the other models were the smallest (

Table 13. Hausdorff distances between the photogrammetric models for the stone bust case study—dataset 6 (distances in mm).

	FZA		ARP		VCM		R3D
AMP	0.82	0.58	1.28	0.89	0.69	0.82	1.03	1.31
FZA			1.21	1.31	1.00	1.15	1.11	1.37
ARP					1.44	1.19	1.68	1.5
VCM							1.21	1.36
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

), while the differences between AMP–FZA models ranged below 1.4 mm (1 σ).

The only fully reconstructed mesh for dataset 7 from AMP had smoothed-out surface features. All reconstructed models from dataset 8 had similar characteristics (Figure 14). Some double surfaces could again be observed on the surface produced with ARP. Surprisingly, the generated surface with better preserved surface features was the one produced from dataset 9 (small sculpture, Exmor RS IMX650) with the pipeline implementing VisualSFM, CMVS, and MeshLab (Figure 15). The differences observed between the meshes generated from dataset 7 ranged below 1 mm (1 σ), while the differences between models from dataset 8 ranged below 0.7 mm (1 σ), which are both considerable taking into consideration that the resolutions of these datasets were 0.09 and 0.02 mm, respectively. Significantly the surface deviation between the dataset 9 VisualSFM model and the high-resolution Metashape model from dataset 8 was below 0.5 mm (1 σ). Some of the measured distances are presented in

Table 14. Hausdorff distances between photogrammetric models for the small sculpture case study—datasets 7–9 (distances in mm).

	AMP–Dataset 7		FZA–Dataset 7		AMP–Dataset 8		VCM–Dataset 9
AMP–Dataset 8	0.70	1.45	0.81	1.53	0.24	0.48	0.28	0.86
	mean abs.	std. dev.	mean abs.	std. dev.	mean abs.	std. dev.	mean abs.	std. dev.

.

5. Further Metric Comparisons

For the capital replica case study, more geometric assessments could be performed, comparing scanning meshes, assumed as ground truth, to the photogrammetric models. The calculation of distances was performed in Cloud Compare, with the cloud-to-cloud distance tool on the final meshes’ vertices, after fine registration with iterative closest point (ICP) was employed. The Hausdorff distances between the scanning and photogrammetric meshes ranged below 3 mm (one standard deviation) for dataset 4, except for the P4D model, and below 3.5 mm (one standard deviation) for dataset 5, except for the R3D model, which contained a great magnitude of noise. The results are presented in detail in

Table 15. Hausdorff distances between the 3D scanning and photogrammetric models from dataset 4 (distances in mm).

	AMP		FZA		P4D		ARP		VCM		R3D
3D X 330	0.84	2.18	1.01	1.57	1.32	2.14	1.17	1.69	1.21	1.79	1.01	1.04
F6 SR	1.72	1.75	1.23	1.40	1.23	1.43	1.48	1.61	1.21	1.39	1.17	1.35
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

and

Table 16. Hausdorff distances between 3D scanning and photogrammetric models from dataset 5 (distances in mm).

	AMP		FZA		P4D		ARP		VCM		R3D
3D X 330	1.46	1.96	1.29	1.90	1.37	2.16	1.42	2.04	1.25	2.09	5.75	3.30
F6 SR	1.53	2.16	1.54	1.41	1.22	1.43	1.51	1.69	1.31	1.44	6.22	3.33
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

. The main differences between the models produced with photogrammetric and scanning techniques were observed at parts of the capital replica that were occluded due to its complicated geometry.

For the stone bust, further geometric assessments could also be performed, comparing photogrammetrically produced surfaces with the scanned model. Accuracies considering as ground truth the scanned F6 SR model are presented in

Table 17. Hausdorff distances between the 3D scanning and photogrammetric models from dataset 6 (distances in mm).

	AMP		FZA		ARP		VCM		R3D
F6 SR	0.63	0.73	0.66	0.65	1.22	1.00	0.75	0.92	1.11	1.38
	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev	Mean abs.	Std. dev

and visualized with Figure 16. The AMP and FZA models were more metrically accurate, considering that the surfaces deviated below 1.3 mm (1 σ) from the ground truth model, representing only 3 ‰ of the object’s smallest dimension. Mean distances and their standard deviation for the models produced from non-commercial software ranged at about 1 mm.

6. Discussion and Conclusions

The presented research carried out a selective comparison on state-of-the-art SfM and MVS image-based modeling solutions of different costs, and on portable scanners for the digitization of small heritage objects. As expected, challenges occur from the different nature of heritage objects, with geometry, surface features, and texture playing important roles in the decision making for acquisition and processing workflows.

Photogrammetric results are affected by the type of camera sensor, sampling distance and coverage of the object on the image space. For complex, featureless, or very small case studies, only the expensive commercial solutions appeared to be able to fully reconstruct the photogrammetric scene, proving that perhaps the more cost-effective solutions are better suited for static scenes, or when the background is homogeneous and of a vastly different color than the object, so that it can be recognized as background by the utilized software.

For the photogrammetric reconstruction from the datasets of the capital replica, and the stone bust, almost all workflows produced similar results. Some noise problems that occurred can be attributed to oblique imagery, and thus can be tackled by acquiring more rigid, dense, and light-consistent imagery datasets.

Although, AMP, FZA and ARP proved to be the more efficient solutions, it should be stated that ARP offers no adjustable parameters and a limitation of 100 images per project, which is an important problem for real-case heritage digitization applications. Furthermore, AMP offers only a few adjustable parameters, with no details being available on the algorithmic approaches it exploits. On the other hand, FZA allows every parameter in the course of the digital reconstruction workflow to be customizable and despite default options resulting in noisy results—relative to FZA—an expert can identify how to optimize its implementation for heritage purposes. The implementation of an edge-preserving meshing algorithm, that also limits the interpolation of the dense point cloud in 3DFlow Zephyr, especially allows the generation of high-quality surfaces, preserving detail similar to the handheld scanners. P4D, as a software mainly oriented towards smaller scale applications and flatter geometries, did not provide sufficient results. The main problematic with the free solutions was the surface noise (due to capturing conditions) which cannot be easily filtered post-production.

Occlusions caused by complex geometries were tackled by image-based methods, but other problematic surfaces may require various combinations of documentation techniques. The textures produced from scanning techniques were not of adequate quality and thus the meshes produced in this manner need to be textured with other methods, ranging from simple image-to-mesh registration, to co-registration with photogrammetric models, to the integration of sensors for multiple data acquisition. Differences between Focus3D X 330 and Stonex F6 SR results can be attributed to the fact that the first one is oriented mainly towards architectural documentation and other construction applications.

The use of a mobile phone camera for photogrammetric purposes also seemed promising, not affecting the metric properties of the results, but had a visible effect on the levels of generated noise. However, despite the high resolution and quality of the mobile camera sensor used, the texture results were of lesser quality than the textures produced from datasets of high resolution DSLR camera. Thus, more experiments need to be conducted in that direction to evaluate the radiometric capacity of smart-phone cameras for the high-fidelity texturing of heritage models. To conclude, the combination of smartphone cameras and web-based solutions provides an exciting potential, for applications where metric quality is not the primary concern, such as rapid recording, dissemination for education, or the promotion of cultural heritage for touristic purposes.