Advancements in 3D Heritage Data Aggregation and Enrichment in Europe: Implications for Designing the Jena Experimental Repository for the DFG 3D Viewer

Abstract

Since the 2010s, various national and pan-European public infrastructures have been emerging around aggregation, viewing, and 3D heritage model collection. The purpose of this article is to focus on the current state and ecosystem for 3D models in Europe through (a) a review of published studies on users, objects, and demands (b) and an overview of the ecosystem for 3D heritage data. As part of the German distributed infrastructure, the DFG 3D Viewer Jena experimental repository serves as a testbed for technology prototyping and testing. Based on the findings of the European ecosystem, we used this repository to test a prototypic approach to (c) acquiring 3D data from multiple sources, (d) enriching data quality, and (e) enabling indexing, searching, and viewing functionalities.

1. Introduction

Since the 1980s, digital 3D models of cultural heritage have been used in a variety of disciplines such as archaeology, architecture and art history, and museology as a tool for research and knowledge transfer. Such models are created primarily through either the retro-digitization of objects that still exist or the interpretative reconstruction of historical objects that no longer exist, based on multiple sources [1]. Although the working methods, data structures, and results of both approaches differ fundamentally [2], there are overarching requirements with regard to visualizing the 3D data contained and ensuring access to metadata and paradata sets.

Within the European Union (EU), there has been a long-standing interest in 3D heritage. Related funding programs previously focused on technological aspects such as cost efficiency, user-friendly tools for creating digital 3D models [3] and their prototypic application, as well as the setup of infrastructures [4]. Overviews of projects and infrastructures in cultural heritage are provided by [5,6,7]. With regards to research infrastructures and their target groups, CLARIAH provides virtual research environments, especially for text-related research. ARIADNE and CARARE are dedicated to supporting archaeological information management [8,9]. The Europeana virtual library [10] is a repository for digital cultural heritage assets and collecting and aggregating institutions such as museums, libraries, and archives [11]. Complementing this, the E-RIHS infrastructure initiative focuses on tools and services for heritage sciences [12] (an extended overview is provided in [2]).

Recently, 3D modeling of cultural heritage gained significant attention after the digital transformation of the cultural heritage sector after the COVID-19 pandemic [13] as well as the strengthening of the digital market through the Digital Europe Data Space [14,15]. This is all linked to a shift toward large-scale digitization and modeling of 3D content. Consequently, the European Commission proposed a campaign in late 2021 to digitize all monuments and sites at risk as well as 50 percent of the most visited monuments, buildings, and sites in 3D by 2030 [16]. This would target 2.4 million 3D assets by 2025 and 16 million assets by 2030. Currently, in 2023, most accessible models are still stored in private sector repositories, such as Sketchfab, while public repositories contain comparatively few 3D model data to date (see

Table 1. The number of 3D models stored in repositories (examples).

Repository	Number of 3D Models Contained
Sketchfab (objects tagged “cultural heritage”)	100,000+ (10/2019) [17]
Europeana	5931 (5/2023) [18]
Kompakkt	261 (4/2023) [19]
DFG 3D Viewer	3922 (4/2023) [20]

).

The purpose of this article is twofold. One aim is to analyze the current state and ecosystem for 3D models, particularly in Europe. This comprises (a) a review of the published studies on users, objects, and demands (b) an overview of an ecosystem for 3D heritage data. Both studies were conducted to derive implications for the design of the national infrastructure for the DFG 3D Viewer. As part of this distributed infrastructure project, the DFG 3D Viewer Jena experimental repository serves as a testbed for technology prototyping and testing. To overcome some of the challenges examined through the studies, we tested some prototypic approaches to (c) acquire 3D data from multiple sources, (d) enrich data quality, and (e) enable indexing, search, and viewing functionalities.

2. The Current State of 3D Repositories Quantified

Various studies have been conducted to date on 3D heritage data (see

Table 2. Surveys related to 3D heritage modeling.

Year	Study	Scope	Participant No.
2013	Conference article review (2000–2013) [22]	Worldwide	478 published articles
2016	FSU Jena author survey [21]	Worldwide	988 participants
2017	ViMM survey [22]	Worldwide	782 responses
2016	INCEPTION survey [26]	EU	53 representatives
2018	CS3DP [27]	US	53 respondents
2019	Europeana 3D Survey [25]	EU	38 individuals
2020	VIGIE Study [23]	Worldwide	420 respondents
2021	Pure3D [29]	NL	48 responses
2022	UK 3D Data Service Survey [28]	UK	Unknown

). The literature-based reviews and surveys by FSU Jena [21,22], by the Virtual Multimodal Museum CSA [22], and the VIGIE study on the 3D digitization of tangible heritage examinedusage scenarios and defined quality criteria [23]. Specific studies on 3D infrastructures include the Dutch Pure3D (2021: 48 valid responses) [24], the Europeana 3D (2019: 38 individuals) [25], the EU INCEPTION project survey [26], and the survey by the US CS3DP group [27] (2018: 53 respondents). Most recently, the UK 3D Data Service Survey [28] was conducted in 2022.

In addition, several surveys on available platforms, repositories, and frameworks were compiled recently [30,31,32,33,34] to provide an overview of particular technologies such as laser scanning [35], photogrammetry [36], machine learning [37], and extended reality technologies [38]. Moreover, several studies analyze data and digitization within specific domains with occasional links to 3D data. These include the ARIADNE+ user survey [39] for archaeology, the ICOM surveys [40,41,42], and the survey by Samaroudi et al. [43] analyzing museums during the COVID-19 pandemic.

Community and 3D objects. The large-scale surveys by FSU Jena [21], ViMM [22], and EU VIGIE [23] provided a comprehensive analysis of a community, as well as objects and technologies for 3D modeling. Most humanities and cultural science actors come specifically from archaeology, art history, architectural history, and historical sciences (see Figure 1). Additionally, there is the involvement of actors from academia, heritage institutions, and commercial companies.

Objects are primarily monuments or architectural objects, followed by sculptural art objects (Figure 2).

With regard to modeling technologies in current surveys, specifically photogrammetry or scans of existing objects are used (see Figure 3). Hand-modeled 3D reconstructions are rarely explicitly mentioned, but according to our previous studies [46], these account for approx. 20% of the models.

Formats and requirements for 3D viewer repositories were collected through multiple surveys on a national level and also by the Europeana survey on a pan-national level. In general, it can be observed that a wide variety of data formats are used, with OBJ being particularly relevant, but also glTF and COLLADA (see Figure 4). Models are predominantly viewed on laptops, while tablets and smartphones are significant, albeit to a lesser extent [24].

Specific requirements include measurement and editing tools, as well as the capability to show/hide parts of the model [24] (see Figure 5). With regard to metadata, Dublin Core as a metadata reference and CIDOC CRM [47] as a top-level ontology for cultural heritage data are both widely used, with specific implementations for domains and purposes. Despite a large plurality of formats, requirements, and formalized processes [44,48], a significant challenge that remains in working with 3D data is insufficient or missing documentation of both metadata as object descriptions and paradata describing the creational process.

3. A 3D Data Ecosystem in Germany and across Europe

Beyond the meta-analysis of studies on 3D infrastructures, a second part of this article is dedicated to mapping an ecosystem for 3D for cultural heritage at the European and national levels, the latter with a specific focus on a German ecosystem. The research used a qualitative methodology adopted from policy analysis including categorizing policy documents, defining fields of activity, and comparing different spatial references (c.f. [49]). Several studies analyzed EU projects on digitization for cultural heritage (recently [4,50,51]). In addition, the Heritage Research Hub [52] maintains an overview of previous and current projects in the field. Other studies have examined specific facets, such as EU-level policies [53] or the state of digitization in European cultural heritage institutions [54]. A recent analysis of national cultural heritage hubs was conducted by the 4CH project, which mapped 51 national and sectoral hubs [55].

With regard to funding, previous programs have primarily focused on technological developments. However, there has been a paradigm shift in funding politics and policies since 2010. Since then, in addition to the still extant priorities on the forming of technical infrastructures as digital research environments or repositories, other topics became increasingly important objects of funding—examples are human resource development, transnational knowledge exchange and cooperation, the enhancement of social and economic impacts, and valorization and dissemination of digital 3D objects [56]. An evaluation of the EU research funding programs until 2014 stated that the aim to “foster the dissemination, transfer and take-up of program results” was only limited and covered by these programs [3]. In response to that, the Horizon Europe program aims for “an understanding of Europe’s intellectual basis” and the usage of “new technologies […] as they enable new and richer interpretations of our common European culture while contributing to sustainable economic growth” [57] (p. 5). The program also aims to strengthen the development of infrastructures to foster the research, education, and publication of “knowledge-based resources such as collections [or] archives” [58] (p. 4), see also [59,60]. Therefore, it is crucial to closely interlink a viewer infrastructure for 3D models with corresponding teaching, community, standardization, and innovation measures (see

Table 3. Overview of an ecosystem on 3D data, 2023.

	International (e.g., EU Level)	Multinational	Germany
Standards for 3D content description	There are various standardization initiatives around 3D heritage data. CIDOC CRM is a generic metadata scheme for heritage objects, with relevant work by ARIADNE+ [61] and 4CH [62] to adopt and extend for digital 3D heritage documentation. Concerning open metadata standards CARARE, 3D ICONS, the Europeana 3D content task force and the ICOM Group have conducted significant work. The IIIF 3D community group [63] coordinates and facilitates open standards for viewing and 3D annotations. Quality measures for 3D content were proposed; for example, by the VIGIE Study [23]. Regarding 3D data formats, the Khronos Group [64] or Web3D, [65], for example, define overarching formats. See [24] for a recent overview.	Various groups, such as the Europeana 3D Task Force [25] and the AG Digitale Rekonstruktion des DHd e.V. [66], examine and monitor the current state of standards.	The DFG IDOVIR project [67] develops a VRE for paradata recording. A metadata scheme alignment initiative of major German infrastructures has been started in 2022 to enable the mapping of their schemes.
Data Infrastructures	The Digital Europe Data Space [15] for Cultural Heritage, maintained by Europeana, provides data repositories and aggregation; for instance, for heritage data. The European Cloud for Cultural Heritage (ECCCH) [68] will develop a toolset for cultural institutions. The European Open Science Cloud (EOSC) [69] provides a set of core services to store and share research data. Various VREs deal with 3D data; for example, E-RIHS [70] for heritage science and ARIADNE+ for archaeological data.	Several national 3D infrastructure consortia are already formed; for example, in Sweden, [71] UK [72], Ireland [73] or France [74], or the Netherlands [75].	The NFDI4Culture [76] provides various tools, such as the FOSS infrastructure [77], for annotating 3D models with Wikidata entries. Baureka.online [78] provides a portal to store and share research data, particularly for historical architectural research. The FID BAUdigital [79] provides information services for Civil Engineering, Architecture, and Urbanism. The DFG 3D Viewer is a multi-source repository.
Education	The KIC CCIS AP 1 and 2 are dedicated to aligning curricula for graduate and postgraduate education in Europe.DARIAH Teach [80] and the TMO Academies [81] provide OER material for learning 3D-related skills.	The Computer-based Visualization of Architectural Cultural Heritage (CoVHer) [82] ERASMUS+ project strives to define applicative/practical guidelines and operational methodologies for 3D models of artefacts that no longer exist or have never been built.	The Virtuelle Akademie zur digitalen 3D-Rekonstruktion [83] and the Digital4Humanities project [84] has developed collections of video tutorials to improve 3D skills.The DFG network for architectural 3D reconstruction has developed a handbook for scholars [44].
Community	Time Machine [85], the Europeana Network Association [18], and the ICOMOS/ISPRS CIPA [86] are domain-independent large networks of heritage professionals. Networks such as CAA [87] for archaeology or ICARUS [88] for archival studies are domain-specific platforms. The 4CH Competence Centre [89] is developing a concept for a Europe-wide support structure for 3D data [90].	The EU Interregional Partnership for Virtual and Smart Cultural Tourism [91] is a multi-regional community and project hub around digital heritage.	The DHd e.V. [92] is the national association for Digital Humanities in German-speaking countries.
Viewer	Sketchfab [93] is the world’s largest 3D data repository and provides an integrated and embeddable viewer. Google’s ScanTheWorld initiative [94] offers 16,000 objects in its object collection.	Various viewers such as the Smithsonian3D, 3DHOP, ATON [95], Ark/k [96], Clara.io, CFIR.science, MorphoSource [97], Stanford 3D, Exhibit, Virtual Interiors, DarkLab, GB3D, CyArk [98], NASA 3D [99], Kompakkt, and Potree (overviews: [23,24,100]) are available and used in multiple projects, primarily at the national level, e.g., [101].
Open Innovation	Gaia-X [102] provides a digital service platform with digital heritage as one of the use cases.The EIT Culture & Creativity [103] is proposed to provide a large-scale framework for cultural innovation.The C4Education Innovation Lab [104] is developing a B2B application platform at the European level.	Various applications utilize 3D heritage data. Overviews: the Virtual Multimodal Museum CSA mapped museum applications until 2017 [105] on augmented reality applications [106] and heritage sites [107].

). Three-dimensional data infrastructures are also developing in a highly dynamic environment, especially in Europe, where numerous national, domain-specific, and international initiatives compete with each other.

4. Implications for the Design of 3D Data Infrastructures

A multitude of implications has been already collected and compiled for the design of 3D data infrastructures [68]. One main differentiation lies in the purpose of 3D data infrastructure, such as data repositories to store and preserve data, viewer infrastructures for publishing and viewing content, and feature-rich virtual research environments (VREs) that provide tool collections and working environments with many infrastructures serving multiple roles. The investigations identified several current challenges, including:

• Public repositories contain comparatively few 3D model data to date: Although numerous infrastructures for 3D models on a national level are currently in formation, the expansion of the publicly hosted model pool still represents a significant challenge. Another issue for preservation is finding repositories capable of accepting the exceptionally large datasets resulting from very high-quality digitization of large objects. Many models are still not publicly accessible due to being stored in local data repositories [2]. Although Sketchfab is not a preservation repository but a private viewer platform, it still contains the majority of publicly accessible models. Due to the characteristics of platform businesses toward winner-take-all dynamics [108], there is a tendency for one platform to dominate the market. This could be addressed by concurring platforms reaching a significant size; for instance, by content aggregation or serving specific purposes, such as privacy and long-term preservation. There are also market gaps visible as opportunities to store (very) large high-quality 3D datasets are missing. A consequent implication is to serve a high-demanded profile, such as long-term preservation, and offer many models, which can be achieved by incorporating reusable and publicly available 3D content.
• Enhancing findability and reusability: Findability and accessibility increasingly become challenging as the number of 3D models increase, as evidenced by our previous analysis on image repositories [109]. Stable identification (ID) systems are a major prerequisite to citing and accessing model data. Indexing and findability of 3D data primarily rely on metadata. Despite extensive research [48] and numerous methods/tools [110,111,112,113,114,115,116], for example, reverse engineering tools to compare models with sources [117,118], documentation tools and methods to record steps and decisions taken during the 3D modeling process [114,119,120] metadata still need to be assigned by the creators in manual processes. With regards to the relevant schema, CIDOC CRM became fixed as an ISO standard for heritage documentation [47]. Nevertheless, the widespread adoption of CIDOC CRM into systems remains of limited outreach, and its implementation into application ontologies is of heterogeneous quality. Regarding metadata creation, currently most metadata descriptions are set manually by users. Numerous initiatives are targeting the development of domain-specific thesauruses to formalize the tagging by metadata; for example, art and architectural history content. Despite the unification by ontologies, manual tagging is limited; for instance, with regard to necessary workload, as well as a limited suitability for massive amounts of data or retro-tagging. Therefore, ensuring and/or automating the process of sufficient metadata generation and verification becomes an important issue.
• Serving heterogeneous user communities: Three-dimensional heritage objects are used in various disciplinary contexts such as art and architectural history studies, museology, archaeology, and heritage conservation for a wide range of purposes in research, education, and heritage management [44]. Several sectoral standards such as IFC for building information modeling [121], GML for geo- and city-scale models [122], and the Digital Twin [123] as a domain overarching paradigm are relevant. In addition to content generated by professionals, there is a substantial amount of 3D heritage content created by enthusiasts. The creation of user-generated 3D content is facilitated by the development and availability of ready-to-use photogrammetric software tools [2] supporting the crowdsourced collecting and processing of images as a prerequisite for 3D photogrammetry [124] and the crowd-based creation of 3D models [125,126]. One of the key success factors of repositories is their ease of use [109]; for instance, due to their slick and user-friendly user interfaces and workflows. Consequently, repositories are required to address a large variety of users and incentivize them to provide content by keeping requirements low, nurturing the content provision, and providing rewards.
• Feature and quality requirements: The visual qualities of 3D web-based viewers have been discussed in various articles [30], with a notable trend to increase (photo) realism. However, these required visual qualities seem highly dependent on use cases. Regarding tools and functionalities, many application frameworks provided by public institutions are VREs that provide a complex working environment for particular communities, such as archaeology [127] or architectural history [128], but require much more experience. For general-purpose 3D viewers, low-level requirements, such as model viewing and viewport navigation, are considered extremely important [28,45]. Higher-level requirements include measurement and editing tools and the ability to show/hide parts of the model [24]. Consequently, viewer tools should be kept simple and focused on the most relevant features.
• Monitoring and fostering standards: Currently, many initiatives are emerging around 3D data, leading to the development of a multitude of viewer frameworks. To address the aforementioned challenges and anticipate changes in the future, a modular adaptability of technological frameworks is required, which might include upcoming viewer technologies, as well as the monitoring of these initiatives on the national and international levels.

5. The DFG 3D Viewer Project

The analysis presented in the previous sections guides the DFG 3D Viewer as a German-based infrastructure development project. This initiative was started in 2014 by the German Working Group for 3D Reconstruction [110]. It led to the DFG 3D Viewer project, a collaboration between FSU Jena, HS Mainz, and SLUB Dresden, which has been underway since 2021. The project aims to expand the current media viewer infrastructure operated by the German Research Foundation and the German Digital Library to add the capability of processing 3D data. The project aims to provide permanent and sustainable access to 3D datasets and associated metadata and enable web-based model viewing.

5.1. User Requirements

With regard to specific use cases, a cross-disciplinary workshop held in 2014 involving 40 German researchers engaged in the digital reconstruction of cultural heritage, who examined the significant demand for zoomable and rotatable 3D model viewing [110]. Within the German ecosystem, the following initiatives are operational:

• Preservation of digital assets is primarily the responsibility of the state libraries, which formed a specific entity, and the German Digital Library as a nationwide data aggregator and service provider [129].
• Research tools and virtual research environments are being developed within a multitude of other initiatives. A notable example in Germany is the National Data Infrastructure with three initiatives (NFDI4Culture, NFDI4Memory, and NFDI4Objects) involving 3D data of cultural heritage [130].
• Making 3D models available for viewing online has been examined as a primary objective by the aforementioned surveys. In response, the German working group for 3D reconstructions (AG Digitale Rekonstruktion des DHd e.V.)—a group of 80 individuals from 40 academic institutions in the German-speaking area [131]—conceptualized and initiated the DFG 3D Viewer project as a national initiative to enable public open access.

5.2. The System Architecture

To be compatible with the multitude of extant and future infrastructures, the DFG 3D Viewer is designed as a modular system that enables the integration of various repositories, viewers, and services. This includes 3D data conversion, reconstruction, documentation, and metadata enrichment, and also the ability to link to other 3D model infrastructures in Germany, such as the Kompakkt viewer [132] or Virtual Research Infrastructures, such as FOSS [77] or the 4D Browser [133]. A range of services incorporated into the DFG 3D Viewer include export options for the conversion and parsing of metadata in container formats (including METS/MODS format) and converters for 3D data formats (e.g., OBJ/MTL, FBX, IFC, PLY, and XYZ) into the glTF format. The glTF format was chosen since it is widely supported by WebGL-based viewers and has numerous relevant features for browser-based viewing, such as single-file containers or progressive loading [134]. Previous articles have provided explanations of the overall project [100], as well as specific components, such as the metadata scheme [135] and usage scenarios [116].

5.3. The Jena 3D Experimental Repository

As of early 2023, the DFG 3D Viewer consortium has been operating two repositories. A semi-production system is operated by HS Mainz and is based on WissKI as a virtual research environment and linked open data management software [136]. This environment is used as a live system to store and deliver a unique model pool. As a second repository, we launched the Jena experimental repository for the DFG 3D Viewer in 2022 to test data aggregation, processing, and enrichment workflows in an experimental and fully controllable environment. It is based on a LAMP stack running in a dockerized VM. The processing pipeline includes various PHP and Python scripts. The overall processing scheme of the Jena experimental repository comprises components for “Data retrieval”, “Data processing”, “Metadata creation”, “Data enrichment”, and “Data visualization” (see Figure 6) and will be the subject of discussion in the next sections.

6. Data Retrieval

6.1. Retrieval Pipeline

To overcome the issue of limited content availability, we have implemented server-based Python scripts to retrieve 3D models from Sketchfab, Europeana, the HS Mainz 3D Repository, and the UrbanHistory4D repository through API calls by name or location. To ensure legal compliance, we selected Creative Commons CC0 or CC-BY [137] licensed content only.

The retrieval service operates as a series of server-side scripts written in Python and PHP, feeding into a SQL database and a Unix file storage system. The scripts gain input from (1) the GeoNames “Major cities of the world” [138] dataset, (2) a keyword-based retrieval from Europeana, (3) JSON-based database retrieval for Mainz and UrbanHistory4D, and (4) locations called by users in our mobile 4D city app, which are resolved into place names using GeoNames [139]. Scripts 1 and 2 are operated on a one-time basis, and 3 and 4 work as cron jobs. As of early 2023, a total of 3922 3D models have been retrieved from these sources (see

Table 4. Number of retrieved models (3/2023).

Source	Number
Sketchfab	2.736
Europeana	906
Mainz 3D	64
UrbanHistory4D	214

).

6.2. User-Generated Content: The 3D Reconstruction Service

As an alternative method for gathering 3D content, we implemented a low-end 3D digitization pipeline for documenting heritage using images captured with a smartphone. The objective is to document cultural heritage by using images and 3D models from user-generated photos and to integrate them into our repository. The 3D reconstruction service includes a webpage frontend providing a guided workflow capturing and uploading images to the portal servers, together with metadata (see Figure 7).

Metadata can be freely added or retrieved from sources such as Wikipedia for object descriptions, ORCID [140] for user information, and GeoNames for location information. Once uploaded, a server-side process is initialized, utilizing a scripted Meshroom [141] pipeline to create 3D models from these images. Currently, this tool is used with an unsupervised pipeline, automatically uploading the models to Sketchfab and then retrieving them into the DFG 3D Viewer repository to reduce error sources or data conversion issues. As a fallback option in the event of automated reconstruction failure, a manual workflow is used, allowing for parameter modifications.

7. Data Processing

After the retrieval process, a series of scripts is used to rename files to ensure unique filenames and modify linked URLs, thereby modifying the glTF XML description files.

8. Metadata Creation

In the following step, METS [142] and Europeana EML XML files are created by modifying a template. The database-stored metadata is injected into the template, resulting in a new description file for each item. To access this data, a PHP-based script creates a JSON listfile.

9. Data Enrichment

An important step to make models findable is the utilization of authority data [143]. This type of data plays a crucial role in the prevention of data silos and linking enclosed projects across different media [144]. Classifying 3D content information is a significant challenge. Currently, simple structures [145] and even complex objects, such as buildings and architecture [146,147], can be automatically segmented [148,149]. Inferences can also be made as to which parts of the image reference which parts of the 3D object geometry [150,151]. As the technological backbone for image segmentation, technologies for object recognition [152,153], and data classification [154,155,156,157,158] are playing increasingly important roles.

9.1. Image-Based Content Retrieval

One step in the DFG 3D Viewer project involves identifying the content of the 3D model. To accomplish this, we render the model into a series of images and employ a content-based image retrieval (CBIR) script. This Python script operates on the server side and uses the Google Cloud Vision API to retrieve corresponding imagery. This enables us to retrieve information, such as a related Wikipedia article.

9.2. Automated Content Classification

Standards enable subject-specific classification of subject matter and make an essential contribution to the unambiguous indexing of cultural heritage. A controlled vocabulary is an organized arrangement of words and/or phrases, which is used to index and/or retrieve content. “It typically includes preferred and variant terms, and has a defined scope or describes a specific domain” [159] (p. 12). Similarly, a thesaurus combines features of synonym ring lists and taxonomies. “A thesaurus is a semantic network of unique concepts, including relationships between synonyms, broader and narrower (parent/child) contexts, and other related concepts. Thesauri may be monolingual or multilingual” [159] (p. 24). Examples are the controlled Getty Vocabularies, such as the Art and Architecture Thesaurus (AAT), the Getty Thesaurus of Geographic Names (TGN), the Cultural Objects Name Authority (CONA), and the Union List of Artist Names (ULAN), as well as Iconclass [160] for iconographic indexing.

In our case, the classification process utilizes a Python server-side script that iterates through the rendered images. These images are classified using the Google Cloud Vision classifier with the three most probable categories per image retrieved. In a second step, related AAT concepts are retrieved for each of those categories.

To assess the quality of automated classification, keywords for 70 randomly selected 3D models were independently assigned manually. The classification rule was based on describing object facets of the model with reference to the preview image, with five keywords per model. The classification was all carried out by one person. The manually assigned keywords were compared to the five best matches of the automated keyword assignment. Matches had to correspond on word-stem (e.g., machinery vs. machine) or top-/sub-level (e.g., art vs. cartoon). As shown in Figure 8, co-occurrences most likely appear for one or two of the classifications (median = 1, mean = 1.25).

In an overall assessment, automated classification retrieved a larger variety of classes and contained more outliers (e.g., “automotive tire” for a theatre light beamer). Conversely, manual classification retrieved more complex concepts, such as names (e.g., “Paris” as a city name) or methods (e.g., “scan” vs. “reconstruction”).

10. Data Visualization

Various standards for 2D user interfaces have been established [161,162,163]. As a paradigm for interface design, we combined 2D maps (e.g., Google Maps [164] or OpenStreetMap [165]), historical images (e.g., Historypin [166]), and a keyword-based search across all content fields (Figure 9). Interaction with these elements triggers content filtering. The interface is written in plain HTML with JavaScript codes for interactions. We used OpenLayers [167] by adding map-based interaction functionalities.

The detail page of each model refers to a Drupal-based instance of the DFG Viewer hosted by SLUB Dresden. The 3D view shown in Figure 10 is steered by a URL parameter and can either refer to Model Viewer [168], or a three.js-based viewer with additional functions, such as cross-sections and model counts.

11. Next Steps

The current repository setup enables a complete workflow for retrieving, processing, and viewing data, as well as making it available for aggregation services provided by Europeana. Although these components have been proven to work even at some scale, they demand a quality assessment by, for instance, comparisons with manual classifications.

Considering that the Google Vision service is general-purpose, the use of classifiers designed specifically for cultural heritage may be of interest, as several alternative approaches are available to serve for classification and CBIR tasks [169,170,171]. Benchmarking with other approaches is, therefore, another next step to take.

Additionally, our group focuses on the automated generation of 3D heritage building models from historical imagery [172]. These approaches are coupled with the visualization of 4D-scaled building models at the world scale. The next step is to link both location-based visualizations and the 3D model repository to visualizing location-based 3D content in the world viewer and vice versa, enriching the model pool by automatically generated building models.

12. Summary

After campaigns to digitize and valorize 2D heritage data as images and texts, the digitization, data aggregation, and utilization of 3D heritage data at a large scale became a major focus in Europe in the 2020s. This comprises setting up national 3D data infrastructures, a Europe-wide program to equip Europeana to become the main 3D data aggregator, and an initiative to digitize 30 million heritage objects from the whole of Europe by 2030. However, in 2023, a significant majority of 3D models of cultural heritage are currently stored by commercial companies in the US, and a European 3D ecosystem is fragmented and still in formation. Consequently, current demands are the expansion of model pools to attract content providers and users, the improvement of usability and findability, and the capability to deal with future developments.

Against this background, an idea we are testing in our Jena 3D repository as an experimental branch of the German DFG 3D Viewer is the large-scale retrieval, enrichment, and visualization of 3D data from multiple sources. By aggregating extant model collections and utilizing ready-to-use services, we were able to create a public cultural heritage 3D model collection with several thousands of models and provide enriched metadata information via various CBIR tools. Based on this proof of concept, the next task is to assess whether and how these workflows could help develop more stable and production-oriented infrastructures and correspond to identified use cases of relevance, such as the reuse of 3D models.