A Built Heritage Information System Based on Point Cloud Data: HIS-PC

Abstract

The digital management of an archaeological site requires to store, organise, access and represent all the information that is collected on the field. Heritage building information modelling, archaeological or heritage information systems now tend to propose a common framework where all the materials are managed from a central database and visualised through a 3D representation. In this research, we offer the development of a built heritage information system prototype based on a high-resolution 3D point cloud data set. The particularity of the approach is to consider a user-centred development methodology while avoiding meshing/down-sampling operations. The proposed system is initiated by a close collaboration between multi-modal users (managers, visitors, curators) and a development team (designers, developers, architects). The developed heritage information system permits the management of spatial and temporal information, including a wide range of semantics using relational along with NoSQL databases. The semantics used to describe the artifacts are subject to conceptual modelling. Finally, the system proposes a bi-directional communication with a 3D interface able to stream massive point clouds, which is a big step forward to provide a comprehensive site representation for stakeholders while minimising modelling costs.

1. Introduction

Heritage and archaeological site management is a challenging task facing many requirements in terms of legal issues, human resources, financial aspects [1] and information management [2]. Nowadays, there exist more and more heritage information systems (HIS) that deal with some of the issues mentioned. However, a unique integrated system cannot fully encompass the concerns raised by heritage site managers. International organisations such as UNESCO pledge for integrated management to deal with aspects such as their historical, social, environmental and human components [3]. The primary objective is preserving the values associated with heritage sites over time [4,5,6]. The development of HIS is essential to achieve better site management, conservation, information dissemination and public mediation, but it requires some significant progress. Moreover, as stated by Poux et al. in [7], different worlds with different perspectives (e.g., geoinformation vs. cultural heritage vs. computer graphics) diminish the flexibility of a system if no balanced approach is mapped to the development of the solution. This is why it is important to design a solid software architecture that is directly intended to integrate diverse viewpoints [8,9].

The focus on information modelling is often viewed as the first development step to propose a comprehensive heritage management. Indeed, the problems of structuring, archiving, conserving and exploiting heritage data are challenging tasks which must be tackled with a compelling long-term vision for a sustainable solution. Acquiring spatial information on heritage sites has become a straightforward task thanks to the development of lasergrammetry, photogrammetry or a combination of both techniques [10,11]. The digital replication of a site can be achieved with high geometric accuracy while keeping a consistent radiometric fidelity. However, the 3D spatial data gathered from the existing site is often part of a massive data set whose use is limited because they are sectorised/degraded or simplified.

In this research, we propose an infrastructure that overcomes, among others, the level of details limitation and gives new solutions for user’s semantics integration. In collaboration with the Coudenberg Royal Palace Museum (Brussels, Belgium), we propose a knowledge-driven built heritage information system based on point cloud data (HIS-PC). The proposed HIS-PC is intended to be an open and evolutive system that links any point cloud to a spatiotemporal and semantic information system dedicated to heritage objects. At this stage of the project, we consider the built heritage component in our developments. The proposed structure may be adapted in the future to non-tangible aspects. The developed system follows a user-centred design and includes generic conceptual data models that encompass the requirements of a flexible HIS. Those generics models are then specialised to fulfil the final specifications of the Coudenberg HIS use case. The approach focuses on the adaptation strategy to evolutions, similarly to [7], concerning the core blocks of data acquisition, visualisation and interaction.

One of the first challenges in a user-centred approach is the definition of the needs gathered from different perspectives and technical background. Often, one is going to store a large amount of data in a repository that will be the main, if not the only, source of information for many applications, also known as a data lake. Data lakes contain a wide variety of data that are accessed by numerous applications. For these applications, the data lake is the only source of truth, and this is the reason why a data lake should never be modified or deleted beyond the factual errors that could be spotted. Information present in a data lake that would be invalidated by historical research must, in our opinion, remain present as the trace of multiple interpretations. Our reflection follows the data lake approach managing 3D data, i.e., to use the most fundamental source of data in a reality capture context: 3D point clouds. Few solutions exist for managing semantics and geometry directly on the point cloud, demanding new data models to capture logical aspects of the data structure. The large data sets that point clouds constitute cannot fit in the main memory entirely, requiring adapted systems that can exploit the information regarding a storage model efficiently. In this paper, we highlight the relevance of a new solution for updating and enriching digital models in the context of dynamic management of geometric, temporal and semantic information related to heritage sites. Our research objectives are linked to the use case where the conceived three-dimensional HIS-PC aims at:

-

Supporting heritage research allowing, for example, the development of hypotheses on the history of the built environment;

-

Serving as an interface to access massive multi-modal information of diverse nature (spatial, temporal, topology, semantics);

-

Producing tools and documents for mediation purposes;

-

Recording conservation monitoring data.

The paper is structured as follows. In Section 2, we review related works linked to pertinent Heritage Building Information Model (H-BIM) or HIS. We focus on platforms that combine semantic and geometric information for heritage information management. Section 3 poses the research context, and Section 4 develops the methodology allowing us to obtain the HIS-PC architecture. Notably, we describe the generic data models and information structuration while explicitly looking at the conception process to the realisation of a conceptual data model. Generic models that support the working infrastructure to manage point cloud data are presented, and link temporal, attribute and spatial information in an interactive HIS. Then, in Section 5, we look at the results in terms of data acquisition, system implementation and user feedback. Finally, we sketch some future perspectives that need to be addressed quickly in Section 6, and we conclude.

2. Related Works

We discuss here several similar works carried out in the field of environments allowing access to heritage data in connection with semantic documentation. We will end this section with a comparison of the selected cases to position the innovative aspect of our contribution.

2.1. Heritage Documentation

The digital heritage documentation grew significantly since the last decade, thanks to the rise of renewed technologies of information acquisition [12]. The heritage documentation encompasses a wide range of activities, including the recording and documentation of reality through geometric and semantic data collection. Several acquisition procedures can be set up to collect the geometric information. Nowadays, terrestrial laser scanning (TLS) and aerial or terrestrial photogrammetry are the two primary techniques used in a digital documentation context [13], or a combination of both [11]. Many examples of heritage sites documentation can be retrieved by which those two techniques are used to collect geometrical information related to the objects [14,15]. For the majority of the application, a four-step process is used. Heritage information is collected over archaeological sites or heritage buildings. Based on the gathered information, a 3D reconstruction of the current state of the artifact is proposed. Then, a semantic enrichment takes place to add contextual information to the model and make it usable in a heritage context. Finally, a representation is presented to communicate through a medium such as a web environment, a 3D simulation software or, more basically, in producing 2D documents such as maps and plans.

Whatever the recording technique that is used on an archaeological or heritage site, it is worth mentioning that the acquisition allows only to gather information related to the heritage object at the time of the analysis. All other information related to the past state of the site has to be reconstructed based on historical knowledge. As mention by Santana, heritage information should provide relevant and timely data to proceed and evaluate the role and action of each of the work phases [16]. The inclusion of all the required information dimension (temporality, semantic, management of heterogeneity, the complexity, the diversity of actors) leads to a significant challenge of information integration [17,18]. The heritage research community has quickly understood the need for the development of information structuration. Data models that can manage heterogenic and complex heritage information are thus sought after.

2.2. Heritage Knowledge Representation—The BIM Process

Following the rise of information complexity, the evolution of heritage information representation has changed profoundly in regards of the growth of the computer-aided design (CAD) environment, geographic information system (GIS), and the advent of the building information modelling systems (BIM). All these information systems carry out a set of information modelling rules and data structuration. Depending on the application perspective, the GIS environment is a tool of great help to organise heritage information into a geographical database. The benefit of such an environment for heritage information management is the access to the querying and reasoning tools provided by the GIS environment allowing to extract new information from data sets. At the same time, numerous cultural heritage information managers have considered BIM as a management process for heritage information [17,19,20,21]. Both approaches do not have to be regarded as concurrent. As proposed by Dore, the 3D GIS and BIM worlds are compatible with heritage information management [22].

Building information modelling refers to a collaborative process among actors of a construction project [23]. The environment helps for the information exchange through an object-based representation. The principle of the BIM process has been extended to the heritage (or historic) building information modelling process when dealing with historical information and representing a heritage element. The collaboration of actors around one or several historical representations of a cultural heritage site is based on the BIM principles (information structuration, non-redundancy, …). The ability to integrate TSL or photogrammetric information via parametric modelling or a manual modelling process also represents an asset of the use of BIM in the cultural heritage field. The use of BIM in heritage science is widely discussed by many authors [24,25,26,27,28]. The interested reader can refer to Pocatello et al. for an extensive review of the BIM process applied to the heritage information [24]. Many authors notice that the built heritage is characterised by complex morphology and non-homogeneous features. The BIM modelling process is more willing to model parametric features that are repeatable. The heritage objects do not comply with the standardised approach of the BIM automatic modelling and require a high human interaction to propose a model close to reality.

2.3. Heritage BIM and Its Limitation

Since the last couple of years, the research community has focused on the ability to interpret surveys as parametric shapes or features to fit the BIM standard modelling approach. Most of the proposed works focused on point cloud data as they now represent the mainstream of captured information on the heritage site, whatever information is collected via TLS or photogrammetric processes. The so-called Scan-to-BIM aims at improving the feature recognition and reduce the treatment time to interpret, structure and create BIM objects from a point cloud [29,30,31,32]. It has to be said that for most of the cultural heritage application, those approaches are not feasible due to the time that is required to generate a complete, accurate and controlled model of heritage sites. Moreover, the interpretation of heritage objects is performed in a certain context that depends on the experience of the operator performing the recognition (or who control the automated feature recognition). As mentioned before, the management of a cultural heritage site requires the collaboration of many specialists. Each of the specialties has to interpret the reality regarding the requirement of their field. It means, for example, that a built archaeologist and an architect will not segment a building in the same way. Therefore, the information structuration is greatly complexified to manage all viewpoints [33]. Some projects tried to face this problem [34,35,36,37,38], the most efficient application rely on data structure managing temporal information [39] or not that are not directly related to BIM information structure but on ontologies [40], relational databases [41,42] or NoSQL data structures [43].

2.4. Heritage Information System

The integration of heritage survey in the classic BIM environment is a complicated task. It is not yet fully operational for time-limited projects in which the modelling time and the interpretation rely on several specialities. This assessment has led to consider HIS in which the semantic information is directly related to the low-level geometric information, i.e., systems in which the semantic information is directly associated with a subset of the acquired point clouds. This approach requires either a segmented point cloud to relate information to a subset of points, or a geo-referenced system in which the semantic information is related to a position in the 3D environment. One recent approach proposes a data structure that directly integrates semantics within the point clouds [44]. Several existing projects offer a HIS in which an information system is related to a three-dimensional representation of a heritage site or building. Among those approaches, we wish to point out some application that is relevant for our proposal: the BIMLegacy project [45], the PHYT project [46] and the web information system for the management, the dissemination of cultural heritage data [47] and the 3DHOP approach [48].

The BIMLegacy project is an online platform aiming at unifying and synchronising heritage architecture information. The contributions of the platform propose a work platform where interdisciplinary stakeholders can synchronise heritage information. The platform has been developed under the design science research methodological approach. This method of development relies on a prototype definition which is enriched and corrected in several iterations. The result is an online heritage platform defined as a common data environment (CDE) that allows synchronising information in real-time. The system architecture is presented in Figure 1. The architecture presented shows that access to the data is done through a platform supported by the Revit application. An API allows access to the data via the Revit Plugin. Figure 1 clearly shows the dependence on the Revit environment and the need to adapt to the data formats of the software.

The approach relies on a BIMLegacy approach which means that the acquire point clouds have to be modelled through a Scan-to-BIM approach. The targeted level of development in the Revit environment is LOD 400. This approach has the benefit of handling light-weighted models and therefore provide a smooth and reliable 3D representation online.

The PHYT project is a vast project that merges numerous partners ranging from parietal art experts, archaeologists to 3D information professionals [46]. In the framework of the project, a 3D environment has been developed as a MeshLab environment plugin. The main objective of the project is to store and structure information from heterogeneous sources of every specialist studying caves. The semantic information is located by a spatial position that serves as an identifier to retrieve information in the database. The 3D cave environment is represented as a 3D mesh. Then, a bi-directional communication is established between the 3D model and the database to perform operations (e.g., selection from the 3D model or visualisation of queries in the 3D environment).

The HIS proposed in [47] (Web for CH) was one of the first to include the dimension of multi-users and multi-application within a shared computing environment. Moreover, once implemented, the information is accessible via the web and therefore, accessible to an entire community through a virtual research environment dedicated to the exploitation of intra-site data. The architecture of the project remains simple. The information is accessed through a web server queried via PHP language and information stored on a MySQL database. The significant input of this research is the proposed data model that relates together the place with the document that describes the archaeological object. This information model is efficient to manage heterogeneous sources from the archaeological environment (see Figure 2). The semantic data model shows a classical approach linking archaeological objects with various documentary sources. The model is built around a generic class “Data”. This class is specialised in the model’s constituent elements, namely the management of location, time, documentation and archaeological objects (tangible elements). This modelling choice, which could be questioned, gathers all the information in a single class. Explicit time management is proposed in the model. However, it is limited to a periodic representation which can lead to incoherence when representing some heritage sources. The system does not allow several interpretations of the same sources as proposed by [49]. Finally, at that time, the primary way to implement such information model was the relational database environment. Despite the usefulness of this approach, the heritage information is too heterogeneous for a replication of the implementation. The NoSQL approaches are now recommended for such applications [50].

The 3D Online Presenter (3DHOP) project is an open-source software package for the creation of interactive web presentation of high-resolution 3D models. The environment is oriented to the cultural heritage field. The strength of this system resides in the variety of stakeholders that it is addressed to. The system is not intended to be a working platform but only a visualisation tool of already implemented semantic information. The visualisation is based on a multi-resolution 3D mesh models that are streamed to the end user. The proposed environment is open-source and therefore easily linkable to a personalised environment such as a dedicated database storing the semantic information.

The analysis of existing standard data environment dedicated to cultural heritage information shows that there are very few working environments based on a point cloud spatial representation (see

Table 1. Summary of cultural heritage management platforms approaches.

Name	Geometric Approach	Semantic Approach	Temporal Approach	ReadOnly/Edition Platform
BIMLegacy	Revit LOD400	Revit	Revit	RO/Edition
PHYT	Mesh	SQL	None	RO/Edition
Web for CH	VRML/GIS	SQL-DataModel	Period	RO/Edition
3DHop	Mesh	To be connected	Managed from a semantic approach	RO

). We believe that providing the rawest source to the stakeholders is the most efficient solution since each interpretation of heritage objects or artifacts is depending on past experience and a context of interpretation. Most of the analysed systems prefer to stream parametric or meshed 3D models because of their relative lightness to be streamed on the web. The data models proposed for the semantic enrichment of 3D models is most of the time closely related to the BIM modelling process. However, most of the authors agree on the frequent incompatibility between reproductive parametric BIM environments with information collected on cultural heritage sites where almost every object is unique.

For all these reasons and as a result of our experiences in the field, we pledge that heritage information should be managed through a heritage (or archaeological) information system by:

-

managing semantic, temporal and spatial component of heritage information;

-

enabling complex spatiotemporal querying and visualisation; and

-

supporting point cloud data, eventually as a core spatial data structure.

Such a system must:

-

be open and adaptative, considering the fast-growing technologies in the field of data acquisition, data management, data visualisation, etc.;

-

meet the user’s requirements and therefore be adaptable to needs of specific organisations; and

-

be as interoperable as possible to allow the sharing of data and system evolution.

3. Context

Following the proposed methodology in the preceding section, we applied a heritage tailored user-centred approach to a real study case, the Coudenberg Museum. Located in Brussels, Belgium, the Coudenberg Museum manages the ruins of the palace which dominated the city of Brussels for centuries (Figure 3).

Charles V and other rulers among the most powerful in Europe lived in this princely residence from the twelfth to the eighteenth century, until a fire destroyed it. The ruins of this palace disappeared for many years in the underground. After excavation work brought them to light, they now present an archaeological site consisting of a network of rooms and underground passages (see Figure 4).

The museum, which manages the entire site, wished to acquire an archaeological information system since 2017. The museum came up quickly with two key deliverables: first, a precise 3D model of the site and secondly, a computer system that allows archaeological information to be stored and located in the 3D model. Based on the museum requirements, we implemented the HIS based-on point cloud (HIS-PC) approach in this precise context. Following the co-construction procedure presented upward, the generic data model is specialised to fit every delicate need and the physical constraints of the Coudenberg environment.

4. HIS-PC Architecture Design

The development of a HIS that answers the identified needs in Section 2 is filled with interesting design choices. It is based on a user-centred design described in Section 4.1, that delivers a robust conceptual data model presented in Section 4.2, on which we base the information system architecture described in Section 4.3. The final HIS-PC is thus interesting for its architecture relatively simple that permit semantic, temporal and geometric queries based on a core point cloud. In the context of this research, we consider the “semantic” as any attribute information related to model objects that are not spatial, temporal and identity aspects [42].

Our proposal is compatible with principles coming either from quality control processes (inspired by ISO 9126) or human-centred design (inspired by ISO9241-210) as presented in Figure 5 and further explained in the following Section 4.1.

4.1. HIS-PC User-Centred Design

There exist various methodologies to design an Information System (IS), mainly proposed by the computer science community. To handle the complex nature—uncertainties, missing data, lack of formalism, and multi-modal roles—of heritage information, we propose to use and adapt an existing IS design methodology based on a user-centred approach. Several authors used a similar approach in the field of heritage information management [51,52,53]. In this sub-section, we present the main guidelines of the method. We describe the four design steps and some of the most crucial outputs to facilitate the development and implementation of a HIS. It is interesting to note that such a complexity of information can be found in several other areas—heritage data, however, has the particularity of requiring multiple interpretations [41,49].

The design process is iterative and follows the human-centred design for interactive systems ISO 9241-210:2019. For an information system to be pertinent, it must meet the requirements of the organisation or the targeted users. The methodological steps illustrated in Figure 6 are the following:

• The analysis of users’ requirements (step 1) which ended up with a list of needs and formalised specifications, both for the IS and data acquisition.
• The conceptual design (step 2) where conceptual data models, system’s interfaces and data acquisition procedures are set up based on the specifications of the first step.
• The design and prototyping (step 3) which consists of an IS prototype development, data acquisition and data integration in the IS prototype.
• The usability evaluation (step 4) which is performed by the designers and the users. Depending on the result of the evaluation, either a new prototyping loop starts or the prototype reaches a further step of implementation and maintenance.

The quality criteria that the system must meet can be taken from existing standards. We propose to draw inspiration from the ISO 9126 standard established for software development. Whenever possible, we encourage the use of database management systems (SQL and/or NoSQL) that are adapted to the heterogeneous data sets that characterises cultural heritage applications. Without excluding spatial representations of vector or mesh types, we encourage the use of point clouds as a geometric core data modality. Connections between the different databases must be ensured, and all the system functionalities required by the users must be accessible through ergonomic interfaces that facilitate the handling and use of the system.

4.2. Data Model

The data model was developed during the conceptual design step (see Figure 6—2.A). During this phase, several iterative processes between the users and developers end up with some conceptual deliverables: a conceptual data model (see Figure 6—2.A), a mock-up of the information system architecture and interfaces (see Figure 6—2.B) and a data acquisition methodology (see Figure 6—2.C).

Heritage information is complex. It is by nature, incomplete and uncertain, usually associated with one or multiple spatial and temporal representations [54]. When creating a conceptual data model for archaeological information, two antagonists’ aspects should be handled: specificity and interoperability. Indeed, the model must fit with the specific needs of the users while the developed system is rooted in a particular organisation. However, the model should also be open to other models and standards: it must be evolutive and mappable to other models or ontologies. Several initiatives exist to offer a conceptual modelling environment for uninitiated individuals lacking knowledge in information modelling (e.g., [55]). We have chosen to start from a very generic metamodel that defines key concepts. Therefore, the methodology imposes to start the iterative process with a metamodel gathering core concepts (classes and relations) helping users to gradually understand the conceptual modelling mechanisms. These core concepts are mainly inspired amongst existing modelling approaches ([39,56]), although differences might occur in a defined vision of the representation of time, uncertainty or multiple interpretations. In a future evolution, we plan to integrate, through a mapping, our model to existing initiatives. A commonly accepted version of the metamodel is then adopted and serve as a framework for the design of the conceptual model of the organisation’s HIS-PC.

The core concepts are four classes (object, state, event and document), three components (spatial, temporal and functional) and the relation between classes (see Figure 7). Each of these four classes can be in relation with another class or with itself and might have spatial, temporal or functional components, which is in line with the well-known Peuquet’s triad [57]. An object is a class referring to the core elements targeted through the modelling process (e.g., a wall). A state is a stable combination in time of several characteristics related to an object (e.g., the state of a wall composed of stones during a given period and at a given elevation and shape). A state does not exist without a clear relation to one object (e.g., a set of stones and mortars combined during a certain period at a particular location is considered as corresponding as a state of a given wall). An event leads to a change of properties which triggers the creation of a new state (e.g., the destruction of part of a wall during a fire leads to the apparition of a new state of the wall with a different size). A document is a source of information which can be linked to an event or a state (e.g., a written report mentions the occurrence of a fire at a given place at a given moment in time). Depending on the applications, an element could be either associated to an object or a document (e.g., a written report can be a document in an archaeological site context, but it could be an object in a library description context).

The adopted version of the metamodel for the Coudenberg project is presented in Figure 8. In particular, only the event class has a temporal component, and only the state class has a spatial component (with a minor exception for one object—“Site”; see Figure 9).

The creation of a conceptual data model based on the metamodel is a collaborative and iterative step, gathering the users and the developers. The proposed modelling formalism is UML [58], but other formalisms could be used. We pointed out ConML, which is a generic formalism dedicated to digital humanities application [55]; it has the benefit to be specialised for cultural heritage applications in its CHAMR (Cultural Heritage Abstract Reference Model) extension [59]. In addition to the metamodel, some modelling strategies can be adopted but are not mandatory. For instance, due to our experience in archaeological modelling, we suggest the following guidelines:

• A distinction between existence and presence. This vision of an object lifecycle allows to sequence all the step of an object definition from its conception, to each transformation and finally to its final destruction. We assume that each transformation, considered as an event, is accessed through documentation that describes the change.
• The temporal states of an object are cumulative. The complete historical sequence is the combination for every state of an object. Consequently, an interpretative sequence is a sub-selection of some states in the complete object history.
• A heritage object is a part of another heritage object. This mereological approach of the heritage components gives considerable latitude to the stakeholders and the final system users to define their vision of the heritage site segmentation. It ensures that the proposed division is closely related to the user’s applications and correspond effectively to their needs.

From this initial design, we derive a more detailed and formal UML class diagram specialised for the Coudenberg HIS, presented in Figure 9. It is worth to mention that this model is specific to the Coudenberg project and does not claim to be the right candidate for other heritage information developments. It corresponds to the best conceptual data model considering the current needs of the users and their validation. In order not to overload the diagram with non-essential elements for general understanding, only important classes, attributes and names of non-explicit associations are represented. Moreover, this model is likely to evolve in order to adapt to users’ new needs (as described in Section 5.2, the implemented solution for data storage must be able to manage evolutive data models).

This data model shows a concrete application of the metamodel. Therefore, the four meta-classes “Event”, “State”, “Document” (“Documentation”) and “Object” (“Element”) are represented. In addition, other classes (“Site”, “Architectural element”, etc.) answer to specific needs of Coundenberg Museums. They can inherit from metaclasses (“Architectural element” is an element/object) or not (“Site” is independent of any metaclass) regarding final users’ requirements. A site is spatially defined as a 2D geo-referenced polygon. A site includes elements which can be any object of archaeological interest (a building, a room, a wall, a stone, a plastron, etc.). In a first approach, we define two classes of elements (more can be added in the future): containers and architectural elements. Architectural elements are physical objects (depending on types and specific materials) while containers are spaces with specific functions like rooms. Containers can be associated to one or several excavations. “Part-of” is a recursive association of “Element” allowing a hierarchical structuration of objects semantic. For example, a wall is part of a room which is part of a palace. Because of spatial changes of elements during their lifetime, geo-referenced geometries are attached to the class “state”.

The spatial definition of “state” can then be either a 3D point (only located), a 3D polygon (surface) or a 3D bounding box (volume). All geometries follow ISO and OGC standards and are defined in a spatial coordinates reference system like WGS84 (EPSG 4326). As far as an inaccurate temporal approach has to be managed, the states are temporally delimited by events themselves characterised by estimated start date and end date. Finally, each event, data modelling, documentation is related to a source managed through a document. The documents are heterogeneous information about states and events (observation collected by archaeologists working on the site, historical images, plans, schemas, etc.). This information is represented by the class “documentation”. Once this conceptual model is set up, we approach the development of the HIS architecture mock-up, described in the following Section 4.3.

4.3. Information System Architecture Mock-Up

The information system architecture mock-up was developed during the conceptual design step (see Figure 6—2.B). Figure 10 represents the three-tier architecture of the HIS. It is composed of the three following layers:

• A client layer regrouping interfaces which are the end users’ entry points to the HIS
• An application layer which allows the communication between the interfaces (client layers) and the data (data layer).
• A data layer regrouping all the data sources (databases or data files), structuring the data lake.

Interfaces of the client layer are tools physically located on the user’s machine. Most of the time, it is a simple web browser interacting with static HTML and dynamic JavaScript codes sent by a web server (application layer). It also can be a business-oriented software (GIS software or point cloud viewer, for instance) able to communicate with the application layer through web protocols or exported files. Business-oriented interfaces (proprietary or open-source) generally cover particular parts of the users’ needs but implies minimal programming efforts from programmers. They thus offer quick solutions to these needs and can be complementary to other interfaces (e.g., web browsers).

In the HIS architecture, we propose a distinction between point cloud interfaces, dedicated to spatial objects (objects defined in space), and semantic interfaces, dedicated to other information (objects defined by time, materials, documentation, etc.). Since everything can be grouped in a single interface, this distinction is not mandatory. However, point cloud and spatial information can quickly be entrusted to business-oriented software based on popular spatial data formats and standards.

No matter the distinction between semantic and point cloud interfaces, these two aspects are merged in the application layer: objects (created, searched or visualised by users) are not only semantic or spatial anymore but both semantic and spatial. It leads to the definition of geographic objects which can be queried and represented based on these two aspects (e.g., “A 3D map showing rooms superior to 25m² and coloured in the function of their last building date”).

Once again, a distinction is proposed in the data layer to easily take advantages of different database management systems (DBMS). Indeed, some tools can be efficient for spatial data (spatial database) while others can be more adapted to heterogeneous information (semantic database) or huge point clouds (point cloud data). These three data components can be separated or possibly grouped into one DBMS if such a tool meets all the users’ requirements. No matter which DBMS is used, it is up to the application layer to query the different DBMS and eventually merge results to rebuild geographic objects.

Finally, end users can perform several operations, as shown in Figure 11, which are managed through the different layers of the HIS architecture. These operations are mainly linked to the semantic and spatial interfaces, being:

• Semantic interface:

  ○

  Creation/modification of semantic information for objects (e.g., creation of an object “wall” with its building date and material)

  ○

  Representation of all the semantic information of objects (e.g., what is the material of this wall and when was it built?)

  ○

  Search of objects based on semantic aspect and/or spatial predicates (e.g., which are the walls made of granite, built after the 13th century and higher than 2 m?)

• Spatial interface

  ○

  Point cloud visualisation

  ○

  Creation/modification of an object’s spatial definition (e.g., a located vector volume around a wall based on point cloud data)

  ○

  Spatial visualisation of objects (e.g., point cloud part intersected by a vector volume defining a wall)

All operations mentioned above and illustrated lead to specific queries distributed in the different databases, and whose eventual results are processed by the application layer before their visualisation in the appropriate interface. Note that point clouds are considered as a persistent data source and support for spatial objects definition and visualisation. With a dedicated DBMS, points clouds could also be involved in queries more directly.

5. Results

The realisation of the system was tested against a complex environment with heterogeneous data distribution. In this section, we propose to display specificities linked to the 3D Data Acquisition (Section 5.1). In the Section 5.2, we show the translation of the HIS-PC architecture into a usable software product, later related to the concise feedback gathered from the different involved partners (Section 5.3).

5.1. 3D Data Acquisition

The acquisition methodology is decomposed in three main aspects:

• The implementation of durable reference points, accurately determined in an adapted geo-referenced system;
• A full photogrammetric and 3D scan survey of parts of the archaeological site linked to the reference points following the work of Poux et al. [60];
• An element-based full photogrammetric reconstruction of independent elements in a local frame of reference such as in [61].

The following shows the detailed acquisition process that allows quality control of the produced data set.

5.2. Reference Point Materialisation

To create a framework of reference points on the area to be surveyed, we have set up 11 stations (see Figure 12). The choice of the location of these stations was made according to the layout of the site and to be able to survey the whole of the requested site in 3D. These stations were marked on the ground either with chalk, temporary glue or surveyor’s nails (outside).

Once the points were established, they were measured with a Leica TCPR 1205 total station using a closed-loop pathway to ensure geometric cohesion. Several points were set up and measured by GNSS on the Place des Palais to determine the coordinates of the stations in the Lambert conical conform 2008 coordinate system. From the principal polygon, seven additional points were implemented and measured at the total station to allow the linkage to Lambert 2008. The geo-referencing accuracy is 1.1 cm. We chose to use this projection because it is the Belgian reference for all geographic data. In this way, it will be possible to easily integrate cadastral data, background plans, or any other georeferenced element into the system.

5.3. 3D Laserscanning Survey

The 3D scanner used is a LEICA P30. The registration was made using common targets between scans whose centre coordinates have been determined by surveying from the previously calculated stations with the above-mentioned total station. The resolution of the scans made it possible to obtain a 3D measurement every 3 mm at 10 m (i.e., every 0.3 mm at 1 m). These incremental measurements are performed at 360° on a horizontal plane, and 270° on a vertical plane to obtain a very dense point cloud, as shown in Figure 13.

Also, several photos (approx. 300 per station) were taken with the internal scanner camera, each with a resolution of 920 × 920 pixels to form a very high-resolution panorama. It is then matched to the depth data to obtain a colourised point cloud (Figure 14).

A total of 110 scans were required to survey the entire Aula Magna and its exterior walls in 3D. The distribution of the scans performed can be seen in the following point cloud section (Figure 15).

The 110 scans were then assembled using the common targets present in these different scans. The result is a point cloud geo-referenced in Lambert 2008 coordinates with a mean absolute assembly error of 1 mm over the entire point cloud (Figure 16).

Photogrammetry Augmentation

Approximately 12,000 shots were taken with a CANON EOS 70D reflex camera equipped with a 15 mm fisheye lens and a flash. The resolution of the images is 5472 × 3648 pixels. These pictures were used to refine the geometry of the scan survey and to allow a better colour balance. After cleaning, the position with the point cloud geometry of the different images used is estimated using the software RealityCapture (Figure 17).

This process was carried out by decomposing the global information in zones that closely correspond to rooms subspaces, to ensure that the laser scanning data stays the backbone of the entire process. Optionally, a mesh is derived from the very dense point cloud, without constraining its complexity. It results in a model of several billion spatially oriented triangles (Figure 18—left).

A homogenised texture is obtained (Figure 18—right) using the oriented images as input, which is then draped over the mesh. The colourimetric deviation is, on average, 0.6 pixels. The final point cloud is composed of 24.315 billion points.

5.4. Implemented Information System

The IS is implemented during the Design and implementation step (see Figure 6—3.A) and can be divided into five components: architecture (Section 5.4.1), backend (Section 5.4.2), frontend (Section 5.4.3), point cloud interface (Section 5.4.4) and interactions between components (Section 5.4.5). We draw inspiration from software engineering principles detailed in [62,63,64,65], more particularly by adopting the waterfall approach detailed in [63].

5.4.1. Implemented Architecture

The information system is conceived such that databases are centralised and accessible from any client terminal. Therefore, the system is designed as a web three-tier architecture. It implies a web browser interface (frontend) connected to databases through the REST API (backend). This first part (Figure 19—left) is designed specifically for the project. The point cloud management (Figure 19—right) was delegated to the software Geoverse. While this can be achieved using the smart point cloud infrastructure proposed in [66], the choice was dictated by the strong deployment constraints on the servers of the Royal Palace, and the user-friendly needed interface. Figure 19 describes in more details the different elements of this architecture as well as their interactions.

5.4.2. Backend

The backend mainly consists of a scalable NoSQL database: a document store managed by open source DBMS MongoDB. It offers the advantage of remaining flexible insofar as the information is stored in the form of JSON documents. It has versatile data schemas, making it usable for a variety of applications. Moreover, the design of the database allows the scalability of the system. The backend is accessible through an Application Programming Interface (API) composed of REST services that will enable implementing all the functionalities necessary for the development of the interface. Thanks to the NoSQL database, the backend system underlying the interface fits the needs of archaeologists whose data are very heterogeneous and likely to evolve.

Nevertheless, the management of spatial information proposed by MongoDB, still lacking maturity for this aspect, did not fit to the project’s requirements: storage of 3D geometries, strong spatial indexation, topological queries, surfaces/volumes computations, coordinate systems transformations, trips simulations, intervisibility analysis, etc. For these reasons, a second DBMS was explicitly used for spatial information: PostgreSQL and its spatial extension PostGIS (both open source). Indeed, coupling NoSQL with relational GIS gather advantages from both sides: respectively flexible semantic model and effective spatial management [67]. In addition to a complete set of spatial functions, it follows OGC standards for the storage of 2D/3D geometries and raster data. Although the Postgres extension PgPointCloud stores point clouds directly in the database, it is not directly compatible with our point cloud interface (Geoverse). In consequence, point clouds are stored as files in a remote server or directly in the user’s machine.

5.4.3. Frontend

The semantic interface was realised in close collaboration with the museum. It can be reached from any web browser and allows users to manage the documentary of archaeological elements (items) as well as the chronology of events and states related to these items (see Figure 20). It allows to introduce links between items and to manage groups of customisable items (Ceramic, Stone, Glass, Bone, etc.) and themes to which the items are related (Conservation, Museography, etc.). Also, the items can be linked to functions (Cellar, corridor, etc.) or systems (Zone 1, Zone 2) to group and search for item sets by spatial or functional group. For the creation of geographic objects, the interfaces allow three main types of operations:

-

From the semantic interface to create new objects (sites, elements, states or events);

-

From the semantic interface to add or modify semantic attributes (numbers, text, dates but also external files like plans, pictures, etc.);

-

From the spatial interface to create locations and shapes (geometries) through a digitalisation procedure based on the point cloud.

5.4.4. Point Cloud Interface

The 3D data collected and processed form a point cloud of several billion 3D points. We use Geoverse Massive Data Manager (MDD) to exploit and archive the data set in a fluid and straightforward way. This software significantly reduces the costs of storing, displaying and interacting with complex data sets generated by 3D scanners and photogrammetry. Thanks to Unlimited Detail technology, Geoverse MDM offers the ability to interact with geospatial data by allowing, to visualise and access data in less than a second on an entry-level computer (Figure 21).

The proprietary choice covers the majority of the initial requests of the technical specifications in a very efficient way, thus was chosen as a strong candidate for managing the 3D data efficiently. It is supplemented by tools developed in open-source that provide access to advanced functionalities. The set-up thus makes it possible to obtain a complete solution covering the needs described in the methodology and which has the advantage of being flexible and evolutive.

As shown in Figure 22, the information encoded in the semantic interface can be attached to specific locations through the point cloud interface. Thus, users can visualise archaeological objects in the point cloud and easily switch from an interface to the other (and vice versa). These interactions between the point cloud and the databases of the HIS are shown in the following sub-section.

5.4.5. Two Ways Interactions between HIS and Point Cloud

The key feature of the proposed environment is the bi-directional relationship between the archaeological information and the point cloud geometries. The selection of the more relevant component for the management of semantic and geometric information led to the use of two concurrent environments. Following the requirement of an efficient user-interface, we developed an automated conversion API which guarantees a smooth transition between the geometries and the archaeological information. The locations and shapes, stored as 3D geometries in PostGIS, can be joined to the semantic NoSQL database through a common identifier to rebuild geographic objects. The transfer of these objects between the databases and Geoverse is possible through a dedicated web API (Figure 23), allowing two types of operations:

• Geoverse2PostGIS: digitised geometries in Geoverse (including ID, 3D geometries and spatial metadata like coordinates reference system) are sent to PostGIS for their storage.
• PostGIS2Geoverse: based on a user’s query, the API returns 3D geographic objects with semantic attributes readable by Geoverse. 3D geometries come from PostGIS, and semantic attributes come from the NoSQL semantic database (based on common ID between the two databases).

The communication between databases and the point cloud interface is based on an XML file describing a Geoverse project: local directory (on the client machine) or remote location of the point cloud, coordinates reference system and data about the 3D objects like geometries, symbolisation, point of view and attributes appearing as labels (for instance). Amongst these attributes, an URL allows users to switch from the point cloud interface to the semantic web interface for a specific object by simply clicking on it.

5.5. Usability Evaluation Report

The usability assessment stage of the Coudenberg project is summarised in

Table 2. Quality assessment of the HIS-PC prototype.

Item	Action	Comment
Functionality	To improve	More functionalities are needed due to the feedback of the users.
Reliability	Satisfying	Some bugs were present in the developed HIS-PC, and we are correcting them progressively
Usability	To improve	We need to merge the 3D point cloud system and web application.
Efficiency	Satisfying	The HIS-PC requires low hardware resources.
Maintainability	To improve	Due to the proprietary aspect of Geoverse, the maintainability of this component is not under control, thus needs migration. The web interface is fully maintainable.
Portability	To improve	The web application may be deployed under different server environment, but Geoverse needs a PC to be operated and used.

, according to the proposed methodology (Figure 24—D5). The software package was put into production, and users were trained in its use. We offered tailored user manuals for these training as well as videos.

For six months, users were able to use the solution developed during training sessions. In general, the learning curve has been relatively fast, which means that the interfaces are sufficiently adapted to be used by a group of users with different profiles. During this period, intensive use allowed users to suggest several interface improvements and new features. Thanks to the software architecture described above, evolutions of the software are possible without the need for extensive re-engineering. The user tests were carried out thanks to the encoding and exploitation of real data. As the use of the system progressed, new functionalities were proposed by the Coudenberg museum. They are mainly related to the visualisation of temporal aspects. A set of requirement records all of the targeted improvements that serve as a guide for future developments.

Concerning the use of the 3D point cloud, it was found that the “model” was entirely usable. However, it was requested that the colour of the point clouds was delighted to allow for virtual lightning simulations. It has enabled us to develop new acquisition guidelines that allow us to produce visually more attractive models under very-low illumination conditions, as seen in Figure 25.

In general, we were able to assess the overall quality of the HIS-PC based on the criteria of the ISO 9126 standard. It is a qualitative assessment of all the deployed components from the initial user-centred design phases that permit to plan the improvement to include. The summary of this assessment is presented in Table 2.

From this, we developed a plan for the future iterative processes to suit the evolving needs of the end users better. These are summarised in Section 6.

6. Takeaways & Research Perspectives

Archaeological or heritage sites require tailored management systems that fit their particular needs. Frequently developed from existing individual components, such information management systems are difficult to evolve and to generalise to other contexts. Moreover, the end users’ requirements are of great importance to ensure the efficient use of the developed prototype. Some problems arise when user-requirements evolve, logically, with the gain of expertise in heritage information management. Indeed, the use of 3D data +sets representing every detail of archaeological sites or of an information system that relates information as never before opens new horizons in term of application and historical interpretation. The learning process of the end users contributes to the evolution of their requirements, the more they use the system. The approach we developed in this the framework of the specific use-case is intended to overcome this consequence by proposing a user-centred development methodology. The four development steps, namely the user requirements analysis, the conceptual design, the design and prototyping, and the usability evaluation allows continuous development of functionalities. We understood that the application developed should be scalable and that data schemas should be able to be modified throughout the HIS life cycle. The constraints imposed by the project (diversity of stakeholders, size of data sets, etc.) led us to develop a dedicated web application whose interface can be separated from the database. The proposed prototype is tailored to meet the specific needs of the Coudenberg Museum. The development time was, for administrative reasons, concise. This fact prompted us to acquire a backend software component that was sufficiently robust and flexible to contain the variety of the requester’s data sets. After a market study, the choice of this component was that of the software package published by Spatiodata, which specialises in maintenance management systems. The use of this software component allowed us to develop the HIS in 30 working days.

At the current stage of implementation, the HIS-PC only manages 3D points, and 3D bounding boxes but more complex geometries (like 3D surfaces or volumes) could be handled thanks to PostGIS. Moreover, the selection of objects in the spatial database could be based on topological relationships. Finally, to maximise interoperability of the HIS, the different exchanges of information should follow standardised protocols. Notably, the transfer of 3D geometries from the point cloud interface to the spatial database (and vice versa) should be a standard OGC Web Feature Service (WFS) [68] to make elements of the HIS architecture easily replaceable (point cloud interface, semantic interface, GIS interface, etc.). The will of the museum’s experts is to fuse each working environment in a single interface. As mention earlier, this is a stiff challenge to stream several billions of points efficiently in a web browser without overpassing the requirement of high-resolution visualisation for the point cloud, but new solutions such as presented by Schutz et al. in [69] will be investigated.

In the current version of HIS-PC, a user can extract “manually” 3D geometric information from the 3D points cloud and link it to an object in the database. We wish to integrate automatic 3D segmentation and classification functionalities to help users populate and enrich the system. These functionalities are currently under derivation from the research works [70,71,72,73], based on device, analytic and domain knowledge. It requires, in particular, a heritage object domain ontology obtained from standardisation works (e.g., CIDOC-CRM [74]) and in line with the conceptual data model of the HIS-PC.

Although the HIS-PC is intended to fulfil multiple needs, it focuses currently on site conservation, site management, archaeological research and inventories. But there is also a trend to communicate cultural heritage digital content to non-specialists better. Virtual, augmented, or mixed reality techniques (VR/AR/MR) are good candidates for improved interactive communication with specialists and non-specialists. Based on our experience and previous works in the field [7,75,76], we wish to add such visualisation techniques to HIS-PC to extend the uses of cultural heritage digital content (tourism, teaching, cultural mediation…).

7. Conclusions

The sustainability of a Heritage Information System (HIS) demands ways of managing the addition of new data and adapting to new demands (output) using the latest technology. What is essential is not the techniques implemented, but the management of the integration of these evolutions into a modular “system”. It is, therefore, essential to move away from the current trend of separating achievements (preservation vs experiences for non-specialists) to allow a more sustainable approach through evolutionary systems. The provided workflow includes metamodels for designing a HIS based-on point cloud data and a concrete illustration over the Coudenberg Royal Palace Museum. We show that a user-centred approach is beneficial to the success of the application design and that interfacing spatial, temporal and semantic should often be multi-faceted to cope with independent limitations. The system finally demonstrates a new way to directly incorporate point clouds while benefitting of advanced temporal and semantic queries capabilities.