Ontology-based semantic conceptualisation of historical built heritage to generate parametric structured models from point clouds

: Nowadays, cultural and historical built heritage can be more effectively preserved, val-orised and documented using advanced geospatial technologies. In such a context, there is a major issue concerning the automation of the process and the extraction of useful information from a huge amount of spatial information acquired by means of advanced survey techniques (i.e., highly detailed LiDAR point clouds). In particular, in the case of historical built heritage (HBH) there are very few effective efforts. Therefore, in this paper, the focus is on establishing the connections between semantic and geometrical information in order to generate a parametric, structured model from point clouds using ontology as an effective approach for the formal conceptualisation of application domains. Hence, in this paper, an ontological schema is proposed to structure HBH representations, starting with international standards, vocabularies, and ontologies (CityGML-Geography Markup Language, International Committee for Documentation conceptual reference model (CIDOC-CRM), Industry Foundation Classes (IFC), Getty Art and Architecture Thesaurus (AAT), as well as reasoning about morphology of historical centres by analysis of real case studies) to represent the built and architecture domain. The validation of such schema is carried out by means of its use to guide the segmentation of a LiDAR point cloud from a castle, which is later used to generate parametric geometries to be used in a historical building information model (HBIM).


Introduction
In the last decades, the necessity to document and store information related to cultural heritage (CH) and historical built heritage (HBH) has led researchers and experts to focus on the techniques and methods to more effectively represent historical buildings, geometric features and knowledge, spatially. Most recent methods and technologies in this respect adapt building information models (BIM), born to assist in building design and construction for the representation and management of heritage information, known as historical BIM (HBIM) [1,2]. However, the geometry usually stored in BIMs is parametrically

Literature Review
In order to conceptualise and represent building components, several well-known standards have been developed during the past decades. This includes 3D city models (considering standards such as CityGML by the Open Geospatial Consortium) and Building Information Models (BIM), for which the reference standards are the Industry Foundation Classes (IFC) developed by buildingSMART. The standards consider a higher level of representation, especially concerning interior parts of buildings, structures, component materials, etc. However, starting from the introduction of the use of BIM technology for Heritage (HBIM) [8], it has become evident that one of the most problematic aspects of its application is the criticality of making the parametric approach compatible with the uniqueness of historical architecture. This is due to the geometric complexity of the artefacts, degradation and material characterisations: they are different from contemporary buildings, which count on a high level of industrialisation and, consequently, on higher repeatability of components and more simple shapes. Reverse engineering is usually necessary to suitably model historical buildings' components.
This strategy and many consequent developments have been adopted to facilitate and better manage the maintenance of existing buildings [9,10], for conservative and restoration requirements [11,12] and also for the structural analysis and evaluation of historical architecture [13][14][15]. Precisely for these reasons, several efforts are directed to the database design with a particular focus on non-geometric information correlated to HBIM that is repeatable on different examples of heritage [16], with the harmonisation of vocabularies and repositories [17].
In the framework mentioned above, it is possible to collocate some already-made studies concerning standard classification in the built historical heritage domain; HBIM is linked to semantic-web studies and the ontology definition for historical centres and buildings. A starting point towards the creation of an ontology for small historical centres individuating useful classifications, descriptions, and concepts able to formalise the selected domain has been presented in [18]. Moreover, the development of identifying a path for historical fortified buildings and structures' semantic definitions underlining the multitude of use cases concerning HBH and historical urban architecture is proposed 7.
Yang et al. [19] reported the worldwide diffusion of integrated approaches that combine BIM (or HBIM) techniques with information management of semantic knowledge. Lopez et al., (2018) [20] showed a significant strategy for our present work: they combined HBIM models and the Getty Art and Architecture thesaurus (AAT) (a vocabulary for architectural elements' definitions, explained below) in SPARQL (an RDF query language), adding ID semantic information in the HBIM model parameters. This work presented a semantically enriched HBIM library for the elements of a castle.
Other studies [21,22] focused on the semantic enrichment of HBIM models (designed from a 3D metric survey) for the interpretation, management and querying of historical buildings organising knowledge using ontologies and databases to store hierarchical information and properties.
As ref. [23] attempted to define IFC in an ontology language (OWL), they connected IFC and RDF, creating a new structure which resulted in an ifcOWL ontology. Very recently, some authors [24] worked to define an ontology-based database providing a vocabulary to query an HBIM model of vaults, and ref. [25] studied an integration workflow connecting HBIM modelling and ontological knowledge.
Other research works attempt to use different standards and vocabularies from the geographic information domain for historical heritage representation. For instance, CityGML [26] is an international standard data model published by the Open Geospatial Consortium (OGC) in 2008 (developed since 2002) to represent multiscale 3D information about entities of cities. It is an open data model for the storage and exchange of 3D city models.
Concerning the architectural heritage, the IFC (Industry Foundation Classes) in the BIM (Building Information Model) environment for parametric modelling must be considered [27].
Regarding the documentation of CH, it is possible to mention two primary standards: the International Committee for Documentation (CIDOC)-CRM (CIDOC conceptual reference model) core ontology [28] and the Getty AAT (Art and Architecture Thesaurus). The CIDOC-CRM was developed by the International Committee for Documentation (CIDOC) of the International Council of Monuments (ICOM) and is the core ontology for managing cultural heritage information, now standard ISO 21127 [29]. This standard is aimed at enabling the exchange and integration of information between heterogeneous data sources. It was initially developed for the representation of the knowledge of museum objects.
The Getty vocabularies [30] propose terms connected to cultural heritage, as premiere references to categorise works of art, architecture, material culture, the names of artists or architects, as well as the geographic categories. One of these is the Art and Architecture Thesaurus (AAT) to describe works of art and architecture. Terms in the AAT may be used to describe art, architecture, decorative arts, material culture, and archival materials. The target audience includes museums, libraries, visual resource collections, archives, conservation projects, cataloguing projects, and bibliographic projects. Recently, the Getty vocabularies also developed a SPARQL version of the thesaurus available on-line [31].
In brief, despite many studies, there is no existing ontology or standard that provides a complete semantic formalisation of HBH and their elements. Nevertheless, there is a wealth of knowledge in terms of core ontologies and legislation documents that could serve as a basis for developing a more adapted domain ontology for the exploration of historical buildings. In this framework, it is possible to mention various studies focused on combining 3D modelling of HBH, precisely HBIM, and knowledge-based or ontological approaches, reusing existing knowledge and standards in the domain of geographic information and built heritage representation. Such an ontology can, subsequently, be used to bridge semantic and geometrical/topological information acquired with advanced technologies (e.g., LiDAR point clouds) to generate parametric structured models of built heritage for their effective documentation, preservation, valorisation, and restoration.

Proposed Ontology-Based Method for the Generation of Parametric Structured Models for Historical Built Heritage
Here we propose a workflow for the conceptualisation and 3D modelling of selected domains of HBH in particular castles and fortified structures in historical centres. The proposed workflow is composed of several steps that allow bridging between an ontological representation of the historical sites with their effective 3D representation using a detailed LiDAR data set. The present study adapts the following reproducible workflow (illustrated in Figure 1): The target audience includes museums, libraries, visual resource collections, archives, conservation projects, cataloguing projects, and bibliographic projects. Recently, the Getty vocabularies also developed a SPARQL version of the thesaurus available on-line [31]. In brief, despite many studies, there is no existing ontology or standard that provides a complete semantic formalisation of HBH and their elements. Nevertheless, there is a wealth of knowledge in terms of core ontologies and legislation documents that could serve as a basis for developing a more adapted domain ontology for the exploration of historical buildings. In this framework, it is possible to mention various studies focused on combining 3D modelling of HBH, precisely HBIM, and knowledge-based or ontological approaches, reusing existing knowledge and standards in the domain of geographic information and built heritage representation. Such an ontology can, subsequently, be used to bridge semantic and geometrical/topological information acquired with advanced technologies (e.g., LiDAR point clouds) to generate parametric structured models of built heritage for their effective documentation, preservation, valorisation, and restoration.

Proposed Ontology-Based Method for the Generation of Parametric Structured Models for Historical Built Heritage
Here we propose a workflow for the conceptualisation and 3D modelling of selected domains of HBH in particular castles and fortified structures in historical centres. The proposed workflow is composed of several steps that allow bridging between an ontological representation of the historical sites with their effective 3D representation using a detailed LiDAR data set. The present study adapts the following reproducible workflow (illustrated in Figure 1): 1. The selection of castles and fortified structures as archetypal models and definitions, and the identification of geometrical characterisation (morphology) of building elements starting from case studies (Section 3.1). 2. A survey and analysis of existing descriptions referred to as HBH in vocabularies and standards, to spatially represent them (Section 3.2). 1.
The selection of castles and fortified structures as archetypal models and definitions, and the identification of geometrical characterisation (morphology) of building elements starting from case studies (Section 3.1).

2.
A survey and analysis of existing descriptions referred to as HBH in vocabularies and standards, to spatially represent them (Section 3.2).

5.
Validation of the developed ontology by means of semantic segmentation of a case study point cloud by detecting building parts based on topological relations and conceptualisations defined in the ontology (Section 5).
This methodological approach follows three different levels, presented in the next sections:

•
First level: concepts and relations derived from ontologies and standards.

•
Second level: domain knowledge definition from instances (case studies).

•
Third level: geometry definition and segmentation (through the 3D model design).

Identifying Specific Morphology through Archetypal Model
To identify the most representative morphological characters of castles and towers (from the medieval period), we have resorted to the constructive solutions of designers of the past who intended to convey universal or archetypal values of fortified structures with their works.
The first example is the Castellina fortress in Norcia ( Figure 2) [32], commissioned by Pope Julius III and designed in 1554 by Jacopo Barozzi da Vignola, an architect and a relevant theorist and treaties writer of architecture in the Italian Renaissance panorama. The fortified structure blends the Renaissance building's stylistic features with the fortress's formal and constructive characteristics typical of the medieval period; the building is a square plan with four mighty corner towers and scarped masonry.

Identifying Specific Morphology through Archetypal Model
To identify the most representative morphological characters of castles and towers (from the medieval period), we have resorted to the constructive solutions of designers of the past who intended to convey universal or archetypal values of fortified structures with their works.
The first example is the Castellina fortress in Norcia ( Figure 2) [32], commissioned by Pope Julius III and designed in 1554 by Jacopo Barozzi da Vignola, an architect and a relevant theorist and treaties writer of architecture in the Italian Renaissance panorama. The fortified structure blends the Renaissance building's stylistic features with the fortress's formal and constructive characteristics typical of the medieval period; the building is a square plan with four mighty corner towers and scarped masonry.
The second essential reference for this study is located in Turin, the Rocca (Figure 3), together with the connected Borgo Medievale, and was designed by Alfredo d'Andrade to symbolise the medieval Italian styles [33]. The Rocca, the innermost keep of a reconstructed hamlet, has the purpose of revisiting some real medieval architectures spread in two Italian regions, Piedmont and Valle d'Aosta. On the occasion of a crucial universal exhibition, the architect aimed to present the characteristics of native architecture to the attendants.
Starting from these examples that intrinsically contain the morphological aspects of construction practices of previous centuries for fortified buildings, we can identify the following distinctive characteristics: • impressive masonries; buttresses masonries; sloping scarp walls; • an angular tower (squared, polygonal and circular); • small vertical openings and windows; • a watchtower (in Italian "garitta" which is a sentry box); • a merlon (in Italian "merli", and they are at the top of the walls); • the presence of a bulwark or moat. The second essential reference for this study is located in Turin, the Rocca (Figure 3), together with the connected Borgo Medievale, and was designed by Alfredo d'Andrade to symbolise the medieval Italian styles [33]. The Rocca, the innermost keep of a reconstructed hamlet, has the purpose of revisiting some real medieval architectures spread in two Italian regions, Piedmont and Valle d'Aosta. On the occasion of a crucial universal exhibition, the architect aimed to present the characteristics of native architecture to the attendants.
Starting from these examples that intrinsically contain the morphological aspects of construction practices of previous centuries for fortified buildings, we can identify the following distinctive characteristics: • impressive masonries; buttresses masonries; sloping scarp walls; • an angular tower (squared, polygonal and circular); • small vertical openings and windows; • a watchtower (in Italian "garitta" which is a sentry box); • a merlon (in Italian "merli", and they are at the top of the walls); • the presence of a bulwark or moat.

Study and Analysis of Definitions of HBH to Identify Their Morphology
This step of the methodology aims at collecting elements and concepts related to fortified structures and castles' architectural characters. It has been developed to compare conceptualisations, hierarchies, and descriptions from existing vocabularies and standards. This analysis allowed us to point out the most relevant elements and notions about castles and towers.

Concepts Related to Castles and Fortified Structures: The Getty AAT, CityGML and IFC as Standards
The present subsection compares concepts and definitions related to castles and fortified structures in the Getty AAT vocabulary, the CityGML and IFC standards. Table 1 shows the main entities. Getty AAT: counterscarps Refers to the outer slopes of encircling defensive ditches; for the inner slopes, use "scarps".
Getty AAT: scarps (fortification elements) Steep slopes immediately in front of and below a fortification; for the outer slopes of encircling ditches of fortifications, use "counterscarps". Getty AAT: fortification elements Architectural elements used to fortify structures.
Getty AAT: fortification openings Openings located in fortresses or fortified structures.

Study and Analysis of Definitions of HBH to Identify Their Morphology
This step of the methodology aims at collecting elements and concepts related to fortified structures and castles' architectural characters. It has been developed to compare conceptualisations, hierarchies, and descriptions from existing vocabularies and standards. This analysis allowed us to point out the most relevant elements and notions about castles and towers.

Concepts Related to Castles and Fortified Structures: The Getty AAT, CityGML and IFC as Standards
The present subsection compares concepts and definitions related to castles and fortified structures in the Getty AAT vocabulary, the CityGML and IFC standards. Table 1 shows the main entities. Table 1. List of characters or elements concerning fortified structures, inferable by standard concepts correspondences.

Concepts from Standards Definitions fortified structures, castles
Getty AAT: fortification General term for any works made to oppose a small number of troops against greater numbers.
Getty AAT: fortified settlements Settlements of any kind with defensive structures such as moats, enclosures, or ramparts.
Getty AAT: counterscarps Refers to the outer slopes of encircling defensive ditches; for the inner slopes, use "scarps".
Getty AAT: scarps (fortification elements) Steep slopes immediately in front of and below a fortification; for the outer slopes of encircling ditches of fortifications, use "counterscarps".
Getty AAT: fortification elements Architectural elements used to fortify structures.
Getty AAT: fortification openings Openings located in fortresses or fortified structures. building

CityGML-CityObject→Site→AbstractBuilding→Building
Allows the representation of thematic and spatial aspects of buildings, building parts and installations in four levels of detail: LOD1 to LOD4.

CityGML-Building→Building parts
Building part is a subdivision of a building that is homogeneously related to its physical, functional or temporal aspects, and may be considered as a building.

Concepts from Standards Definitions
IfcSite→IfcBuilding A building represents a structure that provides shelter for its occupants or contents and stands in one place.

IfcSite→IfcBuilding→IfcBuildingElements
The building element comprises all elements that are primarily part of the construction of a building towers Getty AAT: antiaircraft towers Large antiaircraft gun blockhouses used by the Luftwaffe in World War II to protect against Allied overflights, serve as air-raid shelters, and coordinate air defence. They had concrete walls up to 3.5 m thick and were considered invulnerable.
Getty AAT: defensive towers Towers used specifically for defence purposes.
Getty AAT: watchtowers Any built structure used as a lookout station that affords observation of a surrounding area.
wall IfcBuildingElement→ifcWall Vertical construction usually in masonry or in concrete which bounds or subdivides a construction works and fulfils a load bearing or retaining function. Definition from buildingSMART: The wall represents a vertical construction that bounds or subdivides spaces.

CityGML CityObject→BoudarySurface→SurfaceWall
The embrasure surface on the one side of the door that belongs to the InteriorWallSurface and on the other side to the WallSurface.
Getty AAT: defensive wall Use broadly for all sorts of walls constructed to resist attack.
Getty AAT: bulwarks The strong outer works of a fortification, which are usually low enough to permit defensive fire.

Concepts and Definitions from the Italian Castles Institute to Identify Fortified Structures Morphology
The concepts of fortified structures extracted from standards or vocabularies are rather general; curiously, Getty ATT provides both comprehensive definitions of defensive towers and a specific one related to antiaircraft built structures used to provide shelter from aerial bombings during the Second World War.
Although it is essential to consider the era of construction of fortified structures, this study aims at identifying recurring features of historical fortifications to recognise morphological characteristics that connect the external shape with the geometric nature of the model (e.g., in HBIM), and the semantic values of objects.
Towers and castles are buildings intended for defence. They characterised the construction methods and systems directly connected to the ways of "making war". For this reason, the disciplines of the history of architecture and the history of urban planning have been very interested in the evolution of fortified structures over the different centuries until the modern era. Due to the scarcity of definitions and information in the standards, we looked for other meanings and conceptualisations provided by scientific communities, such as medieval historians of architecture. An association of historians called the Italian Castelli Institute [34] provided a useful castle nomenclature dictionary (Table 2). Masonry expedient of the upper parts of the castle walls and towers, well known under the iconographic profile. It consisted of a symmetrical interruption of the wall, behind which the shooter sheltered to escape from the opponent's reaction. It is called Guelfo or Ghibellino, depending on whether it ended in par or dovetail but the denomination has not always corresponded to the castellans' faction. Other and different shapes were adopted according to the places: triple-toothed, flower-shaped, pyramid-shaped, semicircle (for which the set of battlements was also called lace, prominence). In the most ancient feudal castles, the blackbird was built flush with the wall's outer surface; it was later carved out of the projecting walls to allow a more effective swooping defence. It was made also tapered, double sloping, with curbs or frames to prevent it slipping of arrows or lightning bolts, and with slits. These notes as mentioned above are basic definitions for castles, and many of them contribute to understanding fortified structures' morphology; moreover, we can see that there are many architectural elements or elements of the type of construction that characterise castles and towers (especially from the medieval period) that allow us to identify them clearly.
The structures' architectural morphology enables the identification of geometric and topological rules for the model generation (some examples in Figure 4). These notes as mentioned above are basic definitions for castles, and many of them contribute to understanding fortified structures' morphology; moreover, we can see that there are many architectural elements or elements of the type of construction that characterise castles and towers (especially from the medieval period) that allow us to identify them clearly.
The structures' architectural morphology enables the identification of geometric and topological rules for the model generation (some examples in Figure 4).

Reasoning about Morphology from Case Studies Analysis
We considered these two built heritage samples (the Castellina and the Rocca medioevale) and the related 3D models to link the definitions with the identification of the morphological aspects because, in the architects' intentions, they both embody the archetypal image of the castle and towers.
In this section the recurring morphological elements have been analysed, listing concepts and geometric/topological features: • sloping scarp walls: the scarp masonry and the portion of the vertical wall with which it is connected; they can be identified since they are two contiguous planar surfaces, which share a line that is the intersection between planes. The sloping scarp masonry can be noticed in the Castellina. In the XIX century construction, the scarp walls are missing since other typical features and elements of the fortress were enough to communicate castles' general configuration. • towers: as the definition states, a tower has a simple shape, with a square, circular, or polygonal plan. Then, the definition also includes more complex open shapes, but the most common towers are featured by a square or circular base. Morphology: if the towers are isolated, their geometric characterisation corresponds to a cylinder in the 3D space or a parallelepiped with a square base. If they are not separated, but placed at the corners of the castle, we must first examine the morphology of the court, since the organisation around a courtyard with angular towers is very typical. Another geometric feature is that the towers are higher than the fortified building or castle to which they are related. It can also be argued that the structures, for many centuries, were built with few floors above ground level, so castles and towers are higher than the average height of the buildings of medieval towns. • courtyard: a courtyard is an open-air place, from the geometric point of view; the centroid of the volume of the entire building falls within an empty area and not within its projected polygon ( Figure 5). • windows and openings: in modern construction, starting from the nineteenth-century regulations, indoor environments' healthiness, lighting and ventilation must be guaranteed. Therefore, the present construction rules provide for minimum surfaces of building openings. Simultaneously, the window openings' surfaces in towers and castles cover a much lower percentage than the minimum characters envisaged today or in historic buildings not designated for defence.
polygonal plan. Then, the definition also includes more complex open shapes, but the most common towers are featured by a square or circular base. Morphology: if the towers are isolated, their geometric characterisation corresponds to a cylinder in the 3D space or a parallelepiped with a square base. If they are not separated, but placed at the corners of the castle, we must first examine the morphology of the court, since the organisation around a courtyard with angular towers is very typical. Another geometric feature is that the towers are higher than the fortified building or castle to which they are related. It can also be argued that the structures, for many centuries, were built with few floors above ground level, so castles and towers are higher than the average height of the buildings of medieval towns. • courtyard: a courtyard is an open-air place, from the geometric point of view; the centroid of the volume of the entire building falls within an empty area and not within its projected polygon ( Figure 5). • windows and openings: in modern construction, starting from the nineteenthcentury regulations, indoor environments' healthiness, lighting and ventilation must be guaranteed. Therefore, the present construction rules provide for minimum surfaces of building openings. Simultaneously, the window openings' surfaces in towers and castles cover a much lower percentage than the minimum characters envisaged today or in historic buildings not designated for defence.

Ontology Development
The present section is dedicated to defining an ontology structure following the conceptualisations and definitions carried out in the methodological workflow. The ontology has been designed in Protègè (an open-source ontology editor and framework for building intelligent systems) [35]. Here, different classes (concepts with definitions) with their relations (object properties), characteristics and values (data properties and

Ontology Development
The present section is dedicated to defining an ontology structure following the conceptualisations and definitions carried out in the methodological workflow. The ontology has been designed in Protègè (an open-source ontology editor and framework for building intelligent systems) [35]. Here, different classes (concepts with definitions) with their relations (object properties), characteristics and values (data properties and datatypes) have been added. Moreover, thanks to the implemented tools in Protègè, it was possible to integrate and merge existing ontologies of IFC and CityGML.
To enrich the first level ( Figure 1, concepts and relation), with information relevant to topology and other useful elements for generating the second level (the domain knowledge), we also considered conceptualisations from CIDOC CRM (the core ontology for cultural heritage, by the International Committee for Documentation), CIDOC CRMba (for the documentation of archaeological buildings) and CIDOC CRMgeo (for spatiotemporal properties of temporal entities and persistent items) ontologies. Figure 6 shows the hierarchy classes with their properties. Various classes derive from different standards, ontologies and reasonings about HBH elements. A particular focus has been paid to AbstractBuilding and building classes (building element and building part) definitions from IFC and CityGML, defining meronymic (has-part) and hierarchical relations. Moreover, classes from the cultural heritage domain, such as the architectural element, castle and fortified structure have been added.
Every concept in the developed ontology has URI (Uniform Resource Identifier) and a description derived by standards analyses (above reported) or CIDOC-CRM ontologies documentation. Table 3 presents an example of definitions from CIDOC.
The ontologies' properties have been implemented in Protègè by using object properties and data properties (Figure 7) They correspond to the relationships between concepts (entities/classes defined) and their attributes or values. Some of them are ad-hoc created for the domain ontology of historical built heritage; others are integrated and imported from existing ontologies (such as CIDOC-CRM). Figure 8 shows entities and relations in the WebVOWL application, and Tables 4 and 5 show the reported list of properties with their domain and range entities and datatypes. spatiotemporal properties of temporal entities and persistent items) ontologies. Figure 6 shows the hierarchy classes with their properties. Various classes derive from different standards, ontologies and reasonings about HBH elements. A particular focus has been paid to AbstractBuilding and building classes (building element and building part) definitions from IFC and CityGML, defining meronymic (has-part) and hierarchical relations. Moreover, classes from the cultural heritage domain, such as the architectural element, castle and fortified structure have been added.

CIDOC Concepts Definitions
Built Work Man-made things such as freestanding buildings, components of buildings, and complexes of buildings.

Morphological Building Section
Man-made things that are considered functional units for the whole building (e.g., rooms, foundations, roof, and so forth).

Filled Morphological Building Section
Man-made things that are completely filled with matter. Instances of filled morphological building section are components of the functional units of a built work. These are elements like walls, floors and ceilings

Physical Man-Made Thing
This class comprises all persistent physical items that are purposely created by human activity. This class comprises man-made objects, such as a sword, and man-made features, such as rock art.

CRM Entity
This class comprises all things in the universe of discourse of the CIDOC conceptual reference model.

Person
This class comprises real persons who live or are assumed to have lived.

Place
This class comprises extents in space (particularly on the surface of the earth) in the pure sense of physics, independent from temporal phenomena and matter.

Phenomenal Place
This class comprises instances of place whose extent and position are defined by the spatial projection of the spatiotemporal extent of a real-world phenomenon that can be observed or measured.       Moreover, for the third level of the methodology's workflow, we present a partial example of the conceptual formalisation in the case study of Castellina of Norcia by using some indicative concepts and relations considered in the ontology development (Figures 8 and 9).

Validation: The Use of the Ontology to Extract a Historical Site Component from LiDAR Data
This section explains the validation of the proposed methodology. For this purpose, we present a case study for the extraction of the components of a historical building through a semantic reasoning approach applied on a set of detailed 3D LiDAR point clouds ( Figure 10).

LiDAR Data Acquisition
A complete UAV (unmanned aerial vehicle) survey has been carried out to obtain high-detailed point clouds for Norcia main square and Castellina [36,37]. Over the analysed areas, some low altitude UAV flights were performed using the Phantom 4 PRO and DJI Spark platforms in order to accurately describe the outer part of the castles (an average elevation flight height of 50 m was used). Complying with the consolidated trends in UAV image acquisition, the flights over the fortified structures were planned to acquire nadir and oblique images, to obtain a GSD (ground sampling distance) with a magnitude of less than 2.5 cm, which is suitable for an architectural scale (1:100-1:200).
A geo-reference system was established thanks to the integration of a principal network acquired with a GNSS system, and a vast amount of artificial and natural targets that were measured to obtain a reliable network of controls and checkpoints, useful to optimise and evaluate the bundle block adjustment of the photogrammetric process.

Validation: The Use of the Ontology to Extract a Historical Site Component from LiDAR Data
This section explains the validation of the proposed methodology. For this purpose, we present a case study for the extraction of the components of a historical building through a semantic reasoning approach applied on a set of detailed 3D LiDAR point clouds ( Figure 10).

LiDAR Data Acquisition
A complete UAV (unmanned aerial vehicle) survey has been carried out to obtain high-detailed point clouds for Norcia main square and Castellina [36,37]. Over the analysed areas, some low altitude UAV flights were performed using the Phantom 4 PRO and DJI Spark platforms in order to accurately describe the outer part of the castles (an average elevation flight height of 50 m was used). Complying with the consolidated trends in UAV image acquisition, the flights over the fortified structures were planned to acquire nadir and oblique images, to obtain a GSD (ground sampling distance) with a magnitude of less than 2.5 cm, which is suitable for an architectural scale

Semantic Segmentation of LiDAR Point Cloud for Object Extraction
For the purpose of the extraction of the components of a historical site from LiDAR point clouds presented in the previous section, we chose to use a machine-learning approach. These methods allow more automatic segmentation and extraction of information from the LiDAR data. With the machine learning-based solutions, we attempted to regroup the points belonging to each predefined class of objects in the A geo-reference system was established thanks to the integration of a principal network acquired with a GNSS system, and a vast amount of artificial and natural targets that were measured to obtain a reliable network of controls and checkpoints, useful to optimise and evaluate the bundle block adjustment of the photogrammetric process.

Semantic Segmentation of LiDAR Point Cloud for Object Extraction
For the purpose of the extraction of the components of a historical site from LiDAR point clouds presented in the previous section, we chose to use a machine-learning approach. These methods allow more automatic segmentation and extraction of information from the LiDAR data. With the machine learning-based solutions, we attempted to regroup the points belonging to each predefined class of objects in the proposed HBH ontology. This necessitates the training of a classifier on a data set as the first step.
For this purpose, a feature vector is designed to describe the properties of each point. The feature vector generally consists of quantitative values for each feature. For the task of extracting points belonging to our historical building from point clouds, semantic segmentation based on the feature vectors proposed in [38] is chosen to process the input point clouds presented in the previous section. In this work, among others, features such as (i) "directional height above" compares the height difference between a point and its neighbours in eight directions, (ii) "difference of normal" is based on a normal estimation which can better distinguish when vegetation and man-made objects are used.
Then, following semantic segmentation of the point cloud, they were used in a CADlike segmentation. This approach decomposes historical buildings into their components based on their geometric properties. Each building component is identified as a geometric shape from point clouds. The normal estimation is commonly used to extract the geometric properties such as smoothness of surfaces from point clouds.
A region-growing algorithm is applied to detect the smoothness of surfaces through predefined criteria (e.g., the proximity, smoothness, and curvature) of growing surfaces by comparing the properties of adjacent points. The smoothness is evaluated by the angle between the normal vectors based on the normal estimation of point clouds. An appropriate smoothness threshold depending on point cloud quality (e.g., point density and noise) can mitigate cases with under-segmentation and over-segmentation results. For deciding the smoothness threshold automatically from point clouds, Otsu's method [39,40] was chosen to help to mitigate the interference of edge points during the growing regions. We ran the region-growing segmentation algorithm continually on the segmentation results from the previous step to decrease under-segmentation results for the case that two planar surfaces connect with a gentle angle. Then RANSAC (RANdom SAmple Consensus) algorithm [41] is used to estimate the geometric parameters of primitives representing building components.

Extraction of Topological Relations between Geometric Components
The topological relations between the components are summarised as a 3*3 matrix of the dimensionally extended 9-intersection model (DE-9IM) [42] to represent topological relations. As the description presented in [43], the matrix is composed of the dimension operation on the interior, the boundaries of the two planar regions that represent building elements, and the intersection line between the two plane regions in 3D space. Based on the representation of DE-9IM for recording topological relations between planar regions, the semantic representation of the topological relations between the two planar regions is determined by analysing the relation between the intersection line and two planar regions, and the relations between the intersected parts composed of the intersection operation between planar regions and the intersection line. For example, the topological relations between the intersected parts could be "point-point" relations, "point-line segment" relations and "line segment-line segment" relations.
For describing the topological relations in a semantically meaningful way, the formalised representation of topological relations between the planar regions is described by the four-word description "T p1 -T p2 -T p3 -T p4 ". The first part T p1 describes the overall topological relations between the two planar regions. It could be disjointed, met, and intersected. The second part T p2 represents the relation between planar region A and the intersection line. It could be disjointed, met, and overlapped. The third part T p3 represents the relation between planar region B and the intersection line. It could be disjointed, met, and overlapped as well. The fourth part T p4 represents the relation between the intersected parts on the intersection line. It depends on the intersected parts created by the planar regions and the intersection line. When it is the relation between two points, it could be disjointed and equal. When it represents the relation between a line segment and a point, it could be disjointed, met and contained. When it represents the relation between two-line segments, it could be disjointed, met, overlapped, covered, covered by, contained, contained by, and equal. Finally, the topological relations between two planar regions in 3D space are formalised as semantic descriptions.
Based on the segmentation results of HBH buildings from point clouds, the geometric information of the building components is calculated from segments, and the topological relations between the building components can be identified as well. The geometric information of building components and the topological relations between components make it possible to extract the semantic information related to the building structure constructed from basic geometric and topological information.

Semantic Reasoning for the Recognition of Historical Building Components
For the extraction and recognition of historical building components from point clouds, there are several critical steps as follows.

•
First, building a knowledge base is required to formalise the knowledge about building structure and its components. To this end, we make use of our proposed HBH ontology. In addition, for semantic reasoning purposes, semantic rules are defined based on the concepts, relations, properties, as well as the different constraints on relations and properties. The rules are formalised in a human-readable format. • Second, for linking the segmentation results of buildings and the knowledge base, the segmented building components are translated into instances of concepts. The geometric information is translated into the properties of instances. The geometric and topological relations are translated into relations among instances. The information related to instances is viewed as fact for reasoning in the knowledge base. • Finally, the reasoning process is carried out to recognise the impressive historical building structure based on the segmentation results of buildings from point clouds and prior knowledge of the building structure. Depending on the definition of the semantic rules formalising knowledge, the recognition of the studied complex building structure can benefit from the definition of the connection between basic building components (e.g., wall, roof) based on formalised topological relations. regions and the intersection line. When it is the relation between two points, it could be disjointed and equal. When it represents the relation between a line segment and a point, it could be disjointed, met and contained. When it represents the relation between twoline segments, it could be disjointed, met, overlapped, covered, covered by, contained, contained by, and equal. Finally, the topological relations between two planar regions in 3D space are formalised as semantic descriptions. Based on the segmentation results of HBH buildings from point clouds, the geometric information of the building components is calculated from segments, and the topological relations between the building components can be identified as well. The geometric information of building components and the topological relations between components make it possible to extract the semantic information related to the building structure constructed from basic geometric and topological information.

Semantic Reasoning for the Recognition of Historical Building Components
For the extraction and recognition of historical building components from point clouds, there are several critical steps as follows. • First, building a knowledge base is required to formalise the knowledge about building structure and its components. To this end, we make use of our proposed HBH ontology. In addition, for semantic reasoning purposes, semantic rules are defined based on the concepts, relations, properties, as well as the different constraints on relations and properties. The rules are formalised in a human-readable format.

•
Second, for linking the segmentation results of buildings and the knowledge base, the segmented building components are translated into instances of concepts. The geometric information is translated into the properties of instances. The geometric and topological relations are translated into relations among instances. The information related to instances is viewed as fact for reasoning in the knowledge base.

•
Finally, the reasoning process is carried out to recognise the impressive historical building structure based on the segmentation results of buildings from point clouds and prior knowledge of the building structure. Depending on the definition of the semantic rules formalising knowledge, the recognition of the studied complex building structure can benefit from the definition of the connection between basic building components (e.g., wall, roof) based on formalised topological relations.

Discussion on the Historical Buildings Components Extraction Results
The results of the semantic segmentation are in accordance with the architectural reading of the buildings. This description and reasoning about ontology classes and relations was also useful for detecting and selecting properties for the ontology design.
As shown in Table 6 the vertical walls and scarp walls are identified by analysing the geometric relations between the segments and ground. After the topological detection, the vertical walls above the scarp walls are distinguished. Based on the knowledge that a tower includes a vertical and inclined surface, the group of walls, colourised as red, and the scarp walls are identified as the tower. The opening can be detected from the walls as well. Further, Table 6 shows the ontological classes and relations detected in the morphology analysis used for the semantic segmentation of the point clouds of the case studies. The third column expresses whether the segmentation was able to detect the object classes (for example, in the first line we can assert that the semantic segmentation detected the scarp walls from the vertical walls. In the second line, we can say that if the semantic segmentation detected the buildings bodies, it indirectly detected the courtyard, which is the space located inside the building wings).
As can be observed in the previous paragraphs, the case studies' point clouds representing urban scenes and buildings were segmented according to the terms, rules and topological relations defined in the ontology.
In this sense, the proposed ontology-based method for the generation of parametric structured models for historical built heritage is directly connected to the semantic segmentation, which is one of the validated part of the methodological solution.
The combination of these different approaches is, therefore, the main challenge that we want to address by proposing the described novel workflow.

Critical Conclusions and Future Work
Certainly, harmonising the rules for defining the geometric and semantic contents of the 3D models of built environments is one of the most relevant challenges facing the framework of spatial and urban information management. When these purposes assert themselves in complex cultural and historical heritage domains, which involves considering parts of knowledge that is reciprocally related, the issue becomes more complex and an interdisciplinary approach is highly recommended. The workflow experimented with and presented in this paper shows that the direction is valid for a particular historical heritage sample, such as castles and towers. Moreover, the methodology can be extended to many other morphological featured domains by experimenting with proper rules of semantic segmentation.
The designed ontology incorporates concepts and relations from different ontologies and vocabularies that express different interacting views in the historical built heritage (HBH) domain. Semantic information derived from the various case studies was also used to enrich the knowledge related to fortified structures. Since the field of study is vast, such an ontology could be further used to define relations among semantic information and spatial 3D models ( Figure 13). methodology can be extended to many other morphological featured domains b experimenting with proper rules of semantic segmentation.
The designed ontology incorporates concepts and relations from different ontologie and vocabularies that express different interacting views in the historical built heritag (HBH) domain. Semantic information derived from the various case studies was also use to enrich the knowledge related to fortified structures. Since the field of study is vast, suc an ontology could be further used to define relations among semantic information an spatial 3D models ( Figure 13).
On the other hand, by adopting the ontology-based approach or defining high-leve ontological schemes, specific domain knowledge reuse will increase. The short-term perspective that seems most suitable is critically reviewing the structuring of HBIM models, designed according to current canons, and implementing it with the new classes entities and relations expressed by the outlined ontology ( Figure 14).  On the other hand, by adopting the ontology-based approach or defining high-level ontological schemes, specific domain knowledge reuse will increase. The short-term perspective that seems most suitable is critically reviewing the structuring of HBIM models, designed according to current canons, and implementing it with the new classes, entities and relations expressed by the outlined ontology ( Figure 14).
Appl. Sci. 2021, 11, x FOR PEER REVIEW 21 of 23 methodology can be extended to many other morphological featured domains by experimenting with proper rules of semantic segmentation. The designed ontology incorporates concepts and relations from different ontologies and vocabularies that express different interacting views in the historical built heritage (HBH) domain. Semantic information derived from the various case studies was also used to enrich the knowledge related to fortified structures. Since the field of study is vast, such an ontology could be further used to define relations among semantic information and spatial 3D models ( Figure 13).
On the other hand, by adopting the ontology-based approach or defining high-level ontological schemes, specific domain knowledge reuse will increase. The short-term perspective that seems most suitable is critically reviewing the structuring of HBIM models, designed according to current canons, and implementing it with the new classes, entities and relations expressed by the outlined ontology ( Figure 14).  Finally, another future development could regard the BIM-GIS (Geographic Information Science) harmonisation by checking the geometry, data formats and attribute integrations through the insertion of the 3D model in an ontology-based GIS environment [44].