Semantic HBIM for Heritage Conservation: A Methodology for Mapping Deterioration and Structural Deformation in Historic Envelopes

Abstract

The conservation and intervention of heritage structures require a flexible, interdisciplinary environment capable of managing data throughout the building’s life cycle. Historic building information modeling (HBIM) has emerged as an effective tool for supporting these processes. Originally conceived for parametric construction modeling, BIM can also integrate historical transformations, aiding in maintenance and preservation. Historic buildings often feature complex geometries and visible material traces of time, requiring detailed analysis. This research proposes a methodology for documenting and assessing the envelope of historic buildings by locating, classifying, and recording transformations, deterioration, and structural deformations. The approach is based on semantic segmentation and classification using data from terrestrial laser scanning (TLS) and unmanned aerial vehicles (UAVs), applied to the Palace of Miguel de Mañara—an iconic 17th-century building in Seville. Archival images were integrated into the HBIM model to identify previous restoration interventions and assess current deterioration. The methodology included geometric characterization, material mapping, semantic segmentation, diagnostic input, and temporal analysis. The results validated a process for detecting pathological cracks in masonry facades, providing a collaborative HBIM framework enriched with expert-validated data to support repair decisions and guide conservation efforts.

1. Introduction

The concept of building information modeling (BIM) is considered as one of the technological components in the 3D technology category. This collaborative methodology for the design and management of construction projects is assumed by a new area of knowledge applied to the historic building. Its acronym is HBIM, and it was defined by [1] as the procedure of mapping BIM objects from the point cloud by reverse engineering. The process develops geometric shapes that represent elements of historical construction in a virtual environment.

Consequently, historic building information modeling has become the best tool designed to adapt to the activities of a restoration project. This methodology is specifically designed to record parametric construction models as an information manager by adding semantic components to the model, including information on the different transformations of the historical artifact and allowing continuous progress in the life cycle of the building. Through this BIM methodology, it is possible to increase the phase of aggregation of information to the model, manage collaborative databases among a variety of software, and create new architectural configuration components sequentially. In this way, stratigraphic information is one of the main characteristics that can be studied in this model. In professional practice, comparing the point cloud with digital parametric construction models allows for both qualitative and quantitative assessment of the advanced degradation state of materials. In selecting the set of points captured from the building’s surface, it is necessary to implement classification procedures through segmentation algorithms to identify which elements are suitable for modeling and which are not. These compositional elements can potentially be integrated into a digital library, serving as a systematic method for capturing and recording architectural features as intelligent components [2]. Incorporating the data extracted from point clouds obtained through techniques such as photogrammetry or LiDAR into the BIM environment enhances the completeness of 3D model representations [3]. The point cloud data, referred to as attributes, provide metric accuracy, as well as information on material reflectivity, texture, and colorimetric properties.

2. Literature Review

2.1. The Stratification of Heritage

The temporal evolution of a historic building is evident, and an analysis of the transformations in the structures and construction techniques that have occurred is essential for proper conservation, making the stratigraphic study an efficient tool. Stratigraphic analysis is governed by codified terminologies and knowledge, used by archaeologists and architects in the restoration of historic walls. This methodology is governed by well-agreed theoretical principles and procedures that have been adapted through new research [4,5,6]. With the stratigraphic analysis, the evolution in time or history of the construction of the building event is established, auscultating the walls discovered in the interventions to constitute a clear identification of the chronological phases, always supported by adequate knowledge of the construction techniques of the different analyzed historical periods. Although the methodology has been very effective since its inception, it is necessary to update the graphic techniques of data collection and multidisciplinary management between operators. That is, without forgetting the use of traditional CAD for the analysis of stratifications from orthoimages, it is necessary to give priority to the use of three-dimensional works of the archaeological environment [7]. And the interoperability of data, graphic and non-graphic, with the agents involved in a conservation and restoration process would be achieved with the implementation of the BIM methodology in the heritage.

2.2. The BIM Methodology in the Management of Historical Projects

In any historic building, there is a compositional structure of elements that must be studied when proposing a model of a facade. Modulating complex structures involves recognizing that there are elements of artifacts that, due to their geometric properties, require adaptation mechanisms through meshes generated by software other than BIM. It must also be admitted that although the virtual construction of the walls is executed with theoretical modeling, when comparing the point cloud with the element of the model, discrepancies due to the actual collapse of the walls can be verified, variations that can be shown in both section and elevation [8]. Due to the passage of time, architectural heritage is usually affected by pathologies such as cracks and collapses due especially to the movements of the building, the most common being in masonry and wooden structures. Another component is the integration of the damage detected in the heritage, which can be represented in the digital model itself. García-Gago et al. [9] structured the capabilities of the HBIM by integrating two essential features, geometric and material, and proposed the integration of building damage information into an information model of historic buildings.

From reverse engineering, damage and anomaly detection can be extracted in stones, bricks, and coatings, hence the importance of defining an ontological basis for describing and linking these entities. Thus, directly from a point cloud obtained by photogrammetry, a classification can be made with semantic annotations [10] in which images intervene as a vehicle of detection. In the field of crack analysis, in order to propose hierarchical restoration strategies, [11] proposed a training of the Mask R-CNN model and the classifier to train 270 images. These identification strategies are very common today due to the diversity of existing segmentation algorithms. The latest research [11] implements artificial intelligence in an intervention in architectural heritage, using 3D laser scanning technology to obtain disease details and crack data from masonry and wood walls; using a Mask R-CNN model to detect the crack area; and using an FCN model to identify and calculate individual cracks and, finally, to analyze the causes of the cracks and then propose corresponding hierarchical restoration strategies.

The study of stratigraphy in BIM models has also been a widespread trend in recent years. However, identifying stratification representation procedures in a three-dimensional digital environment is a sharing of [12] facilitating a reading of deformations and tonal changes in materials. The work of Banfi et al. [13] comes to the analysis of a wall of the church of St. Franceso simply by identifying pathologies of a model from an orthophoto, although without establishing identifying methodologies. A similar work, using the same database and images of the previous researchers, is [14], which implements stratigraphic unit parameters in Revit descriptors. In terms of implementation, [15] proposed a collaborative workflow in the HBIM project for an automatic classification of ashlars.

2.3. Monitoring the Historic Building

The HBIM project must be consolidated as a platform that integrates multiple sources of information, very valid for management and maintenance. The conservation of cultural heritage is often affected by changes in temperature and humidity within architectural spaces, so the energy performance and indoor microclimate of historic buildings require adaptation to new prevention studies. Under these premises, the latest research seeks to implement interdisciplinary processes between energy operators in a BIM environment, creating a new framework for the monitoring of energy parameters through sensors linked to the 3D model, fully interoperable and semantically enriched [16,17]. Other authors [18] focus their studies on the evaluation of the structural behavior of buildings that have suffered severe damage from earthquakes over the centuries, using a numerical, refined, and reliable 3D model that adopts updating techniques, sometimes based on artificial intelligence algorithms.

At this point, this research work poses two major challenges: (i) how to rescue the pathologies presented by a facade from massive data acquisition techniques such as photogrammetry and (ii) establishing the management and classification mechanisms of these characteristics inserted in the HBIM model.

3. Objectives

Based on the first theories formulated by the academic Dobby [19], in relation to the concept of architectural intervention, in a classification of “repair” (where there were no changes or total modification) and “restoration” (where changes did occur), the conservation process of a historic building is categorized in detail, in which there are seven degrees of intervention that are carried out at different scales and levels of intensity. Depending on the physical conditions, the causes of deterioration, and the prospective future environment [20], the following are established: (i) prevention of deterioration, (ii) preservation of the existing state, (iii) consolidation, (iv) restoration, (v) rehabilitation, (vi) reproduction, and (vii) reconstruction (also cited by [21]).

All these terms will mark hierarchical categories of intervention. Thus, in the history of the building, possible morphological evolution or transformations in the construction chronology are marked. But in the field of practice in the area of 3D digitization, how can historical recordings be performed on a BIM platform? This is the initial working hypothesis, to record each and every one of the stratigraphic sequences. Therefore, the objective of this work is, based on an adequate interdisciplinary knowledge of the building, to collect each intervention as a semantic unit in the BIM process parameterizing the elements from reverse engineering.

4. Case Study

The Palace of Miguel de Mañara is one of the most emblematic historical buildings of Sevillian architecture. Its original Mudejar-style structure was given a unitary formal configuration in the seventeenth century, consolidating the characteristic elements of the Sevillian Baroque House and incorporating Renaissance details, such as the aulic layout of the main courtyard, the prominence of the main staircase, and the incipient organization of the facade. Currently, the palace houses the institutional headquarters of the General Directorate of Cultural Assets of the Ministry of Culture and Environment of the Junta de Andalucía.

To understand the evolution of the building, it is essential to investigate the various stages of restoration that it has undergone over time. The remodeling of Seville on the occasion of the Universal Exhibition of 1992 prompted numerous interventions in the monumental heritage of the city. For the case study, at least three significant restorations have been recorded. This sequence of restorations has been crucial in preserving and highlighting the architectural and historical value of the Miguel de Mañara Palace, ensuring that its original details and authenticity endure over time.

4.1. Previous Restoration Projects

It is important to reflect the importance of the remodeling of the city of Seville in the years prior to the 1992 Exhibition, not because of the large number of heritage interventions but because of their interdisciplinary nature. As Diego Oliva Alonso and Isabel Santana Falcó emphasize, “the Charterhouse of Santa María de las Cuevas, the Palace of Altamira, and the Royal Monastery of San Clemente, together with the House of Miguel Mañara, all of them in Seville, are pioneering works in the way of understanding archaeological intervention, as research into the built, but they are also so in the way in which the knowledge of historians, archivists, heraldists or restorers of movable property is integrated with architects and archaeologists, together with pure scientists of different specialties” [22].

4.1.1. Restoration of the Year 1991

The rehabilitation of the Miguel de Mañara Palace (1991–1992), led by architect Fernando Villanueva Sandino until his death, was conceived as a multidisciplinary project involving archaeologists, art historians, and engineers. Villanueva emphasized that restoration is not mere reuse but a critical intervention balancing historical value with functional, social, and urban objectives. The project aimed to recover the palace’s 17th-century configuration, its period of greatest significance, through careful restoration of architectural elements such as marble, wood, and tiles, while adapting the space for its new use as the headquarters of the Directorate-General of Cultural Heritage of the Junta de Andalucía.

The intervention preserved the original layout, removed incongruent additions, reintroduced lost mezzanines, enhanced the main staircase’s design, and completed the courtyard’s colonnade based on 16th-century documentation. Restoration of the facade included exposing and consolidating 18th-century polychrome layers, removing later additions, salts, and organic residues. Villanueva underscored that design and execution were inseparable, relying on detailed preliminary studies to guide historically and structurally faithful solutions.

Villanueva was clear that “the development of the rehabilitation project is difficult to understand apart from its execution,” so it was based on studies and preliminary work, constituting these basic sources of the process of designing the solutions. This way, the rehabilitation project will immediately identify what the primary objectives are. The intervention on the facade included cleaning the layers of material until the original layers were exposed, where remains of polychromy were found. The general criterion was to recover the polychrome elements that characterized the building during the eighteenth century, discovering and consolidating 100% of the hidden paintings. The intervention program on the facade included the cleaning and consolidation of the original remains of paint and eliminating non-original layers, salt deposits, and organic elements, in addition to consolidating both the pictorial film and the mortars (Figure 1).

4.1.2. Restoration of the Year 1994

In 1994 a project was drawn up for the refurbishment of the interiors of the palace, directed by the architect José Luis Daroca Bruño. It was mainly aimed at the installation of a library for restricted use, located in the painting room on the north side of the courtyard on its ground floor. Therefore, this project does not affect the facade.

4.1.3. Restoration of the Year 2004

In 2004, a proposal for intervention on the facade of the palace was drawn up with the aim of reviewing and stabilizing the treatments initiated in previous phases, halting the ongoing deterioration of the materials. Additionally, it sought to unify the decorative elements of the facade through a systematic and rigorous reconstruction process, thereby achieving a cohesive appearance free from distorting agents (Figure 2).

During the 2005 restoration, the facade was found to consist of a masonry wall reinforced with stonework at the corners, pilasters, and lower sections, shaped into buttresses to frame the brick facing and support the rammed earth areas. Lime-based mortars mixed with sand and straw were used, clearly distinguishing the two coating layers: the arriccio (rough base layer) and the intonaco (thin finish layer). Material losses were attributed to erosion caused by poor detachment techniques in previous interventions, which left tool marks along the edges. These affected the pictorial intonaco layer and penetrated several millimeters into the arriccio. This type of damage was widespread across the surface. No major cracks were observed, except for one on the roof of the oculus. However, fine fissures in the intonaco were detected, likely caused by shrinkage during drying. Moisture-related damage included plaster carbonation and staining, resulting in darkening and pronounced yellowing of the mortar. Following restoration, the entire facade displayed a rendered finish that preserved the articulation of the individual bricks. Nevertheless, certain paneled areas on the first floor lacked vertical discontinuities in the brick courses (Figure 3).

5. Methodology

The new methodology that was carried out included the historical and evolutionary study of the palace and the inclusion in an HBIM project of all the processes of geometric characterization, semantic identification, and insertion of study components of the building’s diagnosis (Figure 4).

5.1. Historical-Evolutionary Study

From the point of view of restoration experience, the processes of action must follow one another in a progression by progressive and complementary stages, starting with obtaining all valid preliminary information, both from historical documents and from previous restoration projects. In the case of the facade of the Miguel de Mañara Palace, it was very important to collect data on the interventions carried out in the building, including the facade, to make a cataloguing of historical photographs and to know the scope of all the interventions. The insertion of all these data in a BIM model was essential to know the historical-evolutionary value of the building.

5.2. Data Acquisition System

The first phase involved the precise capture of the geometry of the exterior enclosure of the Palace of Miguel de Mañara, a building whose facade has a length of 43.30 m and a height of 6.75 m. This case study is relevant due to the geometric particularities of the facade, which has an inclination of approximately 30° in the first quarter of its length and a considerable collapse in verticality in the last quarter. These characteristics represent a challenge for the applicability of data acquisition techniques as they can cause false recordings and occlusions when short-range photogrammetry is used.

For this specific case, a combination of capture technologies was chosen instead of a single sensor. In this sense, personal laser scanning (PLS) [23] was used, complemented by photogrammetry using unmanned aerial vehicles (UAVs) and a terrestrial reflex camera. As described in [24], this methodology provided complete point coverage over the various elements of the facade. The PLS Leica BLK360 was used in the geomatics surveys, which, thanks to its low weight and scanning capacity of 360,000 dots per second, facilitated detailed capture of the exterior architecture. In addition, the BLK360 was equipped with three HDR cameras with color sensors and an infrared thermal camera, which expands its data-logging capability.

To cover the highest areas of the facade, since the scanner operated from a 1 m high tripod, the PLS was supported with aerial photogrammetry using a 0.6 kg DJI Air2S drone. In a second phase, the photogrammetry capture was complemented with images captured by a Nikon D80 SLR camera, in two rows, one perpendicular and inclined 15° to the facade (focal length of 30/35 mm and distance to the facade of 4 m) and another inclined at 30° (focal length of 27/35 mm and distance to the facade of 7 m, conditioned by the width of the street), thus covering the entire height of the building and covering each section along Calle Levíes. These technologies interacted to cover possible occlusions in the registers, which required a precise fusion of both sets of aligned points.

The alignment between the data obtained from PLS and UAV presented certain complexities, primarily due to the integration of terrestrial and aerial datasets and the requirement for a precise geospatial database as a reference framework for historic building information modeling. To generate an optimal orthophoto for the study, 72 photographs were taken at 4 m and another 36 at 7 m away from the facade plane, ensuring an 80% overlap valid for photogrammetric processing in Agisoft Metashape V.1.8. The dense point cloud generated within a 3D coordinate system was of significant importance for producing a model that faithfully reflected the geometric particularities of the facade. Furthermore, the acquisition of a 3D polygonal mesh was essential to accurately integrate the surface into the HBIM project. As for the orthophoto generation, images were processed by facade sectors according to structural discontinuities, allowing for the creation of distinct frontal views of each section. These were subsequently used to map the exterior face of the parametric walls (Figure 5).

The HBIM project must be properly structured, with PLS and UAV data being a source of information to determine the dimensional characteristics of the structures (length, height, and volume) and the specific geometric elements of the architecture (plinths, skewers, moldings, cornices, breaks in walls, and variations in the usual elevations), and must use a high-precision GPS to reference the control points on the ground and structure the project appropriately. The data interoperability between the post-processing software (Leica Cyclone 3DR 2023.1) and the BIM platform (Archicad 27) must become a solid method for the subsequent modeling processes in the HBIM project. It will be within the HBIM project where the segmented fractions of the point cloud will be managed to proceed with their semantic enrichment.

Based on previous contrasted cases [4], the management of the point clouds began in Cyclone 3DR (C3DR), establishing the origin of the project (0,0,0) at the bottom, end, and left point of the facade. The point cloud of the facade was isolated from the adjoining buildings and exterior pavements, dividing into the doorway, plinth, and facade. All portions were also georeferenced to the same source, to then be exported in e57 format; they were then imported into the HBIM project, all the fractions being properly coupled in automatic mode.

The reference levels were defined in the BIM environment based on the PLS point clouds: the street pavement set the zero level, the threshold of the access door set the ground floor level, the balconies established the level of the first floor, and the upper face of the last cornice established the roof level. The ordinate axes were made to coincide with the structural projections (vertical imposts) of the facade, information provided by the facade point cloud (Figure 6).

5.3. Phases of Intervention of the Property

Another important component is integrating detected damage into the digital representation of heritage assets. García-Gago et al. [9] structured the capabilities of HBIM by incorporating two essential aspects: geometry and materiality. They also proposed the inclusion of building damage information within the information model of historic structures.

In line with this approach, a systematic methodology was proposed that integrated the geometric, textural, segmentation, and semantic components of the model, along with the semantic implementation of the facade analysis. Therefore, the proposed methodology encompassed the entire process from geometric characterization and semantic identification to inserting diagnostic study components related to the building’s condition.

Construction Techniques and Prior Analysis of Materials

Conservators dedicated to the conservation of architectural heritage resort to such experienced techniques as mechanical-acoustic auscultation, which consists of hitting the surfaces in question to perceive the sound of the vibration of the coatings [25]. But the traditional way of perceiving and detecting problems in coatings is established by (i) visual inspection, (ii) infrared thermography, (iii) ultrasonic tests, and (iv) physical analysis of materials. LiDAR scanning and short-range photogrammetry can help these techniques to know the degree to which objects are altered.

Although techniques such as thermography [26] are highly effective in detecting humidity problems, it is also possible to use photogrammetry for the automatic recognition and classification of cracks [27], both terrestrial (PLS) and aerial (UAV), to observe color changes and detect pathologies in the walls. This allows an exhaustive control of the characteristics and pathologies of the building. Issues related to the durability of coatings are a crucial factor, as they determine the properties of materials and their suitability for a specific physical environment. For this reason, there are many studies that continue to use different approaches for thematic texturing or mapping according to specific themes of intervention [28], although this type of work usually develops the original maps from drawings outlined in CAD on orthophotos, as shown in the 2D elevations of the intervention project of the facade of the Palace of Miguel de Mañara in 2004 (Figure 7).

The open research has taken these documents as a review, introducing the CAD planimetry as “Worksheets” in ArchiCAD v.28 [29] on the BIM platform. The system works as work templates, where the operator takes them as references, when they are active and visible, and thus assumes measurements, surfaces, and shapes rescued from these old documents (Figure 8).

An additional step consisted of the comparison of the CAD planimetry with the point cloud obtained by laser scanning. This procedure made it possible to identify discrepancies between the surveys carried out by the original operator, who outlined the plans based on measurements taken in situ, and the data provided by the new technology. All deviations could be analyzed directly from the BIM software itself. In this specific case, a considerable variation (+1.02 m) was identified, which represented ±0.02% of the effective deviations.

Likewise, considering the objective of the research—to develop the characterization of materials and the detection of pathologies—the HBIM project sought to promote an exhaustive analysis of the point cloud. These point clouds, obtained through the two procedures described, were subsequently integrated into the model using the methods already detailed by Moyano et al. [3].

Direct analysis on the parametric elements of the BIM model, as shown in Figure 9, was performed by mapping the original images. However, since these techniques employed diverse procedures, especially in the post-processing of the data, it was necessary to readjust them so that they were complementary and allowed the effective 3D reconstruction of the facade geometries.

Finally, an effective methodology must be developed that includes the analysis of the physical properties of the materials used, such as bricks, masonry mortars, and wall cladding. This information will be extracted directly from the set of points obtained using advanced data acquisition techniques. Based on the geometric identification and analysis of the data provided by the historical documents of the successive interventions, the aim is to implement an experimental technique that facilitates the performance of automatic and interdisciplinary operations for the monitoring of the alterations of the facade. All this will be carried out within the HBIM project itself, with the aim of guaranteeing continuous and multidisciplinary control in the conservation processes.

This process must be well structured to ensure its effectiveness, which requires both structured and semantic segmentation of point clouds. This segmentation will subsequently allow a precise characterization of the defects present in the materials and construction systems involved.

The study of alterations is closely related to the properties offered by point clouds (PLS), UAVs (unmanned aerial vehicles), and high-resolution metric photography. These techniques make it possible to identify particularities, both atmospheric and biological, that can influence materials, especially stone [30], as well as other construction components.

Bruno et al. [31] stressed the importance of evaluating and entering data from visual inspections and diagnostics in historic buildings, transferring this information to the HBIM model. Recent research focuses on developing precise identification processes that allow quantifying the physical alterations that affect traditional masonry. These advancements include user-friendly prototyping, facilitating the agile implementation of effective procedures for monitoring cracks, material losses, and other structural damage [11,22].

5.4. Segmentation of the Geometry of the Historic Building

There is unanimous agreement on the geometric precision provided by laser scanning techniques and photogrammetry, to be a reference both for a faithful modeling of architectural geometry and for the auscultation of materiality within the processes of documentation and preservation of architectural heritage. But arriving at a faithful model of an architecture of complex geometries is a task that involves many hours of work, without achieving a totally identical copy. Therefore, the three-dimensional representation of deformations, peculiarities, or irregular geometries must be carried out outside the parametric process characteristic of a BIM environment. The tools available in BIM software are suitable for the modeling of contemporary architectures, with pure geometries that can be parameterized, a very different scenario from that of Heritage (PA³). In addition, there are no typologies within the libraries of parametric objects that could represent the great diversity of geometries that have arisen in the history of architecture. To reduce this gap, it has been intended to develop a methodology that investigates new semantic segmentation procedures, involving the studies in an efficient scan to mesh process, that is, the segmentation into characteristic planes and geometries of a surface previously obtained by a transformation of the points into a triangulated mesh. The problem with approaching a complex geometry is that the surfaces are generated with a large number of polygons that slow down the work a lot as the computational process increases, which makes it difficult for the BIM project to operate. The solution is to reduce the number of polygons as much as possible without losing important details of the geometry of the sampled element. Another way is to perform a conversion to NURBS (non-uniform rational-splines) surfaces before being imported into the HBIM project, where the object will then be semantically enriched [32].

For the case study, the methodological process has been aided by SfM photogrammetry, derived from UAV surveys and the terrestrial manual camera, providing a geometry of the building envelope and in the form of a meshed and textured surface as faithful as possible to the facade of the Palace of Miguel de Mañara.

5.5. The HBIM Project Interoperable Workflow

For an adequate interdisciplinary management, it is necessary that the geometric data (point cloud and polygonal meshes) be taken to a single container, with the HBIM project constituting that appropriate nucleus to interoperate all the data. We must always safeguard a flexible and efficient workflow, so that all the information provided by the various techniques (PLS and UAV) must be well coupled and georeferenced to the origin of the HBIM project. In [24] the procedure carried out in the PLS and UAV alignments was presented so that the data, both photographic and from the scans, can be used in an alternative and complementary way from a common HBIM environment. For the case presented, the post-processing of the surveys carried out by the Leica BLK360 scanner was very effective and accurate, with the project having a large amount of metric data of the facade of the palace and the closest environment of the San Bartolomé neighborhood of the city of Seville.

5.6. The HBIM Project in the Historical-Conservative Process of the Palace Facade

The integration of paramental study based on an advanced stratigraphic-evolutionary analysis that is aided by geomatic techniques and geospatial technology is still in a slow process of implementation in the disciplines surrounding the study, intervention, and conservation of PA+3. In the case study of the former prison of the Tobacco Factory of Seville, now the headquarters of the University of Seville [33], a stratigraphic analysis of walls in an HBIM model was already presented, where the real ortho-image of the provided tasting served as a basis for the identification of EU stratigraphic units and the introduction of characteristic properties of their historical-temporal evolution, materiality, and construction techniques used. More recent studies on semantic discretization, with the aim of establishing the identification and classification of construction systems and their parts, typical of a historic building, have allowed an automatic classification of ashlars using Python 3.11 programming [34]. The aim of both studies is to bring the synergy of building information modeling (BIM) closer to the peculiarities of the historic building, such as the identification of processes dated to different periods and typification of deterioration and injury, as well as to help the efficient management of heritage in phases of intervention and conservation.

5.6.1. Restoration of Facade Cladding

In a process of paramental reading of the facade of the palace, two phases could be seen because this was intended to be evidenced in the restoration of the years 1991 and 1992. This clear chromatic differentiation was due to the proposals for the reintegration and homogenization of the surface in which it was intended to satisfactorily articulate the faults in the composition, respecting and distinguishing at all times the original remains, understanding them as more historical elements, and at the same time trying to interrelate the two pictorial spaces. The darkest area alluded to the original facade, rediscovered during that intervention (Figure 2). The second area, with a lighter color, had as its main objective the rescue of the chromatic areas of the facade, which were hidden in their entirety. The general criterion of intervention on the facade was to recover the polychrome spaces that made the building unique throughout the eighteenth century, consisting of the discovery of the paintings 100% hidden, the intentions of consolidation, and the necessary cleaning and consolidation work of the original remains of the painting. Specifically, the layers foreign to the original, micro-organic elements and saline deposits were eliminated, in addition to the consolidation of pictorial film and that of mortars. The results after restoration can be seen in Figure 3.

As for the mural paintings on the facade, of approximately 500 m2, the restoration work of the cladding on the facade was aimed at recovering some chromatic spaces made in the sixteenth century [22].

5.6.2. Identification and Classification of Systems and Pathologies

The review of the successive restoration projects of the palace, mainly in the last forty years, has provided data on the structures of the facade, the important problems encountered in the auscultations carried out in the 80s (twentieth century), and the procedures carried out in the progressive interventions, which mainly affected the cladding, until the current architectural conformation of the facade. In this research, a new auscultation was carried out to compare those areas, where fissures and detachments were now located in their finishes, with their states prior to the restoration processes, in addition to analyzing the products and techniques used then. In this way, an adequate diagnosis of their condition and the causes that have motivated the pathologies (decay) detected will be determined.

The procedure for modeling the facade wall and the introduction of semantic properties in the parametric elements was already exposed in the document [24]. The HBIM project had a BIM model faithful to the original stratigraphy of the facade, where the facade was structured using the TLS point cloud, coupling the geometry of the parametric walls. These were made up of multiple layers to differentiate the masonry core from the different layers of the cladding, distinguishing fresco paint on lime, fine stucco mortar, fine lime rendering mortar, coarse lime rendering mortar, and brick or rammed earth (wall core). For the interior face, a brick partition was established separated from the structural thickness by an unventilated chamber that supported the coatings of the intrados: the interior mortar layer, the trim and plaster, and the paint layer.

The BIM methodology was present in the standardization of the processes of the Heritage-BIM project, with a classification of all the construction entities, using a characteristic identifier (ID) with a view to effective cataloguing in the preservation processes or in future interventions. Subsequently, the elements were classified by their constructive function so that the semantic model could be interoperable by all the intervening agents, a semantic classification that was in force in the IFC format. The study model was classified first according to the Andalusian Construction Costs Bank (BCCA), with a correlation between the item ID and the heading extracted from the Bank. As a second classification, the SCFclass V2 standard [35] was used, which complemented the architectural-constructive classification, to get involved in a new, much more ambitious technological context in the management of all types of assets as this classification database was more complete in all areas [24], classifying by assets, companies, spaces, phases, functions, materials, subdivisions, and uses.

6. Results

6.1. Photogrammetric Analysis of Materiality

For the case study of the facade of the Palace of Miguel de Mañara, several processes of analysis of the materiality associated with the exterior appearance were followed, based mainly on a segmentation by classification of the characteristic colors of the point clouds, using the set of points obtained from image-based modeling (IBM). In the first phase, the Agisoft Metashape software was used, which had a color classifier based on the use of RGB images. Fourteen samples were pre-selected from the set of points on the facade from among the different most characteristic regions. The method, described in detail in [24], resulted in the identification and classification of those regions with more intense colors, which could be related to alterations due to humidity or temperature and that alter the chromatic appearance of coatings.

In a later phase, the cladding panels of the facade, restored in 2005, were discriminated, which were more deteriorated, especially with cracked areas and fissures of considerable length. Sectorizing a portion of the orthoimage obtained in the photogrammetric survey, an auscultation was carried out from an image processing software, such as Adobe Photoshop (APh). To help the process of analysis and measurements, several image filters were applied, many of them very effective as they allowed a perfect identification of the problems of cracking and detachment in the wall cladding. The following figure (Figure 10) shows the result when using the “bas-relief” filter (APh).

6.2. Point Cloud Classification

The segmentation and semantic classification of a set of architectural element record points (TLS, LIDAR, and UAV) is a process of exhaustive and precise analysis applying identification algorithms to then generate different regions with homogeneous properties and characteristics. Drzewiecki et al. [36] enunciated several methods for classifying and segmenting a point cloud: methods based on line segments, attributes, and definition of geometric shapes; those based on Hough’s 3D transform; and a method employing the region growth algorithm. In the end, the segmentation process resulted in establishing sets of points, usually of the TLS or SfM type, which must be labeled as materials, construction systems, surface gradients, or anomalies arising from biological problems.

Segmentation of the Facade

SfM photogrammetry, derived from UAV surveys and the hand-held ground camera, provided a well-textured metric surface of the facade (Figure 11a). The mesh was imported into the Leica Cyclone 3DR environment and coupled into the same reference system of the HBIM project. Then, the mesh was segmented by planes, establishing as base parameters a segmentation angle of 15°, an area threshold of 2 m², and a standard deviation threshold of 10 cm. Twenty-one plans of the section of the front facade to Calle Levíes were obtained (Figure 11b). The portions were colored for identification, although they could be shown by their actual colors, as shown in Figure 11c.

6.3. Organization of the HBIM Project

The floors were established as a reference of levels based on the point clouds from the TLS or photogrammetric survey and the use of a structure of axes of abscissas and ordinates that marked the core of the main structural elements and the substantial facts of the building (Figure 12). Thus, before starting with modeling using parametric objects, a governing matrix of the entities must already be available. For the virtual construction of the 3D model, basic construction typologies (wall, slab, and pillar, among others) were used, superimposing them on the clouds of reference points. This made it possible to contrast the real model with the theoretical one, carrying out an exhaustive analysis of the deformations in the horizontal and vertical walls of walls, cornices, and cantilevers.

Considering the limitations of resources, mainly the PC hardware, one should always look for the greatest operability and efficiency. Thus, for the case study, priority was given to the adequate identification and classification of pathologies in cladding. Therefore, it was not necessary to introduce all the mesh areas in the HBIM project, only those that showed evidence of damage and cracks in the facade. These pathologies were finally analyzed, classified, and controlled in the BIM environment.

6.4. The Collapse of the Walls of Facades

The point clouds themselves provided very valid geometric data to study the collapses in the facade walls, detecting important deviations from planes and evidencing the great collapse in several sections included in the PLS point cloud, as shown in the images in Figure 12. Several sections of the parametric wall that were introduced to model the theoretical facade of the HBIM model, represented with red-earth color, were detached from the cloud of reference points due to the real deformations of the two walls (Figure 12a). The deviation analysis was carried out directly with the comparison between the theoretical wall and the point cloud in each of the 175 given sections along the facade (separated by 25 cm). The maximum collapse was located in section S142, between the Y8Y9 axes, with a deviation of the wall in the direction of the public road of 23.7 cm (Figure 12c).

The data interoperability between the post-processing software (Leica Cyclone 3DR) and the HBIM project (supported by Archicad) is exposed in [24], constituting an effective method for the subsequent modeling processes in the HBIM project, where all fractions were classified in sets of specific points. The segmentation of the PLS point cloud was managed in the HBIM project as an object with its own entity, allowing it to be enriched with semantic data, introducing specific properties derived from the process of auscultation, dating, cataloguing, conservation, and restoration.

6.5. Pathological Study of the Facade

For the pathological study of the Palacio de Mañara, both the orthoimages of the facade and the mesh and textured surfaces of the segmented panels were used, defined as a unit flanked by the plinth, the horizontal cornice, and the two vertical extreme imposts. Subsequently, a stratigraphic segmentation was carried out by areas with superficial lesions, using the Morph tool. Its external face was mapped with the real image, showing the lesions in their correct positions. In the library of materials of the HBIM project, a group of specific coatings of the building under study was arranged. Various typologies were created, where each type was associated with the partial ortho-image of the facade wall. For its proper 3D representation, it was given a thickness between 5 and 10 mm. Fissures and cracks caused by structural movements were not of considerable opening, so representation by area was discarded. Simple vector marking using a prominent color pen (such as magenta) was sufficient.

All injuries were classified according to the SCFclass V2 standard, assigning each one a subclass within the category functions: “Auscultation and > Tests General damage > Mechanical origin > Crack” in masonry elements and “Auscultation and tests > Damage > Surface alterations > Crack” in coatings (

Table 1. List of subclasses of the Spanish standard SCFclass V2, within the category functions: “Auscultation and tests > Damage > Mechanical origin > Crack in the factory” and “Auscultation and tests > Damage > Surface alterations > Crack in coating”.

FUN	AUE			FUN.AUE	Auscultation and Rehearsals		Entidad IFC	Tipo IFC
FUN	AUE	010		FUN.AUE.010	Sensors		No aplica
FUN	AUE	010	010	FUN.AUE.010.010	Crack meter	ud	IfcSensor	MOVEMENTSENSOR
FUN	AUE	010	020	FUN.AUE.010.020	Temperature sensor	ud	IfcSensor	TEMPERATURESENSOR
FUN	AUE	010	030	FUN.AUE.010.030	Accelerometer	ud	IfcSensor	USERDEFINED
FUN	AUE	010	040	FUN.AUE.010.040	Displacement sensor	ud	IfcSensor	MOVEMENTSENSOR
FUN	AUE	010	050	FUN.AUE.010.050	Strain gauge	ud	IfcSensor	MOVEMENTSENSOR
FUN	AUE	010	060	FUN.AUE.010.060	Extensometer	ud	IfcSensor	MOVEMENTSENSOR
FUN	AUE	010	070	FUN.AUE.010.070	Fleximeter	ud	IfcSensor	MOVEMENTSENSOR
FUN	AUE	020		FUN.AUE.020	Rehearsals
FUN	AUE	020	010	FUN.AUE.020.010	Sclerometry	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	020	020	FUN.AUE.020.020	Ultrasound	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	020	030	FUN.AUE.020.030	Witness	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	020	040	FUN.AUE.020.040	Tasting	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030		FUN.AUE.030	General damages
FUN	AUE	030	010	FUN.AUE.030.010	Deterioration of finish	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	020	FUN.AUE.030.020	Scour	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	030	FUN.AUE.030.030	Bulging	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	040	FUN.AUE.030.040	Detachment	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	050	FUN.AUE.030.050	Settlement	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	060	FUN.AUE.030.060	Collapse	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	070	FUN.AUE.030.070	Erosion	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	080	FUN.AUE.030.080	Excessive deformation	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	030	090	FUN.AUE.030.090	Drain element damage	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040		FUN.AUE.040	Damage to concrete and masonry elements
FUN	AUE	040	010	FUN.AUE.040.010	Breakage	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	020	FUN.AUE.040.020	Crack	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	030	FUN.AUE.040.030	Fissure	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	040	FUN.AUE.040.040	Chemical damage in element Co/Ma.	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	050	FUN.AUE.040.050	Efflorescence	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	060	FUN.AUE.040.060	Humidity	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	070	FUN.AUE.040.070	Vegetation	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	080	FUN.AUE.040.080	Coke and gravel nest	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	090	FUN.AUE.040.090	Wound washing	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	100	FUN.AUE.040.100	Damage to masonry or ashlar parts	ud	IfcBuildingElementProxy	USERDEFINED
FUN	AUE	040	110	FUN.AUE.040.110	Damage to concrete reinforcement	ud	IfcBuildingElementProxy	USERDEFINED

).

If one is committed to implementing a more sustainable intervention project, based on an interoperable and efficient digital information model, where very specific elements of a restoration project can be implemented, the participation of BIM operators must be coordinated with specialists in the conservation of architectural heritage. The collaborative and multidisciplinary environment of the BIM methodology is a bulwark to advance in the effectiveness of PA³ heritage conservation projects, taking a new step and overcoming past difficulties. Hence, researchers and restoration professionals [37] understand the spatial and functional configuration, the result of a process of continuous evolution that includes social, economic, and cultural aspects.

The HBIM project incorporated the database of the standards established by the institution, region, or country, which made the semantic classification tasks more flexible. Figure 13 and Figure 14 show the direct classification in the classifications and properties section, with the direct mapping in a drop-down data structure.

For the auscultated areas, the marks were enriched with semantic data, with the creation of relationships between the damaged entities and a base of intrinsic properties of the HBIM project, so that the data were endowed with authenticity. In addition, the procedure allowed a correlation between semantic data and the model, helping to maintain consistency between the data. In this way, the marked sector of the image indicated the following: “Area covered with deteriorated lime mortar, with high cracking of the plaster. Possible causes: temperature changes and improper hydraulic setting; improper mixtures in mortars; structural movements. Diagnosis: Vertical cracks are the result of a cause inside the wall, which may be a pipe or another element that causes tensions, and which should be analyzed with a subsequent tasting. The rest of the generalized cracks are hygrothermal, also considering that other organic materials have been added to the lime mortar, which reduces plasticity. The loss of color in the fake brick masonry is due to the application of a very thin layer that, although it may contain organic elements, decreases durability” (Figure 15).

7. Discussion

The preservation of architectural heritage entails continuous control of the structures and cladding of the historic building, which, in the case presented, led to the drafting and execution of successive intervention and restoration projects. The problem arose when the rich information reviewed, analyzed, and newly generated in each phase was isolated. Therefore, the data of each project were not properly recorded and then contrasted, which led to the loss of important data and the carrying out of repeated research work, which could be called preliminary studies, new metric surveys, and repeated pathological studies that fail to consider relevant previous findings.

Starting from the basis that every restoration project needs historical enrichment, it is essential to investigate, compile, and reflect on all the past actions and interventions carried out up to the present. In the case presented, these preliminary studies served to efficiently analyze and contrast the graphic documents, reports, and technical reports of the intervention projects with the new information from the PLS surveys and photogrammetry. The HBIM project of the Palace of Miguel de Mañara was enriched with all the information from the previous restorations, to faithfully represent the structural systems, differentiating the successive layers of cladding and their metamorphosis over time. And referencing the point cloud of the scanned facade with CAD vector elevations helped to update and verify the geometric information. The fact of contrasting plans with elevations of different interventions served to detect discrepancies in dimensions, mainly in the length of the facade, as well as in the delimitation of the areas marked with subsequent deterioration. The methodological processes expressed in the research work were supported by data provided by effective techniques that acquired the real materiality of the envelope of the historic building, allowing a faithful 3D digitization and the subsequent control of deformations and alterations. This monitoring system provided efficiency in the auscultation of canvases that embellished the architectural heritage and reinforced conservation processes. Therefore, its implementation in the PA3 sector was very necessary for interdisciplinary management over time.

The study has contributed to capturing recent and precise data that are very valuable for detecting pathologies in a building that has undergone a long transformation over time. A follow-up was carried out in the HBIM project allowing a three-dimensional analysis of the segmented panels of the facade. First, the point cloud itself was contrasted with the theoretical wall, allowing discrepancies of up to 27 cm to be detected, in a section of the facade that provided the maximum collapse toward the outside. These differential data with respect to a theoretical model were very revealing and of great importance for structural analyses, where experts will determine the causes: attributable to the passage of time or due to differential settlements of the building. Then, a superficial meshing of certain areas of the facade was made, which made it possible to represent the faithful geometry and insert it into the HBIM project. This reverse engineering process avoided the lengthy manual processes of parametric modeling that result in construction elements and systems that will never be equated with the complex reality of heritage.

Subsequently, the conservation processes continued, experimenting with the pathological study of a sector of the facade of the Palace of Miguel de Mañara. The orthoimages of the segmented panels of the facade facilitated stratigraphic auscultation, identifying superficial lesions and then linking them to a parametric element such as Morph; the superficial deterioration that affected a specific area of the cladding was sectorized; and, through a vector color marking, linear lesions were highlighted: fissures and cracks. Finally, all detected lesions were classified by various standards (SCFclass V2 and others).

Finally, the data provided by the technical report, derived from the auscultation and tests carried out, were introduced as semantic properties in the entity of the HBIM model. The specialist, after his inspection, embedded the specific data as an intrinsic parameter by accessing the interactive window and marking the correct cause since these were already pre-established in the project database; since very specific relevant pathologies of the case study were detected, new properties were also created to be introduced into the HBIM conservation project database.

Methodological Advances in Conservation

This case study aimed to provide new advances in the methodology to shape efficient restoration and conservation projects, which justified that we continue researching the HBIM topic, enriching previous work and solving challenges. An intervention in a historic building requires the participation of many disciplines and the appropriate multidisciplinary management of data, to which must be added the continuous maintenance of the heritage. The HBIM project was very valid for structuring the interdisciplinary information that flowed from restoration processes, providing, more than benefits, efficiency. The presented case can be extended to many others included in the PA³ environment, serving as a guide for interventions. To do this, certain guidelines should be taken into consideration.

All historical information must be recorded and compared in a common database, classified by progressive phases in time. The HBIM project must be linked to this database, with a database management system (DBMS) that allows data to be stored and then accessed in a structured and fast way. The DBMS would act as a single source of reliable and permanently updated information. In this way, effective communication and coordination is promoted, with the advance participation of the different agents involved in the conservation life cycle, facilitating, from the beginning, the consideration of all the conditioning factors and consensual decision-making.

When monitoring the building, the BIM environment will allow updated 3D visualization (for example, on the new scans carried out), making it easier to identify pathologies and risk conditions in the PA³ in the model. This will allow more effective decisions to be made in the intervention (between the different restoration alternatives) and in the subsequent conservation.

The technical intervention systems are diverse and evolving, so it is important to record the products and techniques used, for each sector, together with the results obtained after restoration. The HBIM project must be the container of the data (graphic and semantic) assorted from the successive phases. In this way, the mutations that arise later, in structures and coatings, can be properly analyzed, helping the diagnosis and providing an adequate solution to the problem.

8. Conclusions

The results of the research built a methodological process that validated a system for detecting pathological cracks in the architectural heritage of masonry to support the repair work. We worked in an interdisciplinary collaborative environment, such as the HBIM Project, enriched with precise data and validated by the specialist, mainly to provide contrasted reports on the causes that motivated the fissures, cracks, detachment of coatings, and water filtration in facade walls. However, the methodology can also be widely accepted in pathologies detected in horizontal walls, such as interior floors, slabs, and roofs.

The scan to HBIM process implemented in the heritage conservation project contrasts the various techniques available in the sector, analyzing their strengths and weaknesses, to adapt them favorably to the case study. The analysis and tests carried out on the various options by the researchers have finally made it possible to form an efficient methodology that allows the use of different technologies, such as UAV photogrammetry and PLS scanning, which complement each other, in order to sustain and strengthen the HBIM conservation project. These processes are very valid to be applied in the information model of a historic building and to monitor the conservation processes. In this way, the data that flow over time are auscultated, compared, and analyzed, helping to make more accurate and accurate decisions. Without forgetting that the parametric model of the HBIM project is updated semantically and accurately, so the effectiveness and solvency of the data is always guaranteed, establishing an effective and active project for conservation and restoration work.