Detection of Moisture and Surface Wear in Sillar Heritage Structures Using Deep Learning in Arequipa’s Architectural Heritage

Abstract

This study aims to detect pathologies in constructions made of sillar, a volcanic material of great historical and cultural value, commonly used in residential and heritage buildings, in the city of Arequipa, Peru. Due to the uniqueness of sillar and the particular characteristics of its pathologies, such as moisture and surface wear, a non-invasive methodology using digital images is proposed, oriented toward the analysis of heritage constructions, with the objective of developing a method that does not alter or modify the heritage or damage the structure, considering that in invasive studies, sample collection may affect the integrity of the material. The proposed strategy combines computer vision techniques, including clustering methods for preliminary segmentation, with the use of deep neural networks for anomaly and deterioration detection. Furthermore, a validation scheme is introduced that integrates standard segmentation metrics with intersection analysis relative to pathology maps, allowing computational analysis to align more closely with the criteria employed in architectural conservation. The results demonstrate good performance in moisture detection, although with lower accuracy in identifying other types of deterioration, highlighting both the feasibility and the challenges of applying deep learning to sillar diagnostics and laying the groundwork for the development of digital tools that support the documentation and preservation of architectural heritage.

1. Introduction

Arequipa is located at an elevation ranging from 2301 to 3500 m above sea level as it rises from the coast to the plateau. The Chili River and its micro-basins transform a strip of the steppe mountain range surrounded by volcanoes [1]. Furthermore, Arequipa is one of the most important tourist destinations in the Southern Peruvian Andes. The capital of this region, Arequipa has an important architectural heritage from colonial times, comprising churches, convents, and other beautiful buildings, such as the Basilica Cathedral of Arequipa and Santa Catalina Convent [2]. Consequently, heritage conservation and tourism are important activities, especially in its historic center. We look after our heritage because we recognize its value in our society and we want to retain places’ cultural significance [3]. In this context, Arequipa, which received the prestiguious title of UNESCO World Heritage Site in 2001, stands as a symbol of architectural richness, blending European and native influences. This recognition underscores the significance of the “mestizo” architecture within the city’s historic center. This fusion of styles and cultures has been considered important since the Athens Charter of 1931, emphasizing the need to preserve this cultural legacy [4]. Sillar is the principal construction material of traditional Arequipeñan architecture, present in manor houses, churches, portals, fountains, domes, and other structures that represent the most authentic expression of architecture of volcanic origin. The chemical components of sillar are vitreous, crystalline, and lithic in nature potassium feldspar, oligoclase, quartz, glass, biotite, and iron oxide with accessory materials such as pumice, andesite, and lapillitic slag. Its texture is porous and it absorbs liquids without losing cohesion; it withstands heat above 500 °C, is a poor conductor of temperature, and can be found in white, pink, and cream colors [5]. In the case of historical buildings, cathedrals, and archaeological monuments, problems arise when artistic stonework, bas-reliefs, and sculptures become disfigured or disappear due to atmospheric pollution or other causes. This occurs because, when estimating costs, there is a risk of underestimation, as these works are irreplaceable. In the past, the expenses related to the corrosion of buildings and structures exposed to the elements, including damage to ancient monuments and buildings of cultural and historical interest, had been assessed solely in terms of maintenance work. Clearly, the estimates obtained were not entirely accurate [6].

As the value of safeguarding humanity’s historical, cultural, and artistic legacy gains widespread recognition, non-invasive inspection (NII) techniques are attracting increasing attention for their ability to assess and monitor heritage materials without causing harm. Compared to terms such as “non-destructive evaluation (NDE), non-destructive testing (NDT), and non-destructive inspection (NDI)”, non-invasive inspection (NII) refers specifically to techniques that avoid both physical contact and any form of alteration, even minimal or reversible. This stricter definition is particularly relevant in the context of cultural heritage, where the preservation of material integrity is paramount. Nevertheless, NII does not aim to replace the broader set of NDT techniques, many of which are still extensively and effectively used in heritage science. The choice between NII and other NDT approaches depends on various factors, including the nature of the object, conservation priorities, risk assessment, and available resources [7].

This reality highlights the fundamental need for diagnostic methods that respect the ultimate value and fragility of cultural heritage. According to the Criteria for the Conservation of 20th Century Cultural Heritage (ICOMOS, 2017), “As much as necessary and as little as possible shall be done. Any intervention must be cautious. The scope and depth of any intervention must be minimized” [8], emphasizing the principle of minimal intervention in any restoration or conservation process.

In structural health monitoring and condition assessment, deterioration detection and forecast are fundamental aspects. Civil infrastructure, including buildings, bridges, power stations, dams, and so on, are essential parts of society, and most of these structures are made of concrete. However, these concrete structures deteriorate progressively due to the influence of extreme weather conditions, overloading, and lack of periodic maintenance [9].

This persistent concern regarding structural deterioration is not limited to modern concrete infrastructure; historical buildings, conversely, present unique challenges related to the interaction between their ancient building materials and environmental factors. Due to the type of enduring walls in historical buildings, which are mostly constructed with “cantera” stone (volcanic tuff), they absorb water coming from the ground through capillary action, evaporating it through their walls towards the exterior, dragging particles from the structure without affecting it; however, when an impermeable barrier (mortars or coatings) prevents evaporation, this causes damage to the surface [10].

The building and construction industry is slowly but constantly evolving, embracing new technologies such as Digital Twins (DTs), Building Information Modeling (BIM), Artificial Intelligence (AI), Internet of Things (IoT), and Smart Vision (SV) to further enhance the efficiency, productivity, accuracy, and safety of built environments. Industry 4.0, or the fourth industrial revolution, refers to the transformation of traditional industry practices and manufacturing methods into autonomous smart systems using state-of-the-art digital technologies [11]. This is even relevant for an historic building characterized by geometrical and morphological complexity and huge extension, or at risk of collapse. To address these issues, real-time detection of building pathologies from on-site images can support experts in the process of building diagnostics. In this regard, advancements and innovative findings in computer vision (CV) represent a beneficial opportunity in decay recognition, related to several operative fields (road, bridge, or building inspection). CV comprises digital image processing methods in which manually annotated features are used to create rules to recognize selected features. In particular, these methods have been developed to automatically identify features/objects within a set of images [12].

Recent advances in computer vision, particularly convolutional neural networks (CNNs) and self-attention mechanisms, have revolutionized the ability to analyze buildings or cities from images. These technologies have been successfully applied to various aspects of heritage protection, including façade reconstruction, crack and deterioration detection, and mural segmentation [13].

Over time, Artificial Intelligence (AI) has gained great relevance in various fields of knowledge, such as agriculture, medicine, and others. In this context, the objective of this article is to propose a methodology for the detection of the most common pathologies in heritage buildings through the analysis of digital images. Below, we list some of the articles reviewed during our research on pathologies in sillar, their detection, and detection methods, which provide different perspectives from which to address the problem.

1.1. Studies on Pathologies

First, in ref. [14], the authors analyze how atmospheric pollution contributes to the formation of black crusts on historic stone monuments and how these accelerate their deterioration. Through a detailed chemical and mineralogical characterization of the crusts collected at different sites, the authors identified compounds such as sulfates, nitrates, and metallic particles derived from urban pollution, which combine with the minerals of the original rock, promoting disintegration processes, material loss, and chromatic changes on the surface. The study demonstrates that black crusts are not only an aesthetic problem but also trigger internal alteration processes in the ashlar by modifying its mineralogical structure, which in the long term compromises the stability and conservation of heritage.

In ref. [15], an efficient process is proposed for managing deterioration and conserving architectural heritage through the integration of HBIM (Heritage Building Information Modeling) methodologies, applied to the case of the Duomo of Molfetta in Italy. The authors employed photogrammetry and laser scanning surveys to generate high-precision three-dimensional models, which were linked to a database documenting the different types of pathologies observed in the ashlars, such as cracks, surface disintegration, erosion, and material loss. This approach made it possible to spatially map the damages and associate them with technical information on materials, previous interventions, and possible treatments.

In ref. [16], the use of terrestrial laser scanning (TLS) is proposed to detect and quantify the weathering of stone monuments over very short periods. The authors carried out two scanning campaigns in different seasons of the year, and by comparing the obtained samples, they were able to identify material losses in the millimeter range, demonstrating that degradation can be significant even over short time spans. The study shows how factors such as lithology, wall orientation, and surface roughness directly influence the degree of erosion, with more porous stones and the most exposed facades being the most affected.

1.2. Conservation Standards

The law in ref. [17] establishes national policies and a legal regime for the defense, protection, promotion, ownership, and purpose of the assets that constitute the Cultural Heritage of the Nation. Any manifestation of human activity, whether tangible or intangible, is considered part of the Cultural Heritage of the Nation when, due to its paleontological, archaeological, architectural, historical, artistic, military, social, anthropological, traditional, religious, ethnological, scientific, technological, or intellectual importance, value, and meaning, it is expressly declared as such or is legally presumed to be so. These assets may be of public or private ownership, subject to the limitations established by this law.

The purpose of this regulation in ref. [18] is to establish the rules for the identification, registration, inventory, declaration, defense, protection, promotion, restoration, research, conservation, enhancement, dissemination, and restitution, as well as the ownership and legal regime, of the assets that constitute the Cultural Heritage of the Nation, in accordance with the norms and principles established in Law No. 28296—General Law of the Cultural Heritage of the Nation.

1.3. Non-Destructive Testing Studies in Cultural Heritage

According to ref. [19], damage in historical constructions is mainly caused by moisture. Therefore, it is essential to develop a diagnostic protocol to identify the presence of water, assess the damage and vulnerability of the structure, and finally, establish a restoration plan. Furthermore, the work proposes analyzing the robustness of the Infrared Thermography (IRT) method for the quantitative estimation of moisture content and the rising damp process in cultural heritage structures, starting with a basic laboratory test. The main results obtained showed that the analysis protocol established with the IRT method produced good results, demonstrating that the technique can be automated to standardize on-site diagnosis. The cameras used were suitable for transforming qualitative data into quantitative data, and it was concluded that guidelines should be developed for a standard IRT analysis in moisture assessment.

In [20], the author presents a study that treats an individual heritage object as a primary source for Construction Archaeology. Infrared Thermography (IRT) is established as a highly versatile non-destructive testing (NDT) method, aiming at evaluating its scientific applicability. Specifically, the methodology was implemented using a portable infrared camera as a standalone radiometric unit to examine a masonry sample, performing a qualitative thermographic analysis focused on the general thermal pattern (not on quantified values). This qualitative approach proved crucial for visualizing contiguities, joints, and constructive discontinuities, and it is recommended as a standard NDT measurement procedure that facilitates comprehensive examinations and standardized mappings of anomalies and structural configurations. The ability of IRT to detect unknown subsurface structures and adapt to in situ conditions highlights its potential as a preliminary assessment and monitoring tool for heritage assets. Thus, the method not only generates objective measurements for the initial estimation of conditions but also offers vast combinatory potential with techniques such as photogrammetry and 3D documentation, elevating cultural heritage research methodology at any scale.

Ref. [21] focuses on the classification of materials and pathologies in cultural heritage (CH) using multispectral imaging within the optical range. The research concludes that the foundation of a robust assessment lies in the radiometric calibration of sensors and their hybridization (combining active and passive sensors) to achieve comprehensive studies. Passive sensors allow for low-cost acquisitions but are sensitive to light, whereas active sensors, though more expensive, enable the calculation of surface roughness and damage quantification. The importance of proper spectral configuration is emphasized: while the sensor used (Mini MCA-6) is ideal for detecting biological colonization, it needs to be combined with short-wave infrared (SWIR) sensors to analyze construction materials and moisture, a pathology characterized within the near- to mid-infrared range.

1.4. Deep Learning and Segmentation for Detection

In ref. [22], a systematic review of the use of deep learning techniques in the evaluation and management of cultural heritage is conducted, analyzing studies that apply models such as U-Net, Mask R-CNN, YOLO, and convolutional networks to tasks such as crack detection, erosion, disintegration, black crusts, and other forms of deterioration in ashlar stones and walls. The study compiles and classifies the most relevant approaches, highlighting the benefits of deep learning in automating diagnosis, semantic image segmentation, and large-scale monitoring of facades and heritage structures. It also identifies significant limitations, such as the scarcity of heritage-specific datasets.

In ref. [23], an automatic deep learning (DL) model and Artificial Neural Networks (ANNs) are proposed to recognize different types of weathering in stone belonging to cultural heritage through field photographs, with the aim of reducing human errors in conservation tasks. A dataset of images showing common deteriorations, such as cracks, erosion, black crusts, and efflorescence, among others, was collected. The authors trained DL and ANN models and found that the DL model achieved an accuracy of 99.4% with recall values ranging from 96% to 100%, while the ANN model reached an approximate accuracy of 93.95%. These results demonstrate that deep learning models can distinguish with near-perfect fidelity the different types of visual deterioration in historical ashlar stones from images.

In ref. [24], an automatic crack detection approach for masonry facades is proposed, particularly targeting cases where annotated data are scarce. To address this, the authors use Transfer Learning techniques with pre-trained convolutional neural networks as feature extractors, fine-tuning them with a custom dataset of images split into training, validation, and testing sets. They also conduct experiments with data augmentation to improve generalization. The results show that this approach achieves very high accuracy and F1-Score values, reaching up to 100% in some cases of full training.

In ref. [25], an automatic framework based on advanced YOLO models is proposed for crack and damage detection in historical buildings, aiming to make inspections once dependent on human visual observations faster and more accurate. The authors created their own high-resolution dataset, where after capturing the samples, they labeled them into four damage levels (none, minor, moderate, and severe). They compared the performance of YOLOv5, YOLOv8, YOLOv10, and YOLOv11 variants using metrics such as precision, recall, mAP@50, and mAP@50-95. Their findings show that YOLOv10 provides the best overall balance, particularly for minor damages and multiple localizations, while YOLOv8 and YOLOv11 demonstrate good adaptability.

1.5. Contribution to the Analysis of Sillar Deterioration in Architecture

The objective of this article is to propose an approach for the analysis of various pathologies in the historical heritage of Arequipa, where sillar is a principal component of structures. The database of samples was collected from the main streets of the historic center of Arequipa.

The article is organized into six sections. Section 1 presents the introduction. Section 2 provides the background definitions. Section 3 describes the materials and methods. Section 4 reports the results. Section 5 presents the discussion. Finally, Section 6 draws the conclusions, summarizing the achievement of the article’s objective.

2. Background Definitions

2.1. Definitions of Sillar

Locally, the ignimbrites of Arequipa are termed sillar. This term was first first introduced to the volcanological literature when describing the well-known whitish ignimbrite used for construction of colonial buildings in Arequipa. Construction blocks are indurated by vapor-phase recrystallization, in contrast to the non-welded ignimbrites used elsewhere [26].

2.2. Classification

2.2.1. Classification According to Location

There are four layers of ignimbrite in the Arequipa area [26]: Río Chili Ignimbrite (RCI), La Joya Ignimbrite (LJI), Arequipa Airport Ignimbrite (AAI), and Yura Tuffs (YTs) [27].

2.2.2. Petographic Classification

The results of water absorption by capillarity (C) and ultrasound pulse velocity (UPV) demonstrate a slight anisotropy for the beige variety and near-isotropy for the white and pink ignimbrites, which justifies the randomness of the application of the ashlars in the masonry and in the selection of the faces to carve. The petrographic classification was performed using steorology by means of thin sections (polished surfaces), which yields the modal mineralogical composition (representative mean of the sample values of each sheet studied). The classification was deduced from the constituent oxides of the alkaline cations versus the silica content (silicon oxide or SiO₂). The results, consistent with the petrographic classification, revealed a dacitic composition with lower concentrations of K₂O and Na₂O than other ignimbrites characterized [28].

2.3. Properties

2.3.1. Porosimetry

The mercury injection porosimetry revealed very high porosities for the three ignimbrite varieties, with values around 46.5% for the white and beige and 35.5% for the pink variety. The three rocks exhibited a bimodal distribution, with similarities between the white and beige specimens [28].

2.3.2. Density

The apparent density showed values between 1310 kg/m³ and 1350 kg/mm³ for the beige and white ignimbrites, respectively, and 1518 kg/m³ for pink ignimbrites, values lower than those of the Morelia ignimbrites [28].

2.3.3. Absorption

The water absorption by capillarity (C) after 30 min of testing revealed that the Pk variety reached nearly the same saturation as in free immersion after 1 h, with values of capillary absorption around 15%. The Wt and Be varieties presented a capillary absorption of 18% and 21%, respectively, at 30 min, still far from the free-immersion values (Ab) [28].

2.3.4. Permeability

Permeability can be estimated theoretically using capillary data (C). The values obtained were 2.66, 3.13, and 2.72 (10⁻⁶ m/s) for the white, beige, and pink ignimbrites, respectively [28].

2.3.5. Endurance

In the uniaxial compressive strength tests, the lowest values were found for the Be ignimbrites, and the highest were for the Pk ignimbrites. The fractures were both granular (glassy grains) and intergranular depending on the hardness of the component minerals. In volcanic rocks, non-linear behavior has been observed. First, the elastic limit is more clearly seen in the Pk ignimbrite, which corresponds to brittle fracture, with a defined yield point coinciding with the ultimate strength and then a plastic phase before fracture. Second is a small elastic phase, and then a plastic phase that culminates in the ultimate strength before the fracture. Third, there is irregular elastic deformation through the addition of deformations in the elastic phase [28].

2.3.6. Hardness

The average hardness of the ignimbrites is $L{H}_D=329\pm 36$. In combination, the Leebs surface hardness and UPV define the high or low cohesion of ignimbrite components and their surface hardness, respectively, more than isolated measurements [28].

2.4. Definition of Pathology

Some of the main deterioration problems affecting these rocks include the formation of surface deposits (salts and black crusts) [29]. Pathologies in ashlars refer to the processes of deterioration or alteration that affect carved stone pieces used in the construction of walls and architectural structures and usually manifest themselves in the form of cracks, erosion, surface disintegration, sanding, salt efflorescence, or loss of material.

2.5. Causes of Pathologies

A further difficulty in addressing material and building pathology lies in environmental variability. Because deterioration mechanisms are governed by both the inherent properties of materials and the conditions in which they exist, any environmental shift modifies the way these mechanism act. As a result, deterioration rates are controlled by multiple ever-changing factors. The only predictable aspect of the process is that, without intervention, the overall condition of the material and structure will inevitably worsen, while the properties of the unaffected material remain stable.

2.5.1. Thermal Variation

With a few notable exceptions, materials expand as they are warmed and contract when they are cooled. It is those exceptions to the rule regarding thermal expansion and contraction that are particularity important to the field of building pathology [30].

2.5.2. Rising Humidity

Rising damp is one of the most recurrent and well-known hazards to existing buildings and monuments. As the phenomenon of rising damp is quite slow, damage to building materials and structures may become visible only several years after construction [31]. Several effects related to humidity can be considered, such as the crystallization of soluble salts and chemical agents resulting from environmental pollution, which can cause material loss, detachment, and damage to infrastructure.

2.5.3. Chemical Agents

Regarding pathologies in masonry walls, the main causes of alteration in the stones are the action of chemical agents from the atmosphere (carbon dioxide and sulfur dioxide), chemical agents from the materials themselves and from the soil, and physical agents (water, temperature, wind, and living organisms) [32]. Several effects related to humidity can be considered, such as the crystallization of soluble salts and chemical agents resulting from environmental pollution, which can cause material loss, detachment, and damage to infrastructure.

2.6. Main Types of Pathologies

2.6.1. Surface or Granular Disintegration

Granular disintegration is the most frequently observed decay pattern affecting marés sandstone in Palma Cathedral. It manifests as surface powdering, flaking, or exfoliation of the outer stone layer, typically occurring in shaded, poorly ventilated, or moisture-retaining zones [33]. This process causes a gradual loss of particles and weakens the outer layer of the stone. In the early stages, it manifests as a thin layer of dust or a rough surface texture.

2.6.2. Structural Cracking

These are longitudinal openings that extend through the entire thickness of a structural element [34]. Cracks, whether structural or thermally induced, compromise the cohesion of masonry units, create entry points for water and salts, and promote microbial growth. In advanced cases, they can lead to the detachment or fragmentation of blocks.

2.6.3. Eruptions or New Growths

Contaminants in the raw materials used in mortars, such as salts, for example, can cause serious damage to old structures through the process of efflorescence, which can occur even without the presence of water in the walls due to the hygroscopicity of these salts, particularly sodium chloride [35]. However, other salts from various sources can cause efflorescence, such as nitrates, which originate from the decomposition of organic matter, and sulfates, which originate from the use of materials such as cement and plaster.

2.7. Non-Destructive Testing

Analysis of cultural heritage by non-destructive testing methods allows conducting examinations of buildings while preserving their authenticity and integrity, as well as their historical and artistic value. Some of the NDT techniques are X-ray fluorescence spectroscopy, holographic interferometry, Infrared Thermography, sonic/ultrasonic testing, electromagnetic and electrical techniques, and the Schmidt Hammer Rebound (SHR) test. In order for these techniques to be applied non-destructively, the devices must be portable and the work must be done in situ [36].

3. Materials and Methods

The deterioration pattern labeled as surface wear in this work can be technically classified as a type of Differential Erosion according to the ICOMOS-ISCS Illustrated Glossary on Stone Deterioration Patterns (2008) [37]. This type of deterioration is characterized by non-uniform material loss across the stone surface, a typical feature of volcanic ignimbrites such as sillar due to their heterogeneous mineral structure. In addition, the presence of surface moisture was evaluated as a subtype of discoloration (chromatic alteration), appearing as darkened areas (moist areas) that may affect both the surface and the depth of the stone. Moisture was documented through visual inspection and photographic records, allowing for the correlation of erosion patterns with chromatic alteration and environmental conditions. Complementarily, the methodology developed in this research for detecting pathologies in sillar constructions is organized into several stages, ranging from data acquisition to the final segmentation and classification of affected areas in sillar buildings exhibiting moisture- and deterioration-related pathologies. Throughout the study, multiple samples were collected in the form of digital images, which served as the basis for experimentation and validation of the proposed model. This process is illustrated in Figure 1, which depicts the methodological pipeline followed.

3.1. Data Acquisition

Digital images of various sillar buildings exhibiting pathologies such as moisture and surface wear were collected in the city of Arequipa, Peru. The dataset includes walls and façades, as shown in Figure 2. The images were captured under different lighting conditions and shooting angles to ensure adequate representativeness of real-world environments. The dataset reflects the most common pathologies of this construction material, such as surface wear and moisture stains on sillar. In total, approximately 3000 samples were collected, which are available in the following GitHub repository: https://github.com/galeonkil/dataset_sillar (accessed on 6 November 2025). Additionally, the dataset considers the diversity in the conservation state of the buildings, as well as variations in the texture and coloration of sillar, with the aim of providing a robust dataset suitable for subsequent preprocessing, segmentation, and classification tasks.

3.2. Image Preprocessing

The digital images captured in the study underwent a preprocessing stage due to the presence of obstacles during acquisition or elements within the image that may obscure pathologies. Subsequently, a background cleaning process was applied using the SegFormer model, focusing exclusively on segmenting constructions, windows, doors, and walls. This allowed for the precise isolation of the buildings and the removal of elements unrelated to the structure, as shown in Figure 3. As a result, a refined and consistent dataset was obtained, focusing exclusively on sillar surfaces and the visible pathologies present on them, such as cracks, surface wear, or moisture stains. This provides a solid foundation for subsequent segmentation and detection stages.

3.3. Initial Segmentation

In order to identify pathologies on sillar construction surfaces, primarily surface wear and moisture, a preliminary semantic segmentation of the images was implemented, serving as an initial labeling mechanism. Semantic segmentation is one of the high-level tasks that paves the way towards complete scene understanding [38]. First, the images captured in RGB format were transformed into the CIELab color space, due to its higher capacity to represent perceptual color differences compared to the RGB space. In this space, the K-Means clustering algorithm was applied with a predefined number of clusters, using a random sample of pixels to reduce computational complexity. This process grouped regions based on their chromatic and textural similarities. Subsequently, the SLIC (Simple Linear Iterative Clustering) superpixel technique was applied, which clusters pixels by considering both chromatic similarity and spatial proximity. For this, a distance metric was defined as shown in Equation (1) in order to generate a more structured image representation, where each superpixel preserves the spatial and textural coherence of the surface. This representation facilitated the identification of affected regions and reduced noise associated with segmentations strictly based on individual pixels. To delimit the building area and discard the background, a binary mask was generated from color thresholds and further refined through morphological operations (closing and contour filling). This ensured that only relevant constructive zones were preserved while avoiding false positives in the surrounding environment. The segmented regions obtained from the clustering and superpixel processes were subsequently validated and used as approximate labels in different deep learning architectures. This scheme enabled an initial training with weak supervision, where the automatically generated labels served as guidance to achieve more accurate deep segmentation. In this way, the approach combines the automatic exploration of unsupervised methods (K-Means + SLIC) with the refinement provided by deep neural networks, resulting in a hybrid system for the detection and delineation of pathologies in stone-based construction materials, as shown in Figure 4.

$$D=\sqrt{{\left({\displaystyle \frac{d_c}{m}}\right)}^2+{\left({\displaystyle \frac{d_s}{S}}\right)}^2}$$

(1)

where $d_c$ represents the Euclidean distance in the CIELab color space, $d_s$ corresponds to the Euclidean distance in the spatial domain, m is the compactness parameter, and S is the interval between superpixel centers, and D thus represents the overall distance combining color and spatial proximity.

3.4. Deep Segmentation

In this stage, a deep segmentation of the sillar buildings captured in the digital images is carried out in order to detect moisture and surface wear pathologies. To achieve this, a Transformer-based neural network, specifically the Swin Transformer, was selected. The proposed Swin Transformer achieves strong performance in the recognition tasks of image classification, object detection, and semantic segmentation [39]. Given its high accuracy in segmentation tasks and its widespread use in segmentation research, this architecture was selected for its ability to preserve global representations, capture long-range dependencies, and adapt to surfaces with irregular textures. Subsequently, it was trained with the previously collected digital images to evaluate its accuracy in the segmentation of sillar buildings, as shown in Figure 5.

3.5. Validation

In the validation stage, two types of plans are used to evaluate the results obtained. The first corresponds to the pathology plan, elaborated through visual inspection and detailing the pathologies present on the sillar surface, where moisture and wear were identified and delineated, as shown in Figure 6, Figure 7, Figure 8 and Figure 9. The second type of plan corresponds to a combined representation, in which the areas segmented by the Swin Transformer model are superimposed onto the areas detected in the pathology plan, as shown in Figure 10, Figure 11, Figure 12 and Figure 13. This allows for a direct visual comparison of matches and discrepancies. To complement this qualitative comparison, quantitative metrics were also applied. First, the IoU (Intersection over Union) was calculated between the automatic segmentations and the reference annotations in order to measure the model’s accuracy in detecting each pathology. Second, a building-level analysis was performed, quantifying the proportion of the surfaces affected by moisture and wear detected by the model in relation to the total facade area of the buildings. In this way, validation combines both a visual comparison through superimposed plans and a quantitative evaluation using IoU and area-based metrics, thereby providing a more robust assessment of the model’s reliability in the automatic identification of pathologies in sillar constructions.

4. Results

The results of the research demonstrate the model’s ability to automatically detect pathologies present in sillar buildings, primarily moisture and surface wear. The experimental outcomes showed that the model achieved an acceptable performance on exposed and uncovered surfaces, successfully distinguishing between affected and healthy areas, which is useful for generating deterioration maps applicable to heritage conservation. In the validation carried out using the Intersection over Union (IoU) metric and the percentage of affected area with respect to the total façade area, corresponding to Equations (2) and (3), a variable performance was observed depending on the building and the pathology analyzed. For Building A, the IoU reached 60.8% for surface wear and 69.1% for moisture, with affected area percentages of 11.4% and 11.0% compared to the reference values of 18.8% and 16.0%, respectively. For Building B, surface wear prediction was null (IoU of 0.0%), while for moisture, the model achieved an IoU of 80.6%, with an estimated affected area of 17.6% compared to the 21.9% reference value. For Building C, the results showed IoUs of 66.7% for surface wear and 92.3% for moisture, with smaller percentage differences compared to the real areas (10.2% vs. 15.2% and 8.7% vs. 9.4%). Finally, for Building D, the highest levels of agreement were achieved, with IoUs of 93.8% for surface wear and 87.0% for moisture, along with percentages very close to the reference values (37.3% vs. 35.0% and 16.9% vs. 19.4%). Overall, the results reflect that the model exhibits a robust ability to identify areas of moisture and surface wear on exposed surfaces, with particularly high effectiveness in buildings such as Building D, where the predicted values almost matched those in the pathology plan, as shown in Figure 14. However, some limitations were also observed, especially in buildings with surface coatings or in the detection of smaller-scale pathologies such as micro-cracks, which affects the system’s sensitivity in more complex scenarios.

4.1. Intersection over Union (IoU)

The IoU metric is defined as

$$IoU={\displaystyle \frac{{\mathrm{Area}}_{pred}\cap {\mathrm{Area}}_{real}}{{\mathrm{Area}}_{pred}\cup {\mathrm{Area}}_{real}}}$$

(2)

• $Are{a}_{pred}$: Area predicted and segmented by the model (Swin Transformer).
• $Are{a}_{real}$: Real pathology area defined by visual inspection (real).

4.2. Percentage of Affected Area

The percentage of area affected by each pathology with respect to the total facade area of the building is expressed as

$${\%}_{pathology}={\displaystyle \frac{Are{a}_{path}}{tota{l}_{fa}}}\times 100$$

(3)

• $Are{a}_{path}$: Area affected by a specific pathology (moisture or surface wear).
• $Tota{l}_{fa}$: Total surface area of the wall or building analyzed.

5. Discussion

The proposed methodology proved effective for the detection of pathologies in sillar buildings, confirming the feasibility of applying computer vision and deep learning techniques in the field of heritage conservation. The results obtained show that the system can accurately identify visible damage such as moisture and surface wear, provided that the surfaces are exposed and free of coatings. This validates the initial hypothesis that the natural texture and color of sillar are key factors influencing the model’s performance. However, several relevant limitations were identified. First, the methodology faces challenges on painted or coated surfaces, where the presence of external layers alters visual properties and reduces the system’s ability to properly segment. This finding is consistent with other studies in heritage segmentation, where surface variability interferes with the extraction of discriminative textures. Second, the model shows low sensitivity in detecting small-scale pathologies such as micro-cracks or early moisture spots. During preprocessing and segmentation, these subtle patterns are often diluted in the global representation of the image. Additionally, an error inherent to the network was observed, related to its inability to segment multiple pathologies within the same pixel or region. For instance, when an area of surface wear occurred within a moisture zone, the model could only assign a single label, representing it with one color instead of reflecting both conditions. This limitation restricts the system’s ability to capture overlapping or interacting pathologies. Another important aspect is the very nature of sillar, a highly porous material capable of retaining atmospheric pollutants, such as gases from vehicle combustion (CO₂), which cause the progressive darkening of its originally white surface. The Arequipa ignimbrite used in heritage buildings exhibits high porosity values, typically ranging from 35% to 54%, consistent with a medium welding degree (II) [28,40]. This phenomenon can lead to classification errors, as in some cases the model interprets these blackened stains as moisture. A possible future solution would be to incorporate higher-resolution images, deep learning-based super-resolution techniques, or architectures specifically designed for small-object detection. While the use of unsupervised clustering provided a useful preliminary segmentation, the transition to deep architectures still requires further validation with more diverse datasets. It is also recommended to complement this approach with multispectral or hyperspectral techniques, which could improve the differentiation of materials and more subtle pathologies on sillar surfaces. The research proposes a non-invasive methodology that follows the conservation criteria established by [8], which states that “the integrity of 20th-century cultural heritage should not be affected by insensitive interventions or changes. Adequate research, documentation, and analysis of the site’s history and significance are necessary to avoid, minimize, and mitigate potential negative impacts.” Although Artificial Intelligence cannot replace more precise methods or tests for pathology detection, it can serve as a complementary tool to support their identification and monitoring. Overall, the findings of this work highlight both the potential and the challenges of applying computer vision to heritage conservation, showing that while the results are promising, further optimization is still required for more complex and heterogeneous scenarios.

6. Conclusions

The results demonstrate that this methodology based on computer vision and deep neural networks constitutes a promising tool for the non-invasive monitoring of heritage sillar buildings, adding value to the processes of conservation and restoration of cultural heritage. The integration of preprocessing techniques, semantic segmentation, and deep learning makes it possible to obtain satisfactory results in the identification of pathologies such as moisture and surface wear, highlighting its usefulness as support in decision-making for architectural intervention projects. Nevertheless, we identify the need to improve detection on painted or coated surfaces, as well as of small-scale pathologies such as micro-cracks. Future studies should aim to strengthen non-invasive diagnostic methodologies in architecture, highlighting their advantages over traditional techniques. Conventional methods, typically based on visual inspection or destructive testing, are often limited by subjectivity, time requirements, and the potential risk of damaging heritage materials. In contrast, non-invasive approaches supported by computer vision, multispectral imaging, and deep learning enable a more objective, efficient, and repeatable assessment of material deterioration. Further research could focus on incorporating higher-resolution datasets, advanced spectral analysis, and neural network architectures optimized for microstructural detection. These improvements would enhance the accuracy and scalability of damage evaluation, contributing to more reliable decision-making processes in the conservation and restoration of architectural heritage. Taken together, this work lays the foundation for the development of automated systems that can strengthen the management of the built heritage of Arequipa by providing a more objective and systematic diagnosis of damage on sillar surfaces.