Deep Learning-Supported Panoramic Infrared Framework for Quantitative Diagnosis of Building Envelope Thermal Anomalies

Abstract

This study presents a modular diagnostic framework for evaluating thermal degradation in aging building envelopes by integrating infrared thermography, panoramic reconstruction, and deep learning-based semantic segmentation into a unified workflow. The methodology combines image registration, panoramic synthesis, façade component segmentation, and quantitative surface temperature analysis to provide scalable and reproducible diagnostics. By excluding fenestration zones—where infrared measurements are physically unreliable—the framework focuses on opaque wall regions and window surroundings to ensure physically meaningful evaluation. Field validation was conducted on a multi-story office building constructed in 1996. The diagnostic indicators revealed a mean wall surface temperature of 14.3 °C with a standard deviation of 5.6 °C, and a temperature factor ranging from 0.67 to 0.78 under measured conditions. The vulnerable area ratio reached 9.1% for walls, while window areas showed greater vulnerability at 12.74%, with anomalies concentrated at frame–glass interfaces and perimeter seals. These quantitative results confirmed the framework’s ability to detect thermal irregularities and visualize localized anomalies. More importantly, the contribution of this study lies in establishing a systematic and extensible diagnostic pipeline that advances building envelope analysis, supporting large-scale energy audits, retrofit prioritization, and sustainable building management.

1. Introduction

The global imperative to mitigate climate change and improve energy efficiency has elevated building envelope performance as a critical factor in sustainable urban development. Aging buildings, particularly those constructed before the enforcement of modern energy-saving standards, often suffer from insulation deterioration, thermal bridging, and air leakage, leading to excessive energy consumption and reduced occupant comfort. In South Korea, a significant proportion of public buildings were built between the 1980s and early 2000s, and many of these structures now exhibit thermal deficiencies that require systematic diagnosis and retrofit prioritization.

Conventional diagnostic methods, such as heat flow meters, blower door tests, and U-value calculations, provide useful insights but are limited in scalability and practicality. They often require interior access, specialized equipment, and localized measurements, making them unsuitable for large or complex buildings. Infrared thermography (IR) offers a non-invasive alternative by visualizing surface temperature distributions, yet standalone IR imaging lacks spatial continuity and is highly sensitive to environmental conditions, reducing diagnostic reliability [1,2]. International standards such as ISO 6781, ASHRAE Guideline 3, and ASTM C1060 have recognized IR as a valuable tool [1,2,3], while policy frameworks like the IEA projects highlight its role in energy audits [4]. Recent advances in industry and academia—including drone-based IR surveys, AI-driven image analysis, and deep learning segmentation models—further demonstrate the potential of IR diagnostics [5,6,7,8,9,10,11].

Despite these developments, existing studies remain fragmented: most rely on single-frame IR images, lack quantitative consistency, or fail to integrate advanced image processing techniques. No unified framework has yet been established that systematically combines panoramic IR reconstruction, semantic segmentation, and quantitative indicator analysis into a reproducible diagnostic pipeline.

This study addresses this research gap by proposing a modular diagnostic framework that integrates infrared thermography, panoramic reconstruction, and deep learning-based semantic segmentation into a unified workflow. By excluding fenestration zones—where IR measurements are physically unreliable—the framework focuses on opaque wall regions to ensure physically meaningful diagnostics. Key indicators such as mean surface temperature, standard deviation, temperature factor, and vulnerable area ratio are computed to provide quantitative insights. The primary contribution of this research lies not in isolated diagnostic results but in the establishment of a systematic and extensible pipeline that advances building envelope analysis, supporting large-scale energy audits, retrofit planning, and sustainable building management.

2. Related Work

Infrared thermography (IR) has emerged as a non-contact and efficient tool for diagnosing thermal insulation performance in building envelopes. Numerous studies have explored its application in identifying insulation failures, thermal bridges, and moisture-related anomalies, particularly in aging public buildings. International standards such as ISO 6781, ASHRAE Guideline 3, and ASTM C1060 have formally recognized IR as a diagnostic method, while policy frameworks like the IEA projects highlight its role in energy audits and retrofit verification [1,2,3,4].

Table 1. Summary of prior studies on infrared-based building diagnostics.

Author(s)	Year	Diagnostic Focus	Methodology/Technique	Quantitative Metric	Accuracy/Error Margin
Choi, K.S. [5]	2010	Insulation defect detection	IR–visible image alignment; Temperature Difference Ratio (TDR)	TDR	Qualitative evaluation
Choi, D.S. et al. [6]	2017	Thermal transmittance estimation	IR-based U-value vs. heat flow meter	U-value	±20% error margin
Martin, M. et al. [7]	2022	Multi-scale IR applications in the built environment	IR thermography combined with BIM; review	Energy performance metrics	Demonstrated applicability
Jung, H. et al. [8]	2024	Automated façade diagnostics	Deep learning segmentation (Cascade U-Net)	Segmentation accuracy	Improved performance over baseline models
Mirzabeigi, M. et al. [9]	2025	Thermal anomaly detection	IR + Semantic Segmentation	mIoU; anomaly detection	Validated on real buildings
Gertsvolf, N. et al. [10]	2023	ML readiness of IR data	Image preprocessing & enhancement	–	Prepared datasets for ML
Wang, P. et al. [11]	2025	Façade damage segmentation	IR–Visible Fusion + Deep learning	Segmentation accuracy	Improved multimodal performance

summarizes prior studies on infrared-based building diagnostics [5,6,7,8,9,10,11].

One key challenge in IR-based diagnostics is the lack of spatial context in thermal images. To address this, Choi (2010) proposed a method for aligning IR images with visible photographs to enhance diagnostic accuracy [5]. His work introduced the Temperature Difference Ratio (TDR) as a quantitative metric for identifying insulation defects. Environmental conditions such as ambient temperature, solar radiation, humidity, and wind speed also significantly influence IR image quality, requiring correction algorithms and preprocessing techniques such as gamma correction, histogram equalization, and noise filtering.

Other researchers have focused on field-based evaluation of insulation performance. Choi et al. (2017) compared IR-based U-value calculations with heat flow meter measurements, showing error margins within ±20% when emissivity and reflection settings were properly calibrated [6].

Recent studies have further advanced IR diagnostics by integrating machine learning and automation. Martin et al. (2022) provided a comprehensive multi-scale review of infrared thermography in the built environment, demonstrating how IR can be integrated with Building Information Modeling (BIM) to support city-scale energy assessments and highlighting its applicability across building, district, and urban levels [7]. Jung et al. (2024) introduced a Cascade U-Net framework for façade image segmentation, showing that object-wise processing combined with contour guidance significantly improves segmentation accuracy compared to conventional DeepLabV3+, thereby enhancing automated diagnostics of thermal anomalies [8]. More recently, Mirzabeigi et al. (2025) proposed an integrated vision-based framework combining semantic segmentation with IR imaging to detect thermal anomalies in building envelopes [9]. Gertsvolf et al. (2023) emphasized preprocessing and image enhancement techniques to prepare IR data for future machine learning applications [10]. Wang et al. (2025) introduced a multimodal fusion approach using infrared–visible image integration to improve façade damage segmentation accuracy [11].

These studies collectively underscore the potential of IR thermography for building diagnostics, while also revealing persistent limitations: fragmented methodologies, reliance on single-frame images, and lack of reproducible quantitative frameworks. The present study builds upon these foundations by proposing a comprehensive diagnostic framework that integrates panoramic reconstruction, semantic segmentation, and quantitative heat loss estimation, aiming to enhance both accuracy and scalability.

3. Methodology

This section establishes the image processing framework required for diagnosing thermal insulation performance in building envelopes using infrared (IR) thermography. Conventional single-frame IR images lack spatial continuity and therefore cannot provide a comprehensive assessment of façade conditions. To overcome this limitation, the proposed framework integrates four modules: (1) IR–visible image alignment and correction, (2) panoramic thermal reconstruction, (3) semantic segmentation of envelope components, and (4) quantitative surface temperature–based diagnosis.

The overall workflow of the diagnostic pipeline is illustrated in Figure 1. This schematic highlights the input–output relationships among modules and clarifies how each step contributes to the unified process. Specifically, image alignment ensures geometric consistency, panoramic reconstruction provides spatial continuity, semantic segmentation isolates reliable wall regions, and quantitative diagnosis translates thermal data into interpretable indicators.

3.1. Image Alignment and Correction

Accurate alignment between infrared (IR) and visible imagery is essential for contextualizing thermal anomalies within architectural components. Without proper registration, anomalies detected in IR images cannot be reliably associated with specific façade elements such as walls, joints, or fenestration. In the initial stage, feature point extraction and matching were performed to establish geometric correspondence between IR and visible images. Figure 2 illustrates the feature matching example, while Figure 3 presents the final alignment result, confirming geometric consistency across modalities.

The alignment procedure consisted of several steps. First, feature points were extracted from both IR and visible images [12]. These points were matched using similarity metrics such as normalized cross-correlation (NCC) and structural similarity index (SSIM) Once matched, a homography transformation was applied to warp the IR image into the coordinate system of the visible image, ensuring that thermal anomalies could be directly mapped onto architectural features [12]. Lens distortion correction was also performed, as shown in Figure 4, to mitigate radial and tangential distortions inherent in IR camera optics.

Preprocessing techniques were applied to improve image clarity and diagnostic reliability. Gamma correction enhanced contrast in low-intensity regions, histogram equalization redistributed pixel intensities to highlight thermal gradients, and bilateral filtering reduced sensor noise while preserving edge sharpness. Their comparative performance is summarized in

Table 2. Similarity metrics and success rates of image alignment methods.

Preprocessing Method	Similarity Metric	Success Rate (%)
Histogram Equalization	SSIM	92.3
Gamma Correction	NCC	88.7
Bilateral Filtering	MSE	85.4

, which shows that histogram equalization achieved the highest alignment success rate (92.3%), followed by gamma correction (88.7%) and bilateral filtering (85.4%).

Finally, the registration was completed using perspective transformation, which enabled panoramic synthesis and distortion correction. The perspective-based alignment process is illustrated in Figure 5. This approach allowed for continuous analysis of the entire façade, minimized positional errors and temperature deviations, and provided a reliable foundation for subsequent semantic segmentation and quantitative diagnosis.

3.2. Panoramic Thermal Reconstruction

Single-frame infrared (IR) images provide limited spatial context, often obscuring systemic deficiencies in building envelopes. To overcome this limitation, multiple aligned IR images were synthesized into panoramic thermal maps. This process enables continuous visualization of façade surfaces, allowing distributed anomalies and clustering patterns to be identified across large areas.

The types of camera movement used for panoramic acquisition are illustrated in Figure 6, which include horizontal panning, vertical scanning, and rotational motion. These acquisition strategies ensured sufficient overlap between consecutive frames, a prerequisite for accurate stitching.

While an affine transformation provided basic continuity, a perspective transformation was ultimately adopted to correct geometric distortion and align vanishing points. The workflow for distortion correction is detailed in Figure 7, which illustrates how perspective transformation and viewpoint correction compensate for camera tilt and parallax effects.

By applying perspective transformation, the reconstructed panoramic images achieved geometric consistency across the entire façade. This approach minimized positional errors and temperature discontinuities that often arise when analyzing isolated frames.

3.3. Semantic Segmentation

Since different building components exhibit distinct physical properties such as emissivity and transmissivity, it is necessary to apply different conditions for each part. Accordingly, this study separated façade regions into walls, windows, and other installations, assigning appropriate physical values to each category and evaluating the surface temperature of each component individually.

The segmentation model used in this study was DeepLabV3+ with a ResNet-50 backbone [13], which combines atrous convolution with an encoder–decoder structure to capture both global context and fine boundary details. The architecture of the model is presented in Figure 8, illustrating the multi-scale feature extraction and refinement process.

Training was conducted using labeled façade datasets, including Cityscapes [13], ADE20K [14], and COCO-Stuff [15]. An example of segmentation output is shown in Figure 9, where each façade component is successfully segmented. Benchmark performance is summarized in

Table 3. Performance comparison of semantic segmentation models.

Benchmark Data Set	Best mIoU	Key Characteristics
Cityscapes test	0.851	Focused on urban street scenes in Germany; includes roads, buildings, vehicles, pedestrians—highly relevant for façade segmentation.
PASCAL VOC 2012 test	0.905	Object recognition benchmark with 20 classes (people, animals, everyday objects); widely used for general segmentation evaluation.
PASCAL Context	0.605	Extension of VOC with over 400 detailed categories; supports fine-grained scene parsing in complex environments.
ADE20K	0.570	Diverse indoor and outdoor scenes with 150 classes; useful for architectural and spatial analysis.
COCO-Stuff test	0.454	Extension of the COCO dataset, including both “things” (objects) and “stuff” (background elements), enables comprehensive scene understanding.

, which presents both the segmentation results and the key characteristics of each dataset. These results confirm the model’s ability to reliably isolate wall regions, thereby filtering out unreliable IR data and ensuring that subsequent thermal diagnostics reflect actual insulation performance rather than material-specific artifacts. The model achieved a mean Intersection-over-Union (mIoU) exceeding 0.85 on the Cityscapes dataset, outperforming conventional segmentation approaches.

By integrating semantic segmentation, the diagnostic pipeline ensures that quantitative indicators—such as mean surface temperature, standard deviation, and temperature factor—were computed only from valid wall regions. This selective approach enhances diagnostic precision and enables the identification of vulnerable areas in building envelopes.

3.4. Surface Temperature-Based Diagnosis

Thermal infrared imaging non-contactly measures surface temperature distributions across building envelopes. To evaluate insulation performance and detect degradation phenomena, this study derived statistical and physical indicators from segmented wall regions, ensuring that diagnostics are based on thermally meaningful data. As established in Section 3.3, semantic segmentation excluded fenestration zones due to distortion and limited the diagnosis to opaque wall regions and window surroundings. The following diagnostic metrics were employed:

• Mean surface temperature ($T_{surf}$): This represents the baseline thermal state of the wall, providing a general measure of heat retention.
• Standard deviation ($T_{surf}$): This captures spatial variability of surface temperature, serving as a proxy for insulation uniformity. Elevated values indicate localized anomalies such as thermal bridges or construction defects.
• Temperature factor ($f_{Rsi}$):

  $$f_{Rsi}={\displaystyle \frac{T_{surf}-T_{out}}{T_{in}-T_{out}}}$$

  where $T_{surf}$ is the wall surface temperature, $T_{out}$ is the outdoor air temperature, and $T_{in}$ is the indoor reference temperature (assumed 20 °C under steady-state conditions). This factor provides a normalized measure consistent with national standards and is widely used to assess condensation risk and thermal resistance.
• Distributional indicators (Median, Q1, Q3): Quartiles were computed from wall-region temperature distributions. Q1 (25th percentile) highlights cooler anomalies, often associated with thermal bridges or insulation defects. The median (50th percentile) represents the central tendency of the façade’s thermal condition. Q3 (75th percentile) highlights warmer anomalies, which may indicate air leakage or localized heating. These distributional indicators complement mean, standard deviation, and $f_{Rsi}$ to provide a comprehensive profile of façade performance.
• Vulnerable area ratio: This is defined as the fraction of valid wall pixels classified as vulnerable (e.g., $T<$ median or $f_{Rsi}<$ 0.70–0.75) divided by the total valid wall pixels, expressed as a percentage.

Reference values for surface thermal resistance are listed in

Table 4. Surface thermal resistance values from national standards.

	Indoor Surface Resistance (m2·K/W)	Outdoor Surface Resistance (m2·K/W)
		Indirect	Direct
Living Room Exterior Wall (including sides, windows, doors)	0.11	0.11	0.043

, while regional U-values for different envelope components are summarized in

Table 5. Regional U-values by building component.

Component Type	Exposure Condition	Zone 1	Zone 2	Zone 3	Zone 4
Living Room Exterior Wall U-Value (W/m2·K)	Direct exposure to outdoor air	≤0.170	≤0.240	≤0.320	≤0.410
	Indirect exposure to outdoor air	≤0.240	≤0.340	≤0.450	≤0.560

[16].

By comparing measured values with reference thresholds, areas exhibiting low temperature factors or high variability were identified as potential thermal bridges or insulation defects. This quantitative approach ensures that diagnostics are not limited to qualitative visual inspection but are grounded in measurable thermal performance criteria.

Finally, these indicators and thresholds were implemented into a practical diagnostic tool, integrating image registration, panoramic reconstruction, semantic segmentation, and quantitative analysis into a unified workflow. This completes the methodological framework; in Section 4, the tool is applied to a case study building to validate its practical applicability.

4. Field Application and Diagnostic Results

Building upon the diagnostic framework established in Section 3.4, the proposed methodology was implemented into a practical diagnostic tool (Figure 10). This tool integrates image registration, panoramic reconstruction, semantic segmentation, and quantitative indicator analysis into a unified workflow. In this section, the tool is applied to Building A to validate its practical applicability under real operating conditions. The case study demonstrates how the computed indicators and thresholds are used to identify thermal anomalies, quantify vulnerable regions, and support retrofit prioritization.

4.1. Target Building and Measurement Conditions

Building A (Figure 11) is a reinforced concrete public building constructed in 1996, comprising four above-ground floors. The envelope consists of granite cladding (20 mm), concrete walls (20 mm), EPS insulation (70 mm), and gypsum board (12.5 mm), with aluminum-framed composite glazing systems.

Infrared imaging was performed at dawn on 29 November 2023, 6:30 AM, under winter conditions to ensure sufficient indoor–outdoor temperature gradient. A total of 15 infrared images were captured, and additional surface temperature measurements were conducted on window glazing and frames to correct emissivity-related errors. During the measurement, all windows and doors were kept closed, and indoor heating was operated to maintain the indoor temperature at 22 °C. After image acquisition, the diagnostic tool required about ten minutes to complete automated analysis. Measurement conditions are summarized in

Table 6. Measurement Conditions.

Item	Building A
Measurement Date	29 November 2023
Measurement Time	6:30 AM
Outdoor Temperature	−1.2 °C
Wind Speed	1.5–2.5 m/s
Indoor Temperature	22.0 °C
Façade Orientation	South-facing (front)

.

In addition, according to the 1996 Energy-Saving Design Standards for Buildings, the legal U-value requirements were 0.58 W/(m²·K) for external walls and 3.0 W/(m²·K) for windows and doors. Building A was designed to meet these regulatory values; however, since approximately 30 years have passed since completion, deterioration of insulation materials and aging of finishes are expected to prevent compliance with the original U-value requirements [16].

4.2. Diagnostic Results

Infrared thermographic analysis of Building A revealed distinct thermal anomalies across wall and window assemblies. Measurements were conducted under winter dawn conditions (outdoor −1.2 °C, indoor 22.0 °C, wind 1.5–2.5 m/s), ensuring sufficient temperature gradient.

Table 7. Diagnostic results for Building A.

Component	Mean Temperature (°C)	Standard Deviation (°C)	Q1	Median	Q3	Vulnerable Area Ratio
Wall	14.33	5.58	13.06	14.92	16.86	9.05%
Window	14.54	3.85	12.73	14.02	16.41	12.74%

summarizes the computed indicators.

The façade exhibited substantial variability in surface temperature distribution. Wall assemblies showed a standard deviation of 5.58 °C, indicating non-uniform insulation performance. Based on the surface temperatures in Table 7, the temperature factor ($f_{Rsi}$) ranged from 0.67 to 0.78 under the measured conditions, and from 0.74 to 0.83 when recalculated for Zone II design temperature (−8 °C) [16]. These values approach the regulatory threshold (≥0.70), suggesting condensation risk in colder regions. Approximately 9.05 % of the wall surface was identified as thermally deficient.

Window areas displayed greater vulnerability. The vulnerable area ratio reached 12.74%, with anomalies concentrated at frame–glass interfaces and perimeter seals. Diagnostic results (Figure 12, Figure 13, Figure 14 and Figure 15) visually corroborate these quantitative findings, showing clustered cold spots at window perimeters and wall intersections.

4.3. Discussion

The diagnostic analysis of Building A highlights clear patterns of thermal vulnerability in the façade. The variability in surface temperatures indicates non-uniform insulation performance, with localized anomalies concentrated at wall intersections and window junctions. These findings are consistent with the structural characteristics of aging public buildings, where discontinuities in construction often lead to thermal irregularities.

The clustering of anomalies around window perimeters and wall joints suggests infiltration pathways and insufficient sealing. Fenestration systems emerged as dominant contributors to façade vulnerability, while wall assemblies also exhibited localized weaknesses. Importantly, the results emphasize that façade degradation is not uniform but concentrated in specific structural features, which should be prioritized in retrofit planning.

The diagnostic framework proved effective in quantifying these vulnerabilities through statistical indicators and spatial mapping. By translating thermal irregularities into measurable ratios and variability indices, the framework provides actionable data for energy audits and retrofit prioritization. This quantitative approach ensures that interventions can be targeted to areas with the greatest impact on energy efficiency and occupant comfort.

Nevertheless, certain limitations must be acknowledged. The analysis assumes steady-state thermal conditions and uniform emissivity, which may not fully capture transient effects or material heterogeneity. Environmental sensitivity, particularly to wind and solar radiation, remains a challenge despite efforts to minimize external influences during measurement.

Despite these constraints, the study demonstrates that infrared thermography combined with advanced image processing offers a reproducible and scalable diagnostic method. For Building A, the results provide a clear basis for retrofit strategies: reinforcing wall joints to reduce thermal irregularities and improving insulation around window areas to enhance sealing performance. Such targeted interventions are expected to significantly reduce heat loss, enhance indoor thermal comfort, and support long-term energy efficiency goals in public buildings.

5. Conclusions

This study developed and validated a deep learning–supported panoramic infrared framework for diagnosing thermal anomalies in aging building envelopes. By integrating image registration, panoramic reconstruction, semantic segmentation, and quantitative surface temperature analysis into a unified workflow, the framework enabled reproducible and scalable diagnostics.

Field validation confirmed that the framework can effectively identify thermal vulnerabilities in both wall and window areas. In particular, anomalies were concentrated at wall intersections and window joints, indicating priority areas for retrofit strategies.

The main contributions of this study can be summarized as follows:

• Reliable façade segmentation using DeepLabV3+ enabled diagnostics focused on thermally valid wall regions.
• Panoramic infrared reconstruction improved spatial continuity, revealing systemic anomalies not visible in single-frame IR.
• Quantitative indicators allowed for straightforward identification of heat-loss areas in building envelopes.
• The applicability of the automated workflow for IR-based building heat-loss analysis was confirmed.

Despite limitations such as steady-state assumptions, environmental sensitivity, and material heterogeneity, the framework establishes a systematic and extensible diagnostic pipeline that advances building envelope analysis, supporting energy audits, retrofit planning, and sustainable building management.