Urban Morphology-Oriented Streetscape Segmentation via Hierarchical Transformer and Frequency-Aware Feature Learning

Abstract

Semantic segmentation of street-view imagery has become an important computational tool for urban morphological analysis and the evaluation of street spatial quality. However, existing methods still struggle in complex urban environments. Major challenges include large variations in building façade scales, degradation of boundary information, and severe class imbalance. These issues limit the ability of current models to capture structurally meaningful urban forms. To address these challenges, this study proposes a high-resolution street-view segmentation framework, termed HieraWaveSeg. The model aims not only to improve pixel-level segmentation accuracy but also to enhance the interpretability of urban morphology through structured representations of street space. Specifically, a Hiera Transformer backbone is employed to capture hierarchical spatial semantics. A Path Aggregation Network is further introduced to strengthen cross-scale feature interaction and improve structural consistency in complex scenes. In addition, a Wave Fusion module based on the Haar wavelet transform is incorporated to preserve fine-grained architectural details by enhancing high-frequency boundary and texture information during decoding. Unlike conventional segmentation approaches that primarily focus on object recognition, this study introduces a morphology-oriented semantic reconfiguration strategy. This strategy reorganizes original categories into functionally meaningful urban units. As a result, the segmentation outputs can be more directly linked to urban morphological indicators, such as façade continuity, spatial enclosure, and interface permeability, thereby improving interpretability in architectural and urban design contexts. To further address class imbalance, a composite loss function combining weighted cross-entropy and Dice loss is adopted, together with a median frequency balancing strategy. Experimental results on the CamVid and Cityscapes datasets demonstrate that the proposed method consistently outperforms several state-of-the-art baselines in both segmentation accuracy and structural preservation. Beyond quantitative improvements, the results indicate that the proposed framework generates more coherent and morphologically meaningful urban representations, supporting further quantitative analysis in urban morphology and architectural studies.

1. Introduction

1.1. Urban Morphology Background and Research Significance

Urban morphology studies the spatial structure, form, and organization of the built environment, including buildings, street networks, and spatial interfaces [1,2]. It provides a fundamental framework for understanding how cities are physically constructed and how spatial configurations influence human perception and activity [3].

With the rapid development of smart cities and City Information Modeling, urban data acquisition is shifting from traditional remote sensing imagery toward multi-source, multi-view, and human-centered street-level data [4]. Among these, street-view imagery captures the urban environment from a pedestrian perspective and provides detailed façade information, spatial hierarchy, and street interface characteristics [5]. Compared with aerial remote sensing data, it offers a more direct representation of urban form at the human scale, making it highly valuable for morphological analysis in architecture and urban design.

From the perspective of urban morphology, street-view data enables quantitative analysis of façade continuity, street enclosure, spatial rhythm, and visual permeability [6]. These attributes are essential for understanding urban form and evaluating spatial quality in both historical and contemporary contexts. Therefore, fine-grained building segmentation from street-view imagery is not only a computer vision task but also a key computational step for morphological analysis.

By converting visual data into structured spatial representations, semantic segmentation serves as a bridge between digital perception and morphological interpretation of the built environment [7]. This supports more objective and scalable analysis of urban form and enables applications in urban regeneration, historic preservation, and spatial planning [8].

1.2. Research Gap and Limitations of Existing Methods

Despite significant progress in deep learning-based semantic segmentation, most existing methods are designed for general scene understanding and do not explicitly consider the structural requirements of urban morphology analysis [9].

Figure 1 shows the prediction performance of different deep learning models in the street scene semantic segmentation task and their class error distribution, where the red area represents over-detection (False Positives) and the yellow area represents under-detection (False Negatives).respectively. This visualization underscores the persistent challenges in achieving high-fidelity boundary alignment.

Convolutional neural networks are effective at extracting local features but struggle to model long-range spatial dependencies [10], which are essential for representing continuous urban structures such as building blocks and street corridors. Transformer-based methods improve global context modeling [11]; however, their patch-based representations often disrupt spatial continuity, leading to fragmented boundaries in high-resolution street scenes [12].

A key limitation of existing approaches is the lack of morphological consistency across scales. During decoding, progressive upsampling tends to smooth high-frequency details, resulting in blurred building edges and discontinuous spatial interfaces [13]. This weakens the structural integrity required for urban form interpretation [14].

In addition, most models rely primarily on spatial-domain feature learning [15]. As a result, high-frequency geometric information such as edges, corners, and façade discontinuities is gradually lost during feature propagation. This further reduces the accuracy of morphological boundary representation.

Another limitation is the lack of explicit modeling of urban structural semantics. Existing segmentation methods mainly optimize pixel-level accuracy, without ensuring that the results preserve meaningful urban elements such as building envelopes, street boundaries, and spatial interfaces [16].

1.3. Scope, Motivation, and Research Objective

To address these limitations, this study shifts the focus from conventional semantic segmentation to computational urban morphology analysis. The objective is not only to improve segmentation accuracy but also to enhance the structural interpretability of urban scenes [17]. Specifically, this work aims to develop a segmentation framework that can simultaneously preserve global semantic consistency and fine-grained morphological structures under complex street-view conditions. This includes maintaining building boundary continuity, capturing spatial hierarchy, and preserving high-frequency structural details across scales [18]. From a methodological perspective, this study introduces a joint spatial–frequency modeling paradigm. By integrating hierarchical feature representation with frequency-aware enhancement, the proposed approach seeks to overcome the limitations of purely spatial-domain methods and improve the morphological fidelity of segmentation results [19]. From an application perspective, this framework provides a computational foundation for urban morphology analysis [20]. It enables more reliable extraction of spatial structures, supporting downstream tasks such as building façade analysis, street enclosure evaluation, and urban form quantification [21].

1.4. Overview and Contributions

To address the limitations identified above, this study proposes a street-view semantic segmentation framework named HieraWaveSeg, which integrates hierarchical Transformer-based representation learning with frequency-domain feature enhancement [22]. The method jointly optimizes spatial semantic modeling and spectral information preservation, aiming to improve structural coherence and boundary fidelity in complex urban environments [23].

Unlike conventional semantic segmentation methods that primarily focus on pixel-level classification accuracy, this study is situated within the broader context of computational urban morphology analysis [24]. The objective is to develop a framework that transforms visual scene understanding into morphology-relevant spatial representations for architectural and urban studies [25].

1.4.1. Method Overview

Specifically, a hierarchical Transformer is used as the backbone encoder to construct multi-scale feature representations through progressive downsampling. This design enables effective modeling of long-range spatial dependencies while preserving structural continuity across scales, thereby improving robustness to scale variations and perspective distortions in street-view imagery [26]. Based on this backbone, a Path Aggregation Network (PAN) is introduced to enable bidirectional feature fusion. By combining top-down semantic propagation with bottom-up detail preservation, PAN enhances cross-scale feature interaction and reduces the semantic gap between high-level representations and low-level geometric details [27]. This is crucial for maintaining structural integrity in dense urban scenes. To further address high-frequency information loss during decoding, a Wave Fusion module based on the Haar wavelet transform is incorporated into the decoder. Instead of relying solely on spatial upsampling, this module decomposes features into frequency sub-bands and selectively enhances high-frequency components through an edge-aware attention mechanism. This design improves the preservation of boundary sharpness and fine-grained structural details, which are often degraded in conventional decoding pipelines. In addition, to handle class imbalance in street-view datasets, a composite loss function (CEDiceLoss) is introduced by combining weighted cross-entropy with Dice loss. A median frequency balancing strategy is adopted to dynamically adjust class weights, improving sensitivity to minority classes and boundary regions while maintaining training stability [28].

1.4.2. Main Contributions

The main contributions of this study are summarized as follows:

(1) A morphology-oriented semantic segmentation framework for urban analysis

This study proposes a segmentation paradigm that extends beyond conventional visual recognition. The outputs are explicitly aligned with spatial-functional units of the urban environment, enabling a transformation from pixel-level predictions to morphology-relevant representations. This provides a computational basis for urban form analysis in architectural and planning applications.

(2) A hierarchical Transformer–PAN architecture for robust multi-scale representation

A hybrid architecture combining a hierarchical Transformer and a Path Aggregation Network is developed to enhance multi-scale feature learning. Experimental results indicate that performance gains are not only due to model capacity but also stem from the synergistic interaction between hierarchical semantic modeling and cross-scale feature fusion.

(3) A frequency-aware Wave Fusion module for boundary preservation

A wavelet-based feature enhancement module is introduced to mitigate high-frequency information loss during decoding. In addition to improving overall segmentation accuracy, it significantly enhances boundary clarity and structural continuity, which are essential for urban morphological representation.

(4) A balanced optimization strategy for imbalanced urban scene data

A composite loss function combining weighted cross-entropy and Dice loss is designed to address class imbalance in street-view datasets. This improves the model’s ability to learn minority classes and enhances robustness in boundary-sensitive regions.

(5) A bridge between computer vision and urban morphology analysis

From an urban and architectural perspective, the proposed framework enables the extraction of morphology-relevant indicators, including spatial enclosure, façade continuity, interface permeability, and proxies for spatial vitality. This establishes a direct link between segmentation outputs and quantitative urban form analysis, providing a computational pathway for integrating computer vision into urban morphology and architectural informatics.

2. Related Work

2.1. Backbone Networks for Semantic Segmentation

Semantic segmentation performance is largely determined by the capability of the backbone network to extract meaningful visual representations from complex urban scenes [29]. Early models extract hierarchical local features and have demonstrated strong performance in various vision tasks [30].

However, convolution-based architectures are inherently constrained by local receptive fields. As a result, they struggle to capture long-range spatial dependencies, which are essential for understanding continuous urban structures such as building façades, street edges, and spatial interfaces [31]. In street-view imagery, where perspective distortion and scale variation are significant, this limitation often leads to fragmented semantic predictions and unstable global structure representation.

More recent Transformer-based models, improve global dependency modeling through self-attention mechanisms [32]. These methods provide more coherent semantic representations in complex scenes. However, patch-based or window-based partitioning may weaken fine-grained spatial continuity, particularly at object boundaries and architectural edges, which are critical for urban morphology interpretation.

In this context, hierarchical Transformers provide a more balanced solution by progressively constructing multi-scale representations. The Hiera Transformer adopted in this study follows this design principle, enabling stable feature learning across different spatial scales while preserving structural consistency under perspective variation [33].

Importantly, the backbone alone is insufficient to fully resolve the trade-off between global semantic consistency and local structural precision [34]. Therefore, additional feature fusion and reconstruction mechanisms are required in the decoding stage.

2.2. Multi-Scale Feature Fusion and Frequency-Aware Representation

Urban street-view segmentation is fundamentally a multi-scale problem due to the coexistence of objects with highly varying spatial extents. Traditional approaches improve performance by enabling feature exchange across scales through top-down and bottom-up pathways [35].

While these spatial-domain fusion strategies improve semantic integration, they do not explicitly preserve structural details during feature reconstruction. In particular, upsampling operations tend to suppress high-frequency information, leading to blurred object boundaries and degraded representation of architectural edges.This limitation is critical in urban morphology-related tasks, where boundary accuracy directly affects downstream analysis such as façade continuity, spatial enclosure, and interface permeability measurement.

To address this issue, recent studies have introduced frequency-domain modeling techniques such as the Discrete Wavelet Transform. By decomposing features into low-frequency components representing global structure and high-frequency components representing edges and textures, wavelet-based methods provide a complementary perspective to spatial feature learning [36].

In this study, we incorporate this idea through a Wave Fusion mechanism, which enhances high-frequency feature preservation during decoding. Unlike purely spatial fusion strategies, this design explicitly strengthens boundary-sensitive information, improving structural consistency in segmentation outputs.

2.3. Urban Street-View Datasets

This study evaluates the proposed method on two widely used benchmark datasets: Cityscapes and CamVid. These datasets are selected due to their complementary characteristics in terms of resolution, scene complexity, and annotation granularity.

Cityscapes provides high-resolution urban street-view images with fine-grained pixel-level annotations across diverse European cities [37]. It is widely used for evaluating segmentation performance in complex urban environments with rich structural details.

CamVid, in contrast, is derived from driving video sequences and contains temporally continuous frames with relatively lower resolution [38]. It is commonly used to assess model stability and generalization under moderately complex urban conditions.

In this study, both datasets are used to ensure a balanced evaluation across different levels of urban complexity. All images are uniformly resized and normalized before training. The datasets are split into training, validation, and test sets to ensure fair and reproducible evaluation.

From a practical perspective, these datasets simulate real-world urban perception scenarios, making them suitable for evaluating models intended for applications in urban analysis, architectural façade recognition, and digital city modeling [39,40].

2.4. Practical Relevance and Target Users

Although the proposed framework is developed within the field of computer vision, its motivation extends beyond algorithmic performance and is closely aligned with urban spatial analysis and architectural informatics. The model is designed to transform pixel-level segmentation results into structurally meaningful urban representations, enabling quantitative interpretation of street-view environments.

From an application perspective, the segmentation outputs can support various urban analysis tasks, including façade continuity assessment, street interface characterization, spatial enclosure evaluation, and extraction of urban morphological indicators [41]. These representations provide a computational basis for understanding how urban spaces are organized and perceived at the pedestrian scale, particularly in complex and heterogeneous environments.

The intended users of this work include not only computer vision researchers but also urban planners, architects, and scholars in urban morphology and spatial analytics. These fields increasingly require data-driven methods that connect visual perception with spatial interpretation. The proposed framework facilitates this connection by providing structurally consistent segmentation outputs.

In this sense, the contribution of this study lies not only in improved segmentation accuracy but also in enhancing the interpretability and usability of results for downstream urban and architectural applications.

3. Methods

3.1. Experimental Data

3.1.1. Dataset Selection and Characteristics

This study uses two publicly available street-view semantic segmentation datasets: CamVid and Cityscapes. These datasets are widely adopted in urban scene understanding and provide complementary characteristics for evaluating model performance under different levels of urban complexity [42,43].

CamVid is derived from driving video sequences and provides temporally continuous street scenes. It contains relatively low-resolution images and a limited number of annotated samples [44]. As a result, it is suitable for evaluating model stability and basic semantic representation capability in moderately structured urban environments.

In contrast, Cityscapes consists of high-resolution street-view images collected from multiple European cities. It provides fine-grained pixel-level annotations and a rich set of urban categories, including buildings, roads, vegetation, pedestrians, and traffic infrastructure [45]. This dataset is commonly used to evaluate fine-grained segmentation accuracy and structural generalization in complex urban scenes.

From an urban morphology perspective, these two datasets cover a spectrum of spatial complexity, ranging from simple street layouts to dense and heterogeneous urban fabrics. This enables a systematic evaluation of the model’s ability to preserve building boundaries, street interfaces, and spatial continuity across different urban forms [46].

Unlike remote sensing datasets that primarily capture top-down spatial patterns, street-view datasets represent façade-level urban morphology from a pedestrian perspective. Therefore, they are particularly suitable for architectural and urban analysis, where building interfaces and street-scale spatial relationships are key structural elements.

3.1.2. Urban Morphology-Based Semantic Reconfiguration

To enhance the interpretability of semantic segmentation results for architectural and urban spatial analysis, this study does not directly adopt the original fine-grained annotation systems of the CamVid and Cityscapes datasets. Instead, a theory-driven semantic reconfiguration strategy is proposed, grounded in principles of Urban Morphology and spatial behavior theory.

Unlike conventional category systems designed primarily for visual recognition, the proposed reconfiguration aims to establish an explicit linkage between pixel-level semantic labels and spatial morphological units [47]. This transformation enables segmentation outputs to serve not merely as classification results, but as measurable representations of urban spatial structure.

As shown in Figure 2, the original labels are reorganized into six function-oriented categories, each corresponding to a distinct morphological component within the urban spatial system:

(1) Built Structure and Spatial Enclosure Elements (Build)

This category includes buildings, bridges, tunnels, walls, and fences. These elements collectively constitute the street wall system, which is a fundamental concept in urban morphology. They define spatial enclosure, continuity of façades, and the degree of visual and physical containment of public space.

Although typologically distinct elements are aggregated, they share a dominant morphological function—spatial enclosure. From the perspective of urban spatial analysis, this functional abstraction preserves the essential structural role of these elements while reducing semantic fragmentation [48]. It enables the derivation of key indicators such as enclosure ratio, façade continuity, and skyline definition, which are more directly relevant to morphological analysis than fine-grained typological distinctions.

(2) Primary Transportation Space (Main Road)

This category includes vehicular roads and lane markings, representing the primary spatial skeleton of urban environments. It corresponds to the main axis of spatial accessibility and movement flow.

From a morphological perspective, this layer supports the quantification of connectivity, spatial hierarchy, and accessibility patterns. Its integration into a unified category facilitates the extraction of continuous spatial networks, which are essential for analyzing urban structure and mobility systems [49].

(3) Street Interface Appendages (Road Append)

This category encompasses sidewalks, traffic signals, signage, and street furniture. These elements form the human-scale interface layer between built structures and movement spaces.

In architectural theory, such components are often described as “soft edges,” mediating interactions between pedestrians and the built environment. Their aggregation enables the analysis of interface permeability, pedestrian experience, and spatial interaction intensity [31], which are difficult to capture through fragmented object-level labels.

(4) Natural Environment and Spatial Background (Natural)

This category includes vegetation, trees, sky, and terrain. These elements constitute the ecological and visual background of urban space.

From a spatial analysis perspective, this category supports the assessment of openness, environmental quality, and visual continuity [50]. It provides a complementary layer to built structures, enabling the joint analysis of built–natural relationships in urban morphology.

(5) Dynamic Agents (Dynamic)

This category includes vehicles, pedestrians, and cyclists, representing mobile entities within urban space.

Rather than treating these elements as isolated object categories, they are conceptualized as indicators of spatial usage and activity intensity. This abstraction allows segmentation outputs to be interpreted as proxies for urban vitality, facilitating the integration of spatial morphology with behavioral dynamics [51].

(6) Background and Non-structural Regions (Background)

This category includes semantically ambiguous or non-critical regions. It serves to reduce noise and improve the stability of structural boundary learning.

From a modeling perspective, isolating such regions prevents interference with the extraction of meaningful spatial components, thereby enhancing the clarity and reliability of morphological representations [52].

A key distinction of the proposed reconfiguration lies in its ability to support quantitative urban morphology analysis. Each category is not only a semantic label but also corresponds to a measurable spatial attribute (

Table 1. Mapping between urban semantic categories and morphological indicators.

Semantic Category	Morphological Function	Quantitative Indicators
Build	Spatial enclosure system	Enclosure ratio; Façade continuity; Skyline structure
Main Road	Spatial accessibility framework	Accessibility; Spatial hierarchy; Network connectivity
Road Append	Human-scale interface system	Interface permeability; Pedestrian interaction intensity
Natural	Environmental and visual background	Spatial openness; Environmental quality
Dynamic	Activity and behavioral layer	Urban vitality; Activity intensity

):

Through this mapping, segmentation outputs can be directly transformed into morphology-related metrics, enabling a systematic transition from visual recognition to spatial quantification.

In contrast, the original dataset annotations, while rich in semantic detail, are primarily designed for object recognition tasks. Their fine-grained but fragmented structure lacks explicit correspondence to spatial analytical units, which limits their applicability in integrated urban morphology analysis.

The proposed semantic reconfiguration is motivated by three considerations. First, from a theoretical perspective, it aligns computer vision outputs with the analytical framework of urban morphology by reorganizing visual categories into spatial-functional units.

Second, from a data-driven perspective, it alleviates the long-tail distribution problem in street-view datasets. Aggregating semantically related categories, it improves class balance and enhances training stability.

Third, from an application perspective, it improves the usability of segmentation results in urban and architectural analysis tasks, including street interface evaluation, façade morphology analysis, and spatial structure quantification.

In summary, the proposed semantic reconfiguration is not a performance-driven simplification, but a theory-guided structural abstraction. It bridges computer vision and urban spatial analysis by transforming pixel-level classification into a morphology-aware representation (

Table 2. Pixel proportion statistics of semantic elements based on urban morphology.

Semantic Category	Corresponding Spatial Function Description	CamVid Subclasses	CamVid Proportion (%)	Cityscapes Subclasses	Cityscapes Proportion (%)	RGB
Build	“Hard Edge” and physical enclosure framework of urban space	Building, Bridge, Tunnel, Archway, Wall, Column	26.635%	Building, Fence, Wall, Bridge, Tunnel	14.479%	(255, 0, 0) Red
Main Road	Fundamental framework and flow base of horizontal urban space	Road, LaneMkgsDriv, LaneMkgsNonDriv	29.004%	road	21.5786%	(0, 0, 255) Blue
Road Append	Human scale elements and “Soft Edge” of street interfaces	TrafficLight, TrafficCone, SignSymbol, Misc_Sign	8.023%	Sidewalk, Pole, Parking, Traffic sign	5.4542%	(255, 255, 0) Yellow
Natural	Ecological regulation and visual background of the built environment	Tree, VegetationMisc, Sky	27.050%	Vegetation, Sky, Terrain, Ground	13.1537%	(0, 255, 0) Green
Dynamic	Indicators of activity intensity and urban vitality	Pedestrian, Child, Bicyclist, Animal, Car, SUV	6.278%	Car, Person, Bicycle, Dynamic, Truck	5.6598%	(255, 0, 255) Magenta
Background	Semantically neutral areas and non-structural boundaries	Void	3.010%	static	39.6738%	(0, 0, 0) Black
Total	Full coverage of urban street-view elements	all	100.000%	all	100.000%	-

Note: The RGB values and corresponding color labels indicate the visualization colors assigned to each semantic category in the segmentation maps.

).

This enables semantic segmentation to function not only as a perception task, but also as a foundational tool for quantitative urban form analysis.

3.1.3. Dataset Limitations and Preprocessing Considerations

Both datasets are divided into training, validation, and test sets in a 60%, 30%, and 10% split, respectively. This configuration emphasizes validation coverage to ensure reliable assessment of model generalization under complex urban conditions.

All images are resized to a uniform resolution of 640 × 640 pixels to maintain a consistent spatial scale. No dataset-specific augmentation is applied, ensuring that performance differences are primarily attributable to model design rather than data-related bias.

From an urban analysis perspective, this standardized preprocessing protocol improves the reliability of comparative evaluation. It also ensures that segmentation outputs can be consistently interpreted in terms of urban morphological structures, such as building boundaries, street interfaces, and spatial continuity.

3.2. Model Architecture

HieraWaveSeg is a semantic segmentation framework designed for street-view urban scene understanding, with an emphasis on preserving morphological structure. The model follows a hierarchical encoder–decoder architecture with multi-scale feature fusion and frequency-aware refinement [53].

The encoder employs a hierarchical Transformer backbone to extract multi-scale contextual representations. This design captures both global spatial dependencies and local structural details [54], which is essential for modeling urban scenes with strong perspective distortion and scale variation.

To enhance cross-scale feature interaction, a Path Aggregation Network (PAN) is incorporated. PAN combines top-down semantic propagation with bottom-up spatial refinement, enabling effective alignment between high-level semantic features and low-level geometric details [27].

During decoding, a Wave Fusion module is introduced to enhance high-frequency structural information. The module decomposes features into frequency components and selectively strengthens edge-related signals. This is critical for preserving building boundaries and spatial interfaces, which are key elements in urban morphology analysis [55].

Overall, the proposed architecture is designed not only to improve pixel-level segmentation accuracy but also to maintain structural consistency in urban form representation (Figure 3).

3.3. Fusion Module Architecture

The Path Aggregation Network serves as the core multi-scale fusion component. It constructs bidirectional feature pathways to integrate semantic and spatial information across different resolutions [56]. As illustrated in Figure 4.

In the top-down pathway, high-level semantic features are progressively upsampled and fused with lower-level features. In the bottom-up pathway, spatial details are reintroduced into deeper layers through downsampling operations. This bidirectional design ensures balanced representation of global context and local structure.

From an urban morphology perspective, this fusion strategy helps preserve continuity of building façades and improves the representation of spatial interfaces between different urban elements [57].

3.4. Wave Fusion Module

The Wave Fusion module enhances feature representations in the frequency domain. It applies the Discrete Wavelet Transform (DWT) to decompose feature maps into low-frequency and high-frequency components [58]. High-frequency components are critical for representing edges, boundaries, and structural transitions in urban scenes (Figure 5).

Unlike conventional fusion methods that operate purely in the spatial domain, this module explicitly preserves structural information that is often lost during upsampling. As a result, it improves boundary sharpness and reduces feature blurring in complex urban environments.

The module integrates frequency-aware attention with residual learning, enabling adaptive enhancement of structurally informative regions, such as building edges and road boundaries. This improves the preservation of morphological continuity, which is essential for reliable urban form interpretation.

The DWT is implemented using the PyWavelets (pywt) library, allowing a differentiable 2D wavelet transform within an end-to-end framework. Using the Haar wavelet as the basis, the transform is constructed from tensor products of 1D filters.

For the Haar wavelet, the 1D low-pass decomposition filter (L) and high-pass decomposition filter (H) are defined as follows:

$$L={\displaystyle \frac{1}{\sqrt{2}}}[1,1],H={\displaystyle \frac{1}{\sqrt{2}}}[1,-1]$$

(1)

The corresponding reconstruction filters are $\tilde{L}=L^{\top }$ and $\tilde{H}=H^{\top }$.

Based on the separability of 2D wavelet transforms, four 2D convolution kernels are constructed via the tensor product of 1D filter sequences ${\mathbf{W}}_{\omega }$, where $\omega \in \{LL,LH,HL,HH\}$:

$$\left\{\begin{array}{c}{\mathbf{W}}_{LL}=L^{\top }\otimes L={\displaystyle \frac{1}{2}}\left[\begin{array}{cc}1& 1\\ 1& 1\end{array}\right],\left(Approximation\right)\hfill \\ {\mathbf{W}}_{LH}=L^{\top }\otimes H={\displaystyle \frac{1}{2}}\left[\begin{array}{cc}1& -1\\ 1& -1\end{array}\right],\left(HorizontalDetail\right)\hfill \\ {\mathbf{W}}_{HL}=H^{\top }\otimes L={\displaystyle \frac{1}{2}}\left[\begin{array}{cc}1& 1\\ -1& -1\end{array}\right],\left(VerticalDetail\right)\hfill \\ {\mathbf{W}}_{HH}=H^{\top }\otimes H={\displaystyle \frac{1}{2}}\left[\begin{array}{cc}1& -1\\ -1& 1\end{array}\right],\left(DiagonalDetail\right)\hfill \end{array}\right.$$

(2)

For an input feature map $\mathbf{X}\in {\mathbb{R}}^{C\times H\times W}$, the DWT process is implemented through grouped convolution with a stride of 2, decoupling spatial information into four sub-band components. The transformation formula for the C-th channel is:

$$y_{\omega }(c,i,j)={\displaystyle \sum _{m=0}^1}{\displaystyle \sum _{n=0}^1}X(c,2i+m,2j+n)·W_{\omega }(m,n)$$

(3)

The final output feature ${\mathbf{Y}}_{DWT}$ is formed by concatenating the sub-bands:

$${\mathbf{Y}}_{DWT}=\mathrm{Concat}(LL,LH,HL,HH)\in {\mathbb{R}}^{4C\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}}$$

(4)

where $LL$ captures the approximate background, while $\{LH,HL,HH\}$ capture gradient details in horizontal, vertical, and diagonal directions, respectively.

3.5. Loss Function

The HieraWaveSeg model employs a composite loss function, termed CEDiceLoss, designed for imbalanced semantic segmentation in street-view imagery. It combines weighted cross-entropy loss with Dice loss to jointly optimize pixel-level accuracy and region-level overlap.

In semantic segmentation, treating each pixel independently often biases the model toward dominant classes. To address this issue, a weighted cross-entropy loss is adopted:

$${\mathcal{L}}_{CE}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{w}_{y_i}log\left({\widehat{p}}_{i,y_i}\right)$$

(5)

where N is the total number of pixels in a batch, $y_i$ is the ground-truth label of the i-th pixel, and ${\widehat{p}}_{i,y_i}$ is the predicted probability of the i-th pixel belonging to class $y_i$ (obtained via softmax). $w_y$ is the weight for the corresponding class used for differential penalization. Instead of using simple inverse class frequency, this study employs a robust median-based weight normalization strategy:

Let the prior probabilities (pixel frequencies) of each class in the dataset be $\mathbf{p}=[p_1,p_2,\dots ,p_C]$. First, calculate the median m of this list to determine the raw weights:

$$w_c^{\prime }={\displaystyle \frac{m}{p_c}},\forall c=1\dots C$$

(6)

Then, normalize these weights to ensure a mean of 1:

$$w_c={\displaystyle \frac{w_c^{\prime }}{\frac{1}{C}{\sum }_{i=1}^C{w}_i^{\prime }}}$$

(7)

This median normalization strategy effectively mitigates weight explosion from extremely rare classes while maintaining class balance.

To further improve the optimization of regional overlap, a weighted Dice loss is incorporated. This loss optimizes based on the overlap between the predicted probability map and the ground-truth labels, addressing boundary blurring.

For the Dice loss, the network output feature map $X\in {\mathbb{R}}^{B\times C\times H\times W}$ is converted into a probability distribution $P\in {\mathbb{R}}^{B\times C\times H\times W}$ via softmax, and ground-truth labels are converted into one-hot encoding $Y\in {\mathbb{R}}^{B\times C\times H\times W}$. The Dice loss for each category is:

$$Dic{e}_c=1-{\displaystyle \frac{2{\sum }_{b,h,w}(P_{b,c,h,w}·Y_{b,c,h,w})+ϵ}{{\sum }_{b,h,w}(P_{b,c,h,w}+Y_{b,c,h,w})+ϵ}}$$

(8)

where $ϵ={10}^{-6}$ ensures numerical stability. Finally, the weighted Dice loss is calculated using the same class weights $w_c$:

$${\mathcal{L}}_{Dice}={\displaystyle \frac{{\sum }_{c=1}^C{w}_c·Dic{e}_c}{{\sum }_{c=1}^C{w}_c}}$$

(9)

The total loss function merges the weighted cross-entropy and weighted Dice losses:

$${\mathcal{L}}_{CEDice}={\lambda }_{CE}·{\mathcal{L}}_{CE}+{\lambda }_{Dice}·{\mathcal{L}}_{Dice}$$

(10)

This hybrid loss function maintains the precise pixel-level supervision of cross-entropy while optimizing regional overlap via Dice loss. Using a unified median frequency balancing strategy, it enhances the model’s ability to learn minority classes in street-view architectural segmentation.

3.6. Evaluation Metrics

Model performance is evaluated using Overall Accuracy (OA), mean Intersection over Union (mIoU), Precision, Recall, and F1-score. These metrics provide complementary perspectives on segmentation quality.

Overall Accuracy measures global prediction correctness, while mIoU evaluates spatial overlap at the class level. Precision and Recall assess classification reliability and detection completeness, respectively. The F1-score, defined as the harmonic mean of Precision and Recall, provides a balanced evaluation.

From an urban morphology perspective, these metrics also reflect structural fidelity and boundary consistency. Higher mIoU and F1-score values indicate improved preservation of building interfaces and spatial continuity.

The metrics are defined as follows:

Overall Accuracy ($OA$):

$$OA={\displaystyle \frac{{\sum }_{c=1}^{C}T{P}_c}{{\sum }_{c=1}^C(T{P}_c+F{P}_c+T{N}_c+F{N}_c)}}$$

Intersection over Union ($IoU$/$mIoU$):

$$mIoU={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{T{P}_c}{T{P}_c+F{P}_c+F{N}_c}}$$

Precision:

$$\mathrm{Precision}={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{T{P}_c}{T{P}_c+F{P}_c}}$$

Recall:

$$\mathrm{Recall}={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{T{P}_c}{T{P}_c+F{N}_c}}$$

F1-score: The F1-score is defined as the harmonic mean of Precision and Recall, and it serves as a key metric for evaluating the overall performance of the model in positive sample recognition.

$$F1=2·{\displaystyle \frac{\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

3.7. Methodological Novelty Clarification

The proposed HieraWaveSeg framework differs from existing semantic segmentation methods in three key aspects.

First, unlike conventional CNN-based methods such as FPN and PAN, which rely mainly on spatial-domain feature aggregation, the proposed method introduces a hierarchical Transformer backbone. This design improves global context modeling and enables more stable representation of long-range spatial dependencies in complex urban scenes.

Second, compared with Transformer-based models such as SegFormer and Swin Transformer, which primarily focus on efficient global attention modeling, HieraWaveSeg further integrates a bidirectional Path Aggregation Network (PAN). This module explicitly strengthens cross-scale interaction between semantic abstraction and spatial detail, improving structural consistency in high-resolution street-view imagery.

Third, in contrast to existing wavelet-based approaches that typically apply frequency-domain processing after feature fusion or upsampling, the proposed Wave Fusion module integrates frequency-aware enhancement directly into the decoding process. This allows high-frequency information to actively guide feature reconstruction rather than serving as a post-processing refinement step.

Overall, the novelty of this work lies in the unified integration of hierarchical global modeling, bidirectional multi-scale feature fusion, and frequency-aware decoding within a single architecture tailored for urban street-view segmentation.

To ensure reproducibility, all experiments follow a unified preprocessing pipeline, training configuration, and evaluation protocol. Dataset usage, model architecture, and hyperparameter settings are fully specified in the methodology section. All reported results are obtained under identical experimental conditions, ensuring that performance differences are attributable to model design rather than data variation or training bias.

4. Results

To comprehensively evaluate the effectiveness of HieraWaveSeg, extensive comparative experiments and ablation studies are conducted. In the comparative experiments, the proposed method is evaluated against state-of-the-art semantic segmentation models. In addition, learning curve analysis is used to assess convergence behavior and training stability.

For component-level evaluation, the Hiera Transformer backbone is analyzed to quantify its advantage over conventional CNN-based backbones. The proposed Path Aggregation Network and frequency-aware decoder are further validated through controlled comparisons with representative designs. The effectiveness of the composite loss function is also examined under different weighting strategies.

In the ablation studies, the contributions of the fusion module and decoder components are systematically evaluated. Furthermore, feature visualization and activation map analysis are conducted to examine the evolution of feature representations across different stages. These analyses provide intuitive evidence that the model can effectively capture fine-grained semantic structures and complex spatial boundaries in urban street scenes.

4.1. Experimental Setup

All experiments are conducted on a unified computing platform. In terms of hardware configuration, the server is equipped with an Intel Core i7-12700 processor and an NVIDIA GeForce RTX 4090 GPU (24 GB VRAM), providing high-performance parallel computing capability.

In terms of the software environment, the operating system is Ubuntu 24.04.3 LTS (Linux kernel 6.8.0). The implementation is developed using Python 3.12.3, and the deep learning framework is PyTorch 2.9.0, accelerated by CUDA 13.0 to ensure efficient training and inference of deep neural networks (

Table 3. Experimental environment and hardware–software configuration.

Experimental Setting	Details
CPU	Intel(R) Xeon(R) Gold 6346 CPU @ 3.10 GHz
GPU	NVIDIA A40 ×2
Operating System	Linux 6.8.0-101-generic 22.04.1-Ubuntu x86-64
Python Version	Python 3.10.12
Pytorch Version	PyTorch 2.8.0+cu128
CUDA Version	CUDA 12.2

).

To ensure reproducibility and fair comparison, all experiments follow a unified training and evaluation protocol.

Experiments are conducted on two benchmark datasets: Cityscapes and CamVid. Both datasets are split into training, validation, and test sets with a ratio of 60%, 30%, and 10%, respectively. All images are resized to 640 × 640 pixels and normalized using dataset-specific mean and standard deviation values computed from the training set. No additional dataset-specific augmentation strategies are applied to avoid introducing evaluation bias.

During training, the AdamW optimizer is used with an initial learning rate of 1 × 10⁻⁴. A polynomial learning rate decay strategy with power 0.9 is applied. The batch size is set to 8, and the total number of epochs is 200. Weight decay is set to 0.01 to reduce overfitting. All models are implemented under the PyTorch framework and trained on NVIDIA RTX 3090 GPUs with 24 GB memory.

For baseline comparison, representative methods including U-Net, DeepLabV3+, SegFormer, and PIDNet are reproduced using official or widely adopted implementations. All baselines are trained under identical data splits, preprocessing steps, and training settings to ensure a fair and consistent evaluation (

Table 4. Training configuration and dataset preparation.

Training Setting	Details
Learning Rate	0.00001
Total Epochs	200
Weight Decay	0.005
Input Dimensions	640 × 640
Training Set Split	60%
Validation Set Split	30%
Test Set Split	10%

).

4.2. Comparative Results

To further validate the effectiveness of the proposed model for semantic segmentation, comprehensive quantitative comparison experiments are conducted on the re-annotated 6-class CamVid and Cityscapes datasets for building-related scene understanding. Several representative segmentation models are selected for comparison, including DeepLabV3Plus, SegFormer, U-Net, and PIDNet, together with the proposed HieraWaveSeg.

Among them, DeepLabV3Plus is a classical atrous convolution-based semantic segmentation model, which effectively enlarges the receptive field while preserving spatial details through atrous convolution. SegFormer integrates Transformer architecture with a lightweight decoder design, enabling better modeling of global contextual dependencies and multi-scale features. U-Net is a classical symmetric encoder–decoder architecture that effectively fuses low-level spatial details with high-level semantic information via skip connections.

To enable a more comprehensive comparison, the above three baseline models were combined with different backbone networks (including ResNet101, ResNeXt101, MiT-B3, and Hiera_transformer_tiny), resulting in a total of 12 comparison configurations. In addition, PIDNet is specifically designed for real-time semantic segmentation and employs PIDNet-L as its backbone, achieving a strong balance between accuracy and efficiency through a parallel-branch architecture. To ensure fairness and convincingness in performance evaluation, none of the compared models used pre-trained weights, and their parameter counts were carefully configured to remain at a comparable level with HieraWaveSeg.The results are reported in

Table 5. Performance comparison of different models on Camvid and Cityscapes datasets.

Models	Backbone	Accuracy	Precision	Recall	F1-Score	IoU	Params
Camvid dataset
DeepLabV3Plus	resnet101	82.15	68.33	72.53	69.79	57.46	45.78 M
	resnext101	79.73	66.22	71.11	67.92	55.01	45.44 M
	mit_b3	86.83	74.66	76.79	75.40	64.24	45.23 M
	Hiera_tiny	86.45	74.22	77.13	75.24	64.96	39.25 M
Segformer	resnet101	80.44	66.18	70.45	67.70	55.29	43.94 M
	resnext101	78.01	63.52	67.19	64.73	51.96	43.60 M
	mit_b3	86.06	73.32	75.47	74.09	62.74	44.60 M
	Hiera_tiny	86.62	73.25	75.45	74.96	62.25	38.85 M
Unet	resnet101	75.59	64.65	66.38	63.00	49.96	51.62 M
	resnext101	73.40	60.55	65.48	61.48	48.53	51.28 M
	mit_b3	82.55	69.08	75.02	70.76	58.79	47.36 M
	Hiera_tiny	82.21	68.56	75.45	69.82	58.62	41.75 M
PIDNet	PIDNet-L	74.75	61.23	66.05	62.49	49.18	36.99 M
HieraWaveSeg	Hiera-transformer_tiny	91.60	81.86	83.11	81.97	73.15	44.49 M
Cityscapes dataset
DeepLabV3Plus	resnet101	86.06	82.90	82.94	82.49	71.38	45.78 M
	resnext101	85.27	82.18	82.10	81.63	70.21	45.44 M
	mit_b3	88.23	85.61	85.22	84.09	75.05	45.23 M
	Hiera_tiny	88.56	86.24	85.64	85.26	75.29	39.25 M
Segformer	resnet101	85.03	81.78	81.35	80.93	69.22	43.94 M
	resnext101	84.34	81.02	80.54	80.07	68.01	43.60 M
	mit_b3	87.96	85.07	84.80	84.55	74.25	44.60 M
	Hiera_tiny	87.89	85.34	84.14	84.10	74.55	38.85 M
Unet	resnet101	85.94	82.88	82.47	82.12	70.85	51.62 M
	resnext101	85.22	81.89	81.71	81.25	69.64	51.28 M
	mit_b3	88.21	85.61	85.04	85.04	74.99	47.36 M
	Hiera_tiny	88.25	84.11	85.97	85.26	74.49	41.75 M
PIDNet	PIDNet-L	77.81	74.48	73.35	72.67	58.59	36.99 M
HieraWaveSeg	Hiera-transformer_tiny	89.84	87.51	87.04	86.96	77.79	44.49 M

Note: Bold values indicate the best performance results among the compared models on each dataset.

.

On the CamVid dataset, HieraWaveSeg achieved the best overall performance. Specifically, the model attained an Accuracy of 91.60%, Precision of 81.86%, Recall of 83.11%, F1-score of 81.97%, and IoU of 73.15%, significantly outperforming all other comparison methods across every evaluation metric. On the larger Cityscapes dataset, HieraWaveSeg once again delivered outstanding performance, achieving the best results across all metrics, with Accuracy, Precision, Recall, F1-score, and IoU reaching 89.84%, 87.51%, 87.04%, 86.96%, and 77.79%, respectively.

Further analysis reveals that DeepLabV3+, SegFormer, and U-Net all achieved superior performance with lower parameter counts when using Hiera_tiny as the backbone network compared to other backbones. This demonstrates that the feature extraction capability of Hiera_tiny is substantially stronger than that of traditional convolutional networks. Among them, DeepLabV3+ with Hiera_tiny emerged as the second-best model after HieraWaveSeg on key metrics such as mIoU and F1-score. Compared with it, HieraWaveSeg improved the F1-score and IoU by 6.73% and 8.19% respectively on the CamVid dataset, and by 1.70% and 2.50% respectively on the Cityscapes dataset. Moreover, HieraWaveSeg contains only 44.49 M parameters, achieving an excellent balance between high performance and model efficiency. This further validates the model’s strong capability in capturing fine-grained structural details and accurately delineating object boundaries in complex urban scenes.

In this study, learning curves of different segmentation algorithms are plotted on the re-annotated CamVid and Cityscapes datasets, showing the relationship between training epochs and F1-score, IoU, and training loss, as illustrated in Figure 6. The results systematically evaluate model performance and convergence behavior. All metrics exhibit a consistent increasing trend and gradually stabilize as training progresses, indicating continuous optimization and eventual convergence.

Overall, the black bold curve representing the proposed HieraWaveSeg consistently outperforms all comparison models across all evaluation metrics. In both the F1-score and IoU plots on the two datasets, HieraWaveSeg not only achieves the highest final performance but also exhibits the steepest slope in the early training stage. This indicates a faster convergence speed and stronger feature learning capability. In contrast, classical architectures such as U-Net, DeepLabV3+, and Transformer-based SegFormer (with different backbones including ResNet, ResNeXt, and MiT) also show stable performance improvements during training; however, they remain clearly inferior to HieraWaveSeg in terms of final accuracy.

On the Cityscapes dataset, the performance curves of different methods are relatively close, indicating high prediction consistency across models. In contrast, on the more complex or distribution-variant CamVid dataset, the performance gaps between methods become more pronounced. HieraWaveSeg maintains a clear leading advantage on CamVid and demonstrates a smoother and continuously increasing IoU curve, which further confirms its strong robustness and generalization ability across different visual environments.

The loss curves on the right provide an intuitive view of the optimization process. Most models show a rapid decrease in loss within the first 50 epochs, followed by a plateau. Notably, PIDNet exhibits a pronounced spike at the early training stage before gradually decreasing, suggesting potential instability during optimization. In contrast, HieraWaveSeg consistently maintains the lowest loss values and shows the smoothest decreasing trajectory. This suggests that its loss design or parameter initialization strategy is more effective, enabling the model to converge toward a better global optimum more efficiently.

4.3. Comparative Experimental Results and Analysis

To comprehensively evaluate the effectiveness of HieraWaveSeg, we conduct systematic ablation studies from three perspectives: backbone consistency analysis, decoder-level comparison, and module contribution evaluation.

To ensure fairness, all experiments are conducted under a controlled variable setting. The Hiera Transformer backbone is fixed across all variants, so performance differences can be attributed to decoder design and feature fusion modules rather than variations in feature extraction capability.

To avoid bias introduced by heterogeneous backbone settings in previous studies, backbone evaluation is reported separately in Section 4.3.1, while all ablation experiments in this section are conducted under identical backbone conditions.

All experiments are performed on the re-partitioned CamVid dataset. Its moderate size and diverse urban scenes make it suitable for evaluating architectural differences under controlled computational conditions. This ensures stable and reliable assessment of model robustness in street-scene segmentation.

To further validate the proposed Wave Fusion module, decoder-level comparisons are conducted between the SegFormer decoder and the proposed HieraWaveSeg decoder under the same backbone. This ensures that performance improvements are driven by architectural design rather than backbone differences.

4.3.1. Backbone Network Comparison Experiments

The backbone network is a critical component in semantic segmentation, as it determines feature extraction capability and directly influences the representation of complex urban scenes.

To evaluate feature modeling ability, we conduct comparative experiments on the re-partitioned CamVid dataset using representative CNN and Transformer backbones.

CNN-based models include ResNet and ResNeXt, which are widely used for local feature extraction due to residual learning and grouped convolution. EfficientNet is included as a lightweight baseline that improves efficiency through compound scaling of depth, width, and resolution.

Transformer-based models include Swin Transformer, MViTv2, and the Hiera Transformer. Swin Transformer models long-range dependencies using shifted window attention. MViTv2 improves hierarchical representation through multi-scale attention pooling. The Hiera Transformer adopts a simplified hierarchical design that enhances cross-scale feature interaction and improves structural representation in urban scenes.

All backbones are configured with comparable parameter scales to ensure fair comparison. The results are reported in

Table 6. Quantitative analysis and multi-metric evaluation of various backbone networks on selected benchmarks.

Backbone	Accuracy	Precision	Recall	F1-Score	IoU	Params
Efficientnet_b5	83.62	70.21	74.32	71.69	59.97	35.29 M
Mvitv2_tiny	91.55	81.38	82.48	81.87	72.87	37.98 M
Resnet50	89.40	77.47	79.91	78.56	68.74	48.13 M
Resnext50	89.25	77.31	79.75	78.4	68.43	47.62 M
Swin-Transformer_tiny	89.40	78.02	80.15	78.95	68.98	41.94 M
Hiera-Transformer_tiny	91.60	81.86	83.11	81.97	73.15	44.49 M

Note: Bold values indicate the best performance results among the compared backbone models.

.

Among all models, the Hiera Transformer achieves the best overall performance, with 91.60% Accuracy and 73.15% IoU.

CNN-based models such as ResNet50 and ResNeXt50 achieve lower IoU scores of 68.74% and 68.43%, respectively, indicating limited ability to capture global context in complex urban scenes.

Transformer-based models perform better overall. MViTv2 achieves an IoU of 72.87%, close to the Hiera Transformer, indicating the effectiveness of hierarchical attention mechanisms. Swin Transformer achieves 68.98% IoU, which is lower than both MViTv2 and Hiera Transformer, suggesting limited global structural consistency due to window-based attention.From an efficiency perspective, EfficientNet has the smallest model size (35.29 M parameters) but the lowest IoU (59.97%), indicating that excessive model compression reduces semantic representation capacity in complex urban scenes.The Hiera Transformer uses 44.49 M parameters, comparable to other Transformer-based models, while achieving the best performance overall.

Overall, the Hiera Transformer demonstrates strong feature representation capability and stable performance across metrics. Its hierarchical design makes it well suited for urban street-scene segmentation.

To further analyze feature representation behavior, Class Activation Mapping (CAM) is applied across multiple spatial scales (160 × 160 to 20 × 20).This analysis focuses on representation behavior rather than direct performance comparison (Figure 7).

CNN-based models (ResNet50, ResNeXt50) show strong local feature extraction at high resolution. However, their activation maps become increasingly scattered at lower resolutions, indicating limited ability to maintain global semantic consistency.

Swin Transformer shows more globally distributed activation in early layers, but this often introduces background interference and reduces localization precision in later stages.

In contrast, the Hiera Transformer produces more stable and structured activation patterns across all scales. At low resolutions (40 × 40 and 20 × 20), attention is concentrated on semantically meaningful regions such as building facades, road structures, and spatial boundaries. Activation aligns closely with object contours, indicating strong structural consistency.

Compared with MViTv2, which also shows strong global modeling ability, the Hiera Transformer exhibits more stable cross-scale feature continuity, particularly in boundary-sensitive regions.

EfficientNet produces weak and fragmented activation across all scales, reflecting limited structural modeling ability under complex spatial variation.

From an urban morphology perspective, the Hiera Transformer captures not only object-level semantics but also structural spatial patterns, including street boundaries, building envelopes, and spatial enclosure systems.

This indicates that the model learns higher-level spatial organization rather than isolated pixel-level features. In morphological terms, it demonstrates stronger sensitivity to structural continuity and interface integrity, which are essential for street-level spatial analysis.

The visualization results are consistent with quantitative findings. The Hiera Transformer produces more coherent and structurally aligned feature activations across scales, enabling more accurate representation of complex urban scenes.

4.3.2. Comparative Experiments on Feature Fusion Networks

The Path Aggregation Network is a key component of HieraWaveSeg, designed to enhance multi-scale feature representation in urban street scenes. It reduces the semantic gap between high-level contextual features and low-level spatial details by enabling bidirectional information flow in both top-down and bottom-up directions. This design improves semantic completeness, spatial consistency, and structural integrity of learned features.

To ensure a fair evaluation, all experiments in this section are conducted using a fixed Hiera Transformer backbone. This controlled setting removes the influence of backbone capacity, ensuring that performance differences are solely attributed to the feature fusion module. Backbone-related analysis is separately reported in Section 4.3.1 to avoid redundancy and bias.The proposed fusion module is evaluated on the re-partitioned CamVid dataset against several state-of-the-art methods, including PANet, ASFF, ChannelMapper, UniFusion, and GD_Fusion.The experimental results are presented in

Table 7. Performance comparison of different feature fusion networks on the re-partitioned CamVid dataset.

NeckFusion	Accuracy	Precision	Recall	F1-Score	IoU	Params
ChannelMapper WithPooling	90.27	78.86	81.18	79.90	70.39	35.96 M
GDFusion	90.49	79.22	81.44	80.23	70.80	36.81 M
ASFF	90.83	80.09	81.57	80.76	71.40	38.28 M
PANet	90.68	79.84	81.68	80.67	71.19	41.23 M
UniFusion	90.80	79.82	81.95	80.77	71.41	38.34 M
PAN (Ours)	91.60	81.86	83.11	81.97	73.15	44.49 M

.

PANet enhances hierarchical localization through bidirectional feature pathways. ASFF adaptively fuses multi-scale features using learned spatial weights. ChannelMapper performs lightweight channel-level alignment. UniFusion introduces distribution-aware global fusion, while GD_Fusion models global dependency-guided aggregation.

As shown in Table 3, the proposed method achieves the best overall performance, with 91.60% Accuracy, 81.86% Precision, 83.11% Recall, 81.97% F1-score, and 73.15% IoU. It outperforms UniFusion and ASFF, which achieve 71.41% and 71.40% IoU, respectively. Notably, it is the only method exceeding 81% F1-score, indicating stronger structural consistency in pixel-level prediction.

In addition, the proposed method achieves the highest Precision (81.86%) and Recall (83.11%), demonstrating balanced foreground localization and object coverage. In contrast, ChannelMapper shows significantly lower performance despite its lightweight design, indicating that overly simplified fusion strategies are insufficient for complex urban scenes. ASFF and UniFusion perform better but still fall behind the proposed method.

To further analyze feature behavior, activation maps are visualized in Figure 8. The proposed method produces more spatially coherent and semantically focused feature responses across all scales. It preserves clearer boundaries and suppresses background interference more effectively than competing methods.

A key observation is that the proposed fusion strategy improves both multi-scale aggregation and boundary-sensitive feature response. This is particularly evident in thin structures such as road edges and building contours, where baseline methods often produce fragmented or blurred activations. From a modeling perspective, this indicates improved edge-aware representation, enabling better structural continuity in segmentation results.

Although the proposed fusion module introduces 44.49 M parameters, this increase is justified by consistent gains in segmentation accuracy and boundary quality. The results demonstrate a favorable trade-off between computational cost and performance.

From an urban morphology perspective, the proposed method improves the extraction of spatial structures such as building edges, road networks, and interface transitions. Compared with conventional fusion strategies, it provides more stable structural cues for downstream spatial analysis tasks.

In complex urban scenes such as historical districts, baseline methods often suffer from discontinuous segmentation results due to texture complexity and occlusion. In contrast, HieraWaveSeg maintains consistent street interface representation even under vegetation occlusion and irregular building facades. This indicates that the model captures structural continuity rather than relying solely on local texture information. Such capability is particularly important for urban morphology analysis, where the integrity of street walls directly affects enclosure perception and spatial readability.

It should be emphasized that the term interpretability in this study refers specifically to spatial morphological interpretability, rather than generic visualization of network attention. The feature maps are used to support the identification of urban structural elements rather than serving as general visualization outputs.

4.3.3. Comparative Experiments on Decoders

To evaluate the effectiveness of the proposed HieraWaveSeg Decoder, a controlled comparative study is conducted on the re-partitioned CamVid dataset. All decoder variants are implemented using the same Hiera-Transformer-Tiny backbone, ensuring that performance differences arise solely from decoder design.

The comparison includes representative decoders widely used in semantic segmentation, including the U-Net Decoder, SegFormer Decoder, Mask2Former Decoder, RF-DETR Decoder, and SAM2 Decoder. The proposed HieraWaveSeg Decoder extends these designs by introducing hierarchical fusion and WaveFusion-based frequency enhancement. The experimental results are presented in

Table 8. Performance comparison of different decoders on the re-partitioned CamVid dataset.

Deoder	Accuracy	Precision	Recall	F1-Score	IoU	Params
Mask2Former Decoder	87.92	75.30	81.32	77.33	67.09	50.13 M
RFDetrSegment Decoder	90.33	79.15	81.05	79.99	70.35	39.94 M
SAM2 Decoder	91.27	80.74	82.34	81.44	72.23	50.13 M
SegFormer Decoder	90.52	79.47	81.35	80.31	70.79	39.82 M
Unet Decoder	91.07	80.59	82.17	81.30	71.96	45.86 M
HieraWaveSeg Decoder (Ours)	91.60	81.86	83.11	81.97	73.15	44.49 M

Note: Bold values indicate the best performance results among the compared decoder architectures.

.

As shown in Table 4, the proposed decoder achieves the best performance across all metrics, with 91.60% Accuracy, 81.97% F1-score, and 73.15% IoU. It outperforms the SAM2 Decoder (72.23% IoU) and Mask2Former Decoder, demonstrating stronger spatial reconstruction and semantic consistency, particularly in complex urban scenes containing fine structures such as pedestrians, poles, and narrow road boundaries.

The proposed decoder contains 44.49 M parameters, which is lower than SAM2 and Mask2Former (>50 M) and slightly lower than U-Net. Although SegFormer and RF-DETR are more lightweight (39 M parameters), they achieve lower IoU, indicating reduced capability in modeling complex spatial structures.This demonstrates that the proposed decoder achieves a better balance between representation capability and computational efficiency.

Compared with query-based decoders such as Mask2Former, the proposed method improves IoU by 6.06%. This suggests that for street-view data with strong spatial continuity, hierarchical decoding with multi-scale fusion provides a more suitable inductive bias than purely query-driven architectures.

Compared with U-Net, the improvement further confirms the effectiveness of multi-level feature reconstruction in capturing long-range spatial dependencies.

To ensure fairness, all decoder comparisons are strictly isolated from backbone and feature fusion modules (PAN and Wave Fusion). Therefore, all performance gains can be attributed solely to decoder design.

Overall, the proposed HieraWaveSeg Decoder achieves a superior balance between accuracy and efficiency. It consistently improves segmentation quality while maintaining moderate computational cost, making it suitable for high-resolution urban scene understanding under practical constraints.

As shown in Figure 9, the introduction of WaveFusion significantly improves boundary sharpness in complex urban scenes. In particular, for thin structures such as building edges, road boundaries, and traffic poles, the proposed method produces more continuous and well-defined contours compared with SegFormer and DeepLabV3+.

Visualization of segmentation results and boundary consistency analysis. The proposed HieraWaveSeg demonstrates improved morphological interpretability by preserving continuous urban structures such as building envelopes and street interfaces. The zoom-in regions highlight enhanced boundary sharpness and reduced fragmentation compared with baseline methods.

The response maps of WaveFusion exhibit stronger spatial concentration along object boundaries, effectively reducing boundary diffusion commonly observed in standard upsampling-based decoders. This indicates that the proposed frequency-aware enhancement module improves structural alignment at object edges, leading to more accurate urban morphological representation.

To further clarify the interpretability of the proposed model in an architectural context, we extend beyond conventional Class Activation Mapping visualizations and analyze its outputs from a spatial morphology perspective. Unlike standard computer vision models that primarily provide region-level activation, HieraWaveSeg establishes a stronger correspondence between pixel-level predictions and urban morphological structures. Specifically, the predicted semantic maps can be interpreted as functional urban components, including building envelopes, street boundaries, and transitional interfaces. This enables a direct mapping from segmentation results to urban form descriptors such as spatial enclosure and boundary continuity.

Compared with baseline methods, which often produce fragmented boundaries due to insufficient high-frequency preservation, the proposed model maintains more coherent structural edges. This is particularly evident in dense urban street scenes where buildings and vegetation are closely intertwined. The improved boundary consistency demonstrates that the model captures not only visual saliency but also morphological continuity, which is essential for architectural interpretation.

4.4. Ablation Studies and Analysis

To systematically evaluate the contribution of each component in HieraWaveSeg, a comprehensive ablation study is conducted from three perspectives. First, we investigate the sensitivity of the composite loss function by analyzing different weighting strategies between ${\lambda }_{CE}$ and ${\lambda }_{Dice}$, as well as the effect of class re-weighting. Second, a controlled component-wise analysis is performed to quantify the contribution of each architectural module. Third, the WaveFusion module is isolated to assess its role in multi-scale feature refinement.

To avoid confounding effects from backbone variations, all ablation experiments are conducted using a fixed Hiera Transformer backbone. This controlled setting ensures that performance differences are attributable only to decoder design, feature fusion strategy, and loss configuration.

4.4.1. Ablation Study and Analysis of Loss Functions

To systematically evaluate the influence of loss design on model convergence and generalization, a controlled ablation study is conducted focusing on two factors: loss weighting strategy and class re-weighting. All experiments are performed under the same Hiera Transformer backbone to ensure fairness, so that performance variations are solely due to loss configuration.

Four loss configurations are evaluated: no class weighting, class-weighted cross-entropy loss only, class-weighted Dice loss only, and a combined loss with class weighting. For each configuration, three weighting settings are tested, including ${\lambda }_{CE}$ = 1.0, ${\lambda }_{CE}$ = 0.5 with ${\lambda }_{Dice}$ = 0.5, and ${\lambda }_{Dice}$ = 1.0, to analyze different optimization balances.

As shown in

Table 9. Ablation study on the loss function configurations, including class weighting (CW) and different weighting ratios between ${\mathcal{L}}_{CE}$ and ${\mathcal{L}}_{Dice}$ on the CamVid dataset.

No.	${\mathcal{L}}_{\mathit{C}\mathit{E}}$ (CW)	${\mathcal{L}}_{\mathit{D}\mathit{i}\mathit{c}\mathit{e}}$ (CW)	${\mathit{\lambda }}_{\mathit{C}\mathit{E}}$:${\mathit{\lambda }}_{\mathit{D}\mathit{i}\mathit{c}\mathit{e}}$	Accuracy	Precision	Recall	IoU
1	×	×	1:0	87.21	74.33	80.51	66.82
2	✔	×	1:0	87.95	75.12	81.33	67.29
3	×	×	0:1	89.15	78.43	80.14	68.32
4	×	✔	0:1	89.84	79.56	80.45	69.11
5	×	×	0.5:0.5	90.74	80.12	82.04	71.55
6	✔	×	0.5:0.5	91.22	80.44	82.59	72.29
7	×	✔	0.5:0.5	91.31	80.62	82.78	72.43
8	✔	✔	0.5:0.5	91.60	81.86	83.11	73.15

Note: “✔” indicates the corresponding loss component is enabled, while “×” indicates it is not applied. Bold values indicate the best performance results among different loss function configurations.

, cross-entropy loss and Dice loss exhibit complementary optimization behavior. When only cross-entropy loss is used, the model achieves higher recall, indicating stronger foreground coverage. However, boundary precision is weaker due to limited region-level constraints. In contrast, Dice loss improves precision and reduces background misclassification, but slightly decreases recall due to reduced sensitivity to small objects.

A balanced setting with ${\lambda }_{CE}$ = 0.5 and ${\lambda }_{Dice}$ = 0.5 achieves consistent improvements in both F1-score and IoU, indicating that pixel-level supervision and region-level optimization provide complementary guidance. When class weighting is further applied to both losses, the model achieves its best performance, reaching an IoU of 73.15% and an F1-score of 81.97%.

Compared with the non-weighted baseline, IoU improves by 0.86 percentage points. This demonstrates that class re-weighting effectively mitigates foreground–background imbalance by preventing dominant classes from overwhelming gradient updates. As a result, the model better captures minority categories and boundary-sensitive structures.

Further analysis shows that performance does not increase monotonically with stronger constraints. Instead, optimal results are obtained when class re-weighting is combined with balanced dual-loss optimization. This setting provides the most stable training behavior and the best generalization across semantic categories.

Overall, the results demonstrate that the proposed composite loss function does not rely on a single optimization objective. Instead, it benefits from the complementary interaction between classification-level supervision and region-level similarity constraints. This makes it a key factor in improving both segmentation accuracy and structural consistency.

To further analyze the influence of different loss function configurations, confusion matrix-based ablation results are visualized in Figure 10. The red dashed box highlights the best-performing configuration among different loss function strategies. The objective is to evaluate how loss weighting strategies and class re-weighting affect class-wise prediction behavior in semantic segmentation under a controlled experimental setting.

A total of twelve configurations are evaluated, forming a 3 × 4 experimental grid. Each column indicates whether class weighting is applied, while each row corresponds to a different loss weighting ratio. The confusion matrices use predicted classes on the horizontal axis and ground truth labels on the vertical axis, where diagonal elements represent correct classifications for each category.

From the perspective of loss weighting, the configuration with ${\lambda }_{CE}$ = 0.5 and ${\lambda }_{Dice}$ = 0.5 produces the most balanced confusion distribution. The diagonal responses are more concentrated than in single-loss settings, indicating improved intra-class consistency. In contrast, single-loss configurations exhibit more scattered error patterns, particularly for structurally complex categories such as buildings and vegetation, where misclassification into background regions occurs more frequently.

This result indicates that combining Cross-Entropy Loss and Dice Loss provides complementary optimization signals. Cross-Entropy Loss improves pixel-level discrimination, while Dice Loss enhances region-level consistency, jointly improving boundary robustness in complex urban scenes.

Regarding class re-weighting, the configuration applying weighting to both losses achieves the most stable performance across categories. This setting reduces off-diagonal misclassifications while maintaining strong diagonal responses, indicating better balance between dominant and minority classes, especially for categories with high spatial variability.

Combined with the quantitative results in Table 9, these findings confirm that class re-weighting improves optimization stability under imbalanced class distributions. The model achieves better separation between visually similar categories and reduces ambiguity in boundary regions.

Overall, the confusion matrix analysis demonstrates that performance gains are not driven by a single factor but result from the interaction between loss composition and class balancing strategy. This provides consistent evidence that the proposed loss formulation improves both classification reliability and structural consistency in semantic segmentation tasks.

4.4.2. Ablation Study and Analysis of Key Components

To systematically evaluate the contribution of key components in HieraWaveSeg to semantic segmentation performance, controlled ablation experiments are conducted under a fixed Hiera Transformer backbone. This ensures that performance variations are solely attributed to architectural components rather than backbone differences.

As shown in

Table 10. Ablation results of key components in HieraWaveSeg.

Neck Fusion	MLP	Wave Fusion	Accuracy	Precision	Recall	F1-Score	IoU
-	-	-	86.97	73.68	77.84	75.32	64.55
√	-	-	90.37	79.03	81.45	80.11	70.52
-	√	-	88.80	76.50	79.33	77.71	67.60
-	-	√	90.35	78.95	81.43	80.07	70.47
√	√	-	90.74	79.66	81.72	80.57	71.16
-	√	√	90.49	79.02	81.41	80.09	70.65
√	-	√	90.45	79.33	81.67	80.39	70.83
√	√	√	91.60	81.86	83.11	81.97	73.15

Note: “√” indicates that the corresponding module is enabled. Bold values indicate the best performance results among different component configurations.

, the experiments analyze three core modules: Neck Fusion, Multi-Layer Perceptron (MLP), and WaveFusion. The full model, which integrates all components, achieves the best overall performance, reaching an accuracy of 91.60% and an IoU of 73.15%. Compared with the baseline configuration without these modules, IoU improves by 8.6 percentage points, demonstrating the effectiveness of the overall architectural design.

From individual component analysis, both Neck Fusion and WaveFusion independently contribute to performance improvement, with IoU values exceeding 70%. Neck Fusion enhances cross-scale spatial aggregation, while WaveFusion improves feature refinement through multi-scale and frequency-aware representation. However, neither module alone achieves optimal performance.

Further analysis of partial combinations shows that all two-module configurations outperform single-module variants but remain inferior to the full model. This indicates that the improvements are not purely additive but arise from structured interactions among components. In particular, WaveFusion complements Neck Fusion by refining features after spatial fusion, leading to more stable semantic representations.

The MLP module contributes to feature transformation and nonlinear representation learning, but its effect is more significant when combined with the other two modules. This suggests that it plays a supportive role in improving feature adaptability rather than acting as the primary source of performance gains.

To further clarify the contribution of each component, we explicitly analyze the performance gain between the PAN-based baseline and the proposed WaveFusion-enhanced model. As shown in Table 6, the model equipped with Neck Fusion and MLP achieves an IoU of 71.16%. After integrating the WaveFusion module, the IoU increases to 73.15%, yielding a consistent gain of 1.99%. This improvement demonstrates that, beyond spatial feature aggregation provided by PAN, the frequency-aware WaveFusion module provides additional refinement in modeling high-frequency structural details. In particular, it improves the representation of object boundaries and thin structures, which are often degraded in purely spatial feature fusion.

Overall, the results show that PAN and WaveFusion are not independent components but form a complementary hierarchical design. PAN provides robust multi-scale semantic aggregation, while WaveFusion introduces frequency-aware local refinement. Their integration enables the model to capture both global structural context and fine-grained boundary details, leading to improved segmentation performance in complex urban scenes.

4.4.3. AAblation Study and Analysis of WaveFusion

To further investigate the contribution of individual components within WaveFusion to change detection performance, detailed ablation experiments are conducted. As shown in

Table 11. Ablation results of WaveFusion.

WaveFusion			Accuracy	Precision	Recall	F1-Score	IoU	Glops	Inference Time
DWT Edge Attention	Concat	Resnet
√	√	-	91.09	80.58	82.11	81.27	71.99	114.14	0.0441 s
-	√	√	91.07	80.52	82.04	81.21	71.91	107.54	0.0218 s
√	-	√	89.77	78.44	80.93	79.56	69.61	94.33	0.0145 s
√	√	√	91.60	81.86	83.11	81.97	73.15	114.14	0.0144 s

Note: “√” indicates that the corresponding module is enabled. Bold values indicate the best performance results among different WaveFusion configurations.

, controlled experiments are designed by removing or combining three key modules: DWT Edge Attention, Concat, and ResNet. This setup enables a systematic evaluation of the impact of each component on the overall model performance.

The experimental results demonstrate that the model achieves the best performance across all evaluation metrics when all three core components—DWT Edge Attention, Concat, and ResNet—are enabled simultaneously, attaining an Accuracy of 91.60%, F1-Score of 81.97%, and IoU of 73.15%. This result significantly outperforms all incomplete configurations, providing strong evidence of the notable synergistic effect among the three modules. The frequency-domain boundary enhancement provided by DWT Edge Attention, the multi-scale feature fusion enabled by Concat, and the residual optimization from ResNet are organically integrated, collectively forming the most robust feature extraction and fusion framework within the WaveFusion module.

Further quantitative analysis reveals that the Concat module contributes the most significantly. Removing it causes a substantial IoU drop from 73.15% to 69.61% (a decrease of 3.54 percentage points), underscoring its critical role in maintaining information flow integrity and enabling cross-dimensional feature fusion. In comparison, removing DWT Edge Attention results in an IoU decrease of 1.24 percentage points (73.15% → 71.91%), while removing ResNet leads to a drop of 1.16 percentage points. These findings indicate that each component provides valuable performance gains.

Regarding computational efficiency, the complete WaveFusion module requires 114.14 GFLOPs with an inference time of 0.0144 s. After removing DWT Edge Attention, the FLOPs decrease to 107.54 GFLOPs (a reduction of approximately 6.6%), while the IoU drops by 1.24 percentage points. This shows that the DWT Edge Attention module achieves meaningful performance improvement and superior boundary quality at a relatively small computational cost (approximately 6.6 GFLOPs). Notably, the full model’s inference time (0.0144 s) is even faster than some ablation configurations, further indicating that incorporating the DWT operation does not significantly increase the actual inference burden.

Overall, the ablation experiments not only validate the rationality of the WaveFusion module’s architectural design but also demonstrate its excellent precision-efficiency trade-off. Although each individual component brings incremental improvements, only the synergistic integration of all three enables the model to achieve optimal performance across key metrics, including Accuracy, Precision, Recall, F1-Score, and IoU, while maintaining lightweight computational overhead. This allows the model to deliver efficient and reliable performance in complex street scene segmentation tasks.

4.5. Morphological Interpretability Analysis

To address the limitations of conventional visual explanations, we further analyze the interpretability of HieraWaveSeg from a morphological perspective. In this study, interpretability refers to the alignment between semantic segmentation outputs and urban spatial structure, rather than attention visualization.

The predicted categories are reorganized into functional morphological elements. The “Building” class represents the urban street wall system, which defines spatial enclosure. The “Road Append” class captures transitional soft boundaries such as sidewalks and street furniture, which are critical for pedestrian-scale perception. The “Dynamic” class reflects spatial vitality and human activity distribution.

Unlike standard segmentation models, which often produce fragmented or blurred boundaries, HieraWaveSeg preserves continuous urban interfaces through frequency-aware feature modeling. This is achieved by the WaveFusion module, which enhances high-frequency structural components such as edges, corners, and façade discontinuities. As a result, the model provides more reliable geometric representations for urban morphology analysis, including enclosure estimation and interface permeability assessment.

5. Discussion

5.1. Overall Morphological Interpretation

The experimental results demonstrate that HieraWaveSeg consistently outperforms existing semantic segmentation methods in complex urban street environments. The improvements are most evident in Intersection over Union (IoU) and boundary-related metrics, indicating stronger structural consistency in pixel-level predictions.

From a morphological perspective, the improvement is not limited to classification accuracy. It reflects a better alignment between predicted semantic regions and actual urban spatial structures. In particular, the model produces more continuous building edges, clearer street boundaries, and more coherent spatial partitions, which are fundamental elements in urban form analysis.

This study positions semantic segmentation not only as a computer vision task but also as a computational approach for urban morphology analysis. The proposed interpretability therefore refers to the model’s ability to preserve and reveal architecturally meaningful spatial structures rather than merely visualizing network attention responses.The objective is to move beyond pixel-level classification toward structural understanding of urban form, including spatial hierarchy, enclosure, and continuity of the built environment.

Unlike conventional convolution-based models that mainly rely on local texture information, the proposed method integrates hierarchical global context with frequency-aware refinement. This enables a more stable representation of spatial hierarchy, which is essential for interpreting urban spatial structures at multiple scales.

5.2. Hierarchical Representation and Urban Spatial Structure

The Hiera Transformer backbone enhances the representation of urban form across multiple spatial scales. Its hierarchical design supports progressive abstraction from local architectural details to global street patterns.

Experimental results show that hierarchical modeling significantly improves segmentation stability in dense urban environments. This is particularly important in areas where building façades, vegetation, and street infrastructure are tightly interwoven.

From an urban morphology perspective, this capability corresponds to improved recognition of spatial hierarchy, including relationships among buildings, street corridors, and open public spaces. The model not only classifies objects but also preserves their structural organization within the urban fabric.

5.3. Multi-Scale Fusion and Spatial Continuity

The Path Aggregation Network strengthens interaction between semantic and spatial features across scales, reducing the loss of structural continuity that often occurs in standard encoder–decoder architectures.

In urban scenes, morphological coherence depends on maintaining consistent boundaries across scales. For example, building edges and road networks must remain stable under occlusion or perspective distortion.

The proposed fusion strategy improves this continuity by enabling bidirectional information flow between deep semantic layers and shallow spatial layers. As a result, segmentation outputs exhibit stronger structural integrity, particularly in boundary-dense regions such as intersections and building clusters.

5.4. Frequency-Aware Refinement and Boundary Morphology

The WaveFusion module introduces frequency-domain decomposition to explicitly enhance structural details. By separating low-frequency semantic components from high-frequency geometric information, the model improves its ability to preserve morphological boundaries.

This is particularly important in urban environments, where structural clarity depends on accurate representation of edges, corners, and discontinuities. Experimental results confirm that WaveFusion significantly improves boundary sharpness and reduces segmentation fragmentation.

From a morphological perspective, this mechanism enhances the interpretability of spatial form by preserving geometric continuity of built elements. Building outlines, road edges, and transitional zones become more structurally consistent, which is critical for downstream urban analysis tasks.

5.5. Comparison with Existing Methods in Urban Context

Compared with existing semantic segmentation methods, including both convolutional and Transformer-based architectures, HieraWaveSeg demonstrates superior performance in preserving spatial coherence and structural consistency.

Many existing approaches primarily focus on improving classification accuracy or global context modeling. However, they often fail to preserve fine-grained morphological structures in complex urban scenes.

In contrast, the proposed method integrates hierarchical representation with frequency-aware refinement, improving both global structural understanding and local boundary precision. This dual capability is essential for urban morphology applications, where both macro-scale organization and micro-scale boundaries must be accurately represented.

5.6. Morphological Interpretability and Architectural Relevance

In this study, interpretability does not refer to generic attention visualization. Instead, it refers to the correspondence between segmentation outputs and urban morphological structures.

Class activation mapping is used only as a standard visualization tool in computer vision to highlight response regions. It is not treated as an explanatory mechanism for architectural reasoning. Instead, interpretability is evaluated based on how well segmentation outputs align with meaningful spatial structures in the built environment.

Case analysis shows that the model reveals spatial patterns that are often missed by traditional segmentation methods. For example, in dense historical districts, it preserves continuous building envelopes even under vegetation occlusion. It also distinguishes subtle transitions between pedestrian pathways and architectural boundaries, which are often fragmented in baseline models.

These results indicate that the model provides additional analytical value for urban designers and architects by supporting the identification of spatial continuity, enclosure quality, and interface transitions in complex street environments.

Therefore, the enhanced interpretability should be understood as improved morphological readability of urban space rather than an explanation of internal neural mechanisms.

5.6.1. Case-Driven Urban Morphology Analysis Based on Feature Activation

To address the limitation of relying solely on class activation maps as a generic visualization tool, this study further conducts a case-driven urban morphology analysis to interpret model behavior in real spatial contexts.

Rather than treating activation maps as abstract attention representations, representative urban scenes from both datasets are analyzed, including dense historical blocks, modern grid streets, and mixed-use transition areas.

In historical urban districts, the proposed model consistently highlights continuous building envelopes even under vegetation occlusion. Unlike baseline methods, which often fragment façades, HieraWaveSeg preserves street wall integrity and maintains clear spatial enclosure boundaries. This indicates that the model captures spatial continuity rather than relying only on local texture similarity.

In modern grid-based street environments, activation responses concentrate along road axes and building alignments, reflecting correct recognition of orthogonal urban structure. The model distinguishes primary circulation spaces from secondary service areas, which are often missed by conventional methods.

In mixed urban transition zones, where buildings, vegetation, and transportation infrastructure intersect, the model focuses on functional boundaries rather than visual noise. It successfully identifies pedestrian–vehicle separation zones and semi-public transitional spaces, which are difficult to capture using appearance-based approaches.

These observations confirm that the proposed method improves not only pixel-level accuracy but also structural understanding of urban form. The learned representations align with key concepts in urban morphology, including enclosure, permeability, and spatial continuity.

Figure 11 further demonstrates the architectural interpretability of the proposed framework through representative urban morphology cases. Unlike conventional segmentation models that mainly emphasize pixel-level semantic consistency, HieraWaveSeg preserves structurally meaningful spatial relationships that are directly relevant to architectural and urban analysis.

In historical street environments, the proposed model maintains continuous building envelopes despite severe vegetation occlusion, allowing architects to identify coherent street wall systems and enclosure conditions that are fragmented in baseline methods. In modern grid-based urban scenes, HieraWaveSeg more clearly captures road axes and building alignment patterns, supporting the interpretation of spatial hierarchy and urban order. In mixed-use transition areas, the model preserves functional spatial interfaces, including pedestrian–vehicle separation zones and semi-public transitional boundaries, which are often blurred or discontinuous in conventional approaches.

These results indicate that the proposed framework improves not only segmentation accuracy but also the morphological readability of urban space. More importantly, it enables the extraction of spatial patterns that are difficult to identify using standard semantic segmentation methods, thereby providing additional analytical value for architectural interpretation and urban morphology studies.

5.6.2. Architectural and Urban Design Usability Analysis

Beyond algorithmic performance, the practical value of HieraWaveSeg lies in its applicability to architectural and urban design workflows. The model provides structured semantic representations that support early-stage spatial analysis and morphological evaluation.

One major challenge in urban analysis is the lack of precise and scalable tools for extracting spatial boundaries and enclosure systems from complex street environments. Manual mapping is time-consuming and subjective, while conventional vision models often fail to preserve architectural continuity at the block scale.

The proposed method addresses this limitation by generating segmentation outputs that are directly interpretable in terms of built form. Building regions correspond to façade systems, road regions correspond to circulation networks, and transitional areas represent semi-public interfaces. This mapping enables rapid understanding of spatial composition without manual vectorization.

In practical design scenarios, the model supports several key tasks. First, it enables rapid assessment of spatial enclosure quality for evaluating street comfort and urban vitality. Second, it helps detect discontinuities in building edges, useful for analyzing morphological fragmentation in historical districts. Third, it provides a structured basis for comparing urban typologies in evidence-based planning.

Importantly, the model does not replace architectural judgment but complements it with consistent spatial measurements derived from large-scale visual data. It therefore serves as a computational support tool for urban morphology analysis rather than a purely predictive vision system.

From this perspective, the enhanced interpretability acts as a bridge between computer vision outputs and architectural spatial reasoning, enabling more direct translation from segmentation results to morphological indicators.

5.7. Limitations and Future Work

Despite strong performance across benchmark datasets, several limitations remain. First, frequency-domain operations improve boundary preservation but increase computational complexity, which may limit real-time or large-scale deployment under constrained resources.

Second, evaluation is mainly conducted on street-view datasets. Although representative of pedestrian-level perception, they do not cover aerial imagery, 3D city models, or indoor environments. Therefore, cross-modal generalization requires further validation.

Future work will focus on improving computational efficiency through lightweight frequency-domain approximations and more efficient decomposition strategies. The framework will also be extended to broader urban tasks such as spatial typology classification and morphological evolution analysis. In addition, incorporating geographic context and architectural priors may further improve interpretability in real planning scenarios, strengthening the connection between data-driven outputs and urban design decision-making.

5.8. Urban Morphological Implications of the Six-Class Reconfiguration

The proposed six-class semantic reconfiguration is not a simple adjustment of label granularity but a deliberate abstraction designed to align segmentation outputs with interpretable units of urban morphology. By consolidating fine-grained categories into six functional classes—building, road, vegetation, sky, vehicle, and pedestrian—the model shifts from object-level recognition to spatial-functional decomposition of urban space.

From an urban morphology perspective, this reconfiguration enables a more structured interpretation of street space. It reduces intra-class noise and improves continuity of spatial entities such as building façades and street corridors. As a result, key morphological attributes, including enclosure, interface continuity, and spatial openness, can be more consistently derived from segmentation outputs. This is particularly important in dense urban environments where fine-grained categories often introduce fragmentation in interpretation.

At the same time, this simplification improves interpretability for downstream urban studies. Rather than focusing on detailed object distinctions, the six-class scheme emphasizes structurally meaningful components directly related to morphological characterization. This makes the results more suitable for applications such as street morphology quantification, visual permeability estimation, and built-environment composition analysis.

Regarding comparability, the reconfiguration is applied consistently across all models and datasets. Both baseline methods and HieraWaveSeg are trained and evaluated under the same six-class mapping, ensuring fair comparison. Although this differs from original dataset annotations such as Cityscapes and CamVid, the goal is not to redefine benchmarks but to establish a unified abstraction level aligned with urban morphology analysis.

Therefore, the proposed reconfiguration does not reduce comparability. Instead, it introduces a task-oriented abstraction layer that bridges pixel-level segmentation and urban form interpretation, enabling both consistent evaluation and improved relevance for architectural and urban studies.

6. Conclusions

This study addresses persistent challenges in street-view building segmentation, including large-scale variation, boundary ambiguity, class imbalance, and the loss of high-frequency structural information. To address these limitations, we propose HieraWaveSeg, a segmentation framework that integrates hierarchical feature representation, multi-scale spatial fusion, and frequency-aware detail enhancement.

Unlike existing methods that rely primarily on spatial-domain feature aggregation, the proposed approach introduces a complementary frequency-domain perspective to better preserve structural discontinuities in urban scenes. By combining hierarchical encoding, bidirectional feature fusion, and wavelet-based high-frequency enhancement, the model improves both semantic consistency and boundary preservation in complex street environments.

Extensive experiments conducted on Cityscapes and CamVid demonstrate that the proposed method consistently outperforms representative state-of-the-art models, including DeepLabV3+, SegFormer, U-Net, and PIDNet. The improvements are particularly evident in boundary-sensitive metrics, indicating stronger structural representation capability in visually complex urban scenes.

Beyond quantitative performance gains, the results suggest that preserving high-frequency structural information is crucial for reliable urban scene interpretation. The proposed framework produces more coherent representations of building façades and street interfaces, which are essential elements in urban morphology analysis. In this sense, the segmentation output can be interpreted not only as pixel-level classification results, but also as a structured representation of urban form.

From an application perspective, the proposed method is relevant to urban planners, architects, and researchers in computational urban studies. It provides a data-driven tool for analyzing spatial continuity, façade organization, and interface structure, thereby supporting more systematic and quantitative approaches to urban morphological assessment. This bridges the gap between computer vision-based perception and architectural interpretation of urban space.

However, several limitations remain. First, the current framework is limited to two-dimensional street-view imagery and cannot fully capture three-dimensional spatial relationships, such as building volume or depth structure. Second, performance may degrade in highly occluded or visually ambiguous environments where semantic cues are insufficient. Third, the model focuses on static imagery and does not yet incorporate the temporal dynamics present in urban environments.

Future work will focus on lightweight model design for real-time deployment, extension to video-based urban scene understanding, and integration of multi-modal data such as LiDAR and remote sensing imagery to support three-dimensional urban modeling and digital twin applications. In addition, improving the interpretability of frequency-domain mechanisms will be an important direction for enhancing the transparency of urban morphology analysis.

In summary, HieraWaveSeg provides a unified framework that links semantic segmentation with urban morphological interpretation. It demonstrates that incorporating structural preservation mechanisms into deep learning models can significantly enhance their relevance for architectural and urban studies, particularly for tasks requiring a detailed understanding of urban spatial form.