Closed-Set vs. Open-Vocabulary Object Detectors for Urban Architectural Typology Classification: A Comparative Study on Athenian Heritage Buildings

Abstract

Architectural typology classification plays an important role in large-scale documentation and analysis of urban cultural heritage. Recent advances in computer vision enable automated approaches for detecting and categorizing buildings from street-level imagery, yet the suitability of different detection paradigms for architectural typology analysis remains insufficiently explored. Despite recent advances in computer vision for architectural analysis, no systematic comparative study has evaluated closed-set CNN-based detectors against open-vocabulary vision–language grounding models for urban architectural typology classification. This study presents a comparative evaluation of closed-set convolutional object detectors and open-vocabulary vision–language grounding models for the classification of Athenian architectural typologies. A dataset of 3349 street-view images containing 11,111 annotated building instances was compiled and organized into five typological categories: Neoclassical, Neoclassical-Eclectic, Interwar-Eclectic, Interwar, and Postwar. The experiments compare several YOLO-based detection configurations with Grounding DINO under zero-shot inference, parameter-efficient adaptation (e.g., Kiw Rank Adaptation—LoRA), and full fine-tuning. Results show that supervised YOLO-based models achieve robust detection and classification performance with high localization accuracy and consistent typology discrimination in dense urban scenes. In contrast, open-vocabulary grounding models demonstrate limited reliability in zero-shot settings and require substantial adaptation to approach comparable performance levels. Analysis of confusion patterns further reveals that most classification errors originate from intrinsic architectural similarities between transitional styles rather than from model instability. The findings highlight the advantages of supervised object detection frameworks for scalable urban heritage documentation and provide insights into the current limitations of vision–language models for fine-grained architectural typology classification.

1. Introduction

Architectural typologies have historically evolved in response to the social, economic, political, and religious contexts of each region. At the same time, technological progress, the availability of construction materials, and local climatic conditions have significantly influenced the formal and structural characteristics of the built environment. As a result, the study of architectural history represents a central research domain for architects, archaeologists, historians, and cultural heritage (CH) specialists. Such investigations require advanced training, disciplinary expertise, and frequently in situ documentation. The systematic survey of an urban architectural landscape is therefore a demanding process. It typically involves multiple evaluators, structured documentation protocols, robust data management systems, and consistent evaluation methodologies. Beyond documenting stylistic characteristics, these surveys play a crucial role in tracing architectural transformation patterns and inferring construction periods, since building types evolve gradually across historical phases [1,2,3,4,5].

In computer vision, it is important to distinguish between related but conceptually different tasks such as image tagging, image-level classification, and object detection. Tagging assigns labels at the image level without spatial localization, while classification assigns semantic categories either at the image or object level. Object detection extends classification by jointly localizing and categorizing individual object instances within an image. In this study, architectural typology classification is formulated as a fine-grained object-level classification problem implemented through object detection, where each building instance is localized and assigned to a predefined typological class. The typological scheme constitutes a domain-specific architectural taxonomy derived from historical and morphological criteria, enabling a structured mapping between computer vision outputs and architectural interpretation.

Recent advances in deep learning and computer vision have introduced automated methods for large-scale architectural classification. Closed-set object detectors, based on convolutional neural networks, such as YOLO, have demonstrated strong performance in real-time object detection and classification tasks. However, these systems operate under predefined and fixed class labels. Such restrictions limit flexibility when dealing with stylistic transitions, ambiguous typologies, or evolving category definitions, which are common in architectural heritage analysis.

In parallel, a new generation of vision–language grounding models has emerged. These open-vocabulary systems aim to detect and classify objects using natural language prompts, allowing greater conceptual flexibility compared to traditional closed-set detectors. In principle, such models could offer advantages for cultural heritage applications, where stylistic categories are historically nuanced and linguistically defined. Despite this potential, there is currently no systematic comparative study evaluating closed-set CNN detectors against vision-language grounding models in the specific context of fine-grained urban architectural typology classification from facade-level urban imagery. This gap motivates the purpose of the present research to investigate how different computer vision detection paradigms can support architectural typology classification as a methodological tool for urban heritage documentation.

Hence, the research question in this paper is “How do real-time CNN-based detectors compare to vision-language grounding models in urban architectural typology classification?” This research question is particularly relevant in the context of urban heritage documentation, where the systematic identification and classification of architectural typologies constitute a fundamental step for recording and interpreting the built environment. Urban architectural landscapes are characterized by high visual complexity, stylistic continuity, and transitional forms, making manual large-scale documentation both time-consuming and subjective. At the same time, emerging computer vision approaches offer the potential for scalable and reproducible analysis, yet their suitability for fine-grained typological classification remains unclear. Evaluating how different detection paradigms perform under these conditions is therefore essential for developing reliable, methodologically grounded tools that can support heritage documentation workflows. This article extends our previous conference paper, which focused exclusively on closed-set detection approaches, by introducing a fundamentally different model family (Grounding DINO), expanding the experimental framework, and providing an in-depth comparative and theoretical analysis.

This study contributes:

• A comprehensive comparative evaluation of YOLO-based closed-set detection and Grounding DINO open-vocabulary grounding for architectural typology classification.
• An extended experimental framework including detection accuracy, classification performance, computational efficiency, and robustness analysis.
• A detailed error taxonomy highlighting stylistic confusion patterns in visually similar architectural categories.
• Insights into the suitability of each paradigm for scalable urban heritage documentation and monitoring.

Therefore, unlike previous works focusing on the application of individual detection models for architectural classification, this study positions itself as a comparative methodological analysis between closed-set and open-vocabulary detection paradigms, aiming urban heritage documentation, where architectural typology classification serves as a structured methodological framework for recording, analyzing, and monitoring the built environment at scale.

2. Related Work

2.1. YOLO-Based Methods for Building Classification

Recent studies have explored the application of YOLO-based deep learning models for recognizing architectural characteristics and building typologies from facades. YOLO architecture is particularly suitable for this task due to their ability to perform real-time object detection, while capturing spatial relationships between facade elements such as windows, balconies, and doors. For example, research on facade element detection has demonstrated that YOLO-based models can accurately identify architectural components—including windows and doors—achieving higher precision and recall compared to traditional detection frameworks, which supports downstream tasks such as facade understanding and typological analysis [3]. Similarly, improved YOLOv3 architectures have been used to detect facade openings such as windows, incorporating optimized anchor boxes and enhanced feature extraction to improve detection of small facade elements in urban street imagery [4]. More advanced approaches focus on multi-scale facade element detection, where specialized YOLO variants such as BFA-YOLO incorporate attention mechanisms and multi-scale feature fusion to identify facade attachments (e.g., balconies, billboards, air-conditioning units, and curtain walls) in multi-view building images, improving mean average precision compared with baseline YOLO models [5]. These detected facade components can serve as morphological descriptors that help characterize architectural style or building typology.

Ji and Jun [6] proposed a two deep learning approaches for the classification of buildings and the detection of architectural elements in East Asian structures. A region-based convolutional neural network (R-CNN) is used to classify buildings by country, while the YOLO algorithm detects and localizes structural components. Two datasets were used for training: one containing 3872 images of buildings from Japan, Korea, and China, and another including 16,478 images of Gong-pos and columns in Korean architecture. The authors also note that although YOLO enables faster real-time detection than R-CNN, its performance may decrease when objects are captured from different viewpoints or partially occluded scenes.

Finally, newer versions of YOLO-based models (YOLO v8 and YOLO v11) have also been applied directly to the classification of architectural heritage buildings using facade images, demonstrating the potential of object detection frameworks to automatically categorize buildings according to typological and historical attributes within urban datasets [7,8] that provided strong foundation for the current research.

In conclusion, all these studies indicate that YOLO-based methods provide an effective framework for façade-based building typology recognition by detecting architectural features and learning visual patterns associated with specific building categories.

2.2. Grounding DINO-Based Methods for Building Classification

Recent advances in vision–language object detection have enabled flexible building detection and classification workflows in remote sensing and urban analysis. The Grounding DINO framework introduced a transformer-based open-set detector that integrates language and visual features, allowing objects to be detected using textual prompts rather than fixed class labels [9]. To the best of our knowledge, no prior work has investigated the application of Grounding DINO to facade-level building classification, leaving this research direction largely unexplored.

Specifically, related work explores open-vocabulary grounding models such as Grounding DINO for zero-shot object detection and segmentation in domains including remote sensing, plant phenotyping, and defect inspection [10,11,12,13]. These studies show that zero-shot grounding can localize objects without task-specific training, but performance degrades in cluttered environments and when classes are visually similar or densely distributed. Results are strongly dependent on prompt formulation and semantic alignment between text and visual features.

Several approaches attempt to address these limitations. Some combine Grounding DINO with task-specific preprocessing or frequency-domain analysis to improve fine-grained discrimination [12], while others propose few-shot adaptation strategies that update a limited subset of model parameters while keeping the backbone frozen [13]. Parameter-efficient adaptation demonstrates substantial improvements over zero-shot inference, particularly in visually complex scenes, and can outperform fully fine-tuned detection models when training data are scarce. However, most studies evaluate object presence or coarse localization rather than consistent closed-set classification across multiple adjacent categories.

In the context of architectural typology classification, buildings exhibit strong inter-class similarity and appear in dense urban configurations. These characteristics challenge prompt-based zero-shot grounding and require controlled adaptation. Motivated by these findings, the present work systematically evaluated zero-shot inference, LoRA-based parameter-efficient tuning, and full fine-tuning of Grounding DINO for multi-instance architectural typology classification, providing a structured comparison between open-vocabulary grounding and closed-set architectural discrimination.

3. Theoretical Background

3.1. Closed-Set Object Detection

Closed-set object detection refers to detection systems that operate on a fixed and predefined set of classes. During training, the model learns to localize objects and assign them to one of the known categories included in the dataset. At inference time, the model cannot predict classes outside this predefined set. However, such models can be retrained from scratch or be fine-tuned to be able to apply their powerful architecture for the detection of objects of a different domain. In this work, the YOLO-based models follow a closed-set formulation. The architectural typologies (Neoclassical, Neoclassical-Eclectic, Interwar, Interwar-Eclectic, and Postwar) are defined in advance and used consistently during training and evaluation. The detector simultaneously predicts bounding box coordinates and class probabilities for these specific categories [14].

The closed-set approach provides several advantages. It allows strong optimization for a well-defined classification task and enables stable evaluation using standard metrics such as precision, recall, F1-score, and mean Average Precision (mAP). However, its limitation lies in the inability to generalize to unseen categories without retraining. The model performance is therefore tightly coupled with the dataset quality and annotation consistency. In structured classification problems, such as architectural typology analysis with predefined classes, closed-set detection provides controlled and measurable results.

3.2. Open-Vocabulary Object Detectors

Open-vocabulary object detection extends traditional detection by incorporating language-conditioned representations. Instead of predicting only fixed class labels, the model receives textual prompts and aligns visual features with language embeddings. This allows the system to detect objects described in natural language, even if these categories were not explicitly defined during supervised training [15,16]. Grounding DINO [17] belongs to this category and combines transformer-based visual encoders with text encoders and performs alignment between image regions and textual queries. Hence, inference is driven by prompts rather than by a strictly predefined classification head.

In the context of architectural typology classification, this formulation introduces a different problem setup. Instead of learning only from annotated typology labels, the model must interpret different architectural descriptions and align them with visual patterns. This enables zero-shot detection, where no task-specific training is performed. However, open-vocabulary models introduce additional complexity. Performance depends not only on visual features but also on the semantic clarity of prompts. Ambiguous or broad architectural descriptions can reduce class separability. Furthermore, evaluation becomes sensitive to score thresholds and prompt design.

4. Dataset and Experimental Setup

4.1. Dataset Description

The image dataset used in this study comprises 3349 views and scenes of Athenian buildings and urban environment developed after 1834, the majority of which include multiple buildings per image. During data acquisition, images were acquired so that the primary building of interest is positioned centrally, while secondary neighboring structures remain visible. Additional manual image collection was performed to enrich under-represented classes, particularly Neoclassical buildings, which are less frequent in the contemporary urban environment and represent objects of high interest.

More specifically, the dataset was compiled using three complementary acquisition strategies. First, a number of buildings were documented through direct photographic capture. Photographs were taken using a digital camera from multiple viewpoints in order to examine the robustness of the classification process across different observation angles. The images primarily depict the full facade of each building, while in several cases they also include partial views focusing on characteristic architectural elements such as the ground floor, upper structural parts, openings, or decorative features.

Second, screenshots from Google Street View were incorporated into the dataset to increase variability in image quality and viewpoint conditions. These images often represent the same building from alternative perspectives, enabling additional validation of classification results while simultaneously expanding the dataset with additional samples. In many cases the captured scenes intentionally contain more than one building (e.g., building rows or foreground–background compositions) in order to evaluate the algorithm under realistic urban conditions rather than ideal single-object projections. Images with partial visibility caused by urban obstacles—such as parked vehicles, traffic infrastructure, signage, or vegetation—were also included to test the robustness of the detection and classification pipeline. Furthermore, buildings that have undergone alterations due to later uses (for example commercial ground-floor modifications, changes in opening dimensions, conservation degradation, or multiple construction phases) were also incorporated in order to assess whether the original architectural typology could still be identified despite subsequent transformations.

Third, additional images were collected from online sources. These include crowdsourced digital repositories documenting Athenian architectural heritage (e.g., the non-publicly available ModMov from the Society for the Environment and Cultural Heritage–ELLET and the Listed Buildings Archive from the Hellenic Ministry of Culture-HMC) as well as social media platforms such as the Athenian Modernism Facebook page [18]. The inclusion of these sources further increased the diversity of the dataset, introducing variations in image quality (including faded black-and-white photographs) and different capture perspectives, such as aerial views, complementing the predominantly street-level imagery. The inclusion of restricted-access archival material was carried out under appropriate permissions (ELLET and HMC), and all data was used solely for research purposes.

While full dataset release is limited by access restrictions, the experimental setup, annotation methodology, and model configurations are fully documented to support methodological reproducibility.

Description of Classes

The residential architecture of Athens after 1834 can be classified into the following stylistic categories that correspond to different historical periods and architectural influences.

(a)

Neoclassical architecture: Neoclassicism, which emerged in Europe during the eighteenth century, is characterized by formal balance, compositional harmony, and elaborate decorative articulation inspired by classical antiquity. In nineteenth-century Athens, this architectural language was consciously adopted as a reference to the city’s ancient heritage. Typical features of Athenian neoclassical buildings include symmetrical compositions, clear geometric forms, and decorative elements such as columns or pilasters, pediments, friezes, balconies supported by corbels, and ornamental railings [19].

(b)

Eclectic architecture: Eclecticism became prominent in Athens during the late nineteenth and early twentieth centuries. Influenced by artistic movements such as Art Nouveau, the Vienna Secession, and the École des Beaux-Arts, eclecticism combines architectural elements drawn from multiple historical styles and traditions. This approach enabled architects to create compositions that blended classical references with Baroque or other decorative influences. Eclectic buildings are often distinguished by extensive ornamentation, curved forms, floral or organic motifs, and the incorporation of newer materials such as steel and glass [20].

(c)

Interwar architecture. Interwar architecture developed during the period between the end of the First World War and the beginning of the Second World War, particularly after the Asia Minor Catastrophe of 1922. In contrast to the richly decorated styles of the nineteenth century, this architectural phase emphasized functional design, structural clarity, and a significant reduction in ornamental elements [21]. Influenced by the principles of the International Style and the Bauhaus movement, buildings from this period typically feature simplified volumes, geometric forms, and the use of modern construction materials such as reinforced concrete and metal.

(d)

Postwar architecture: Following the Second World War, Athens experienced extensive urban expansion driven largely by internal migration from rural areas to the capital. This period gave rise to the widespread construction of multi-storey reinforced-concrete apartment buildings, which eventually became a defining feature of the city’s urban landscape [22]. Although significant architectural works were produced during this era, the dominant building production was primarily driven by mass urban development rather than by individual design influence.

During transitional periods between the above architectural phases, hybrid stylistic tendencies frequently appear, resulting in buildings that combine elements from multiple traditions. For instance, late neoclassical buildings often incorporate richer decorative programs that approach eclectic aesthetics. Similarly, during the interwar period many architects did not fully adopt the strict principles of the Modern Movement or Bauhaus, instead producing buildings that retained eclectic decorative elements while simultaneously introducing modern materials, modified building volumes, and enlarged window openings [23]. Since eclectic features frequently appear in combination with either neoclassical or interwar architectural forms, the typological classification adopted in this study distinguishes five categories: (i) neoclassical, (ii) neoclassical-eclectic, (iii) interwar-eclectic, (iv) interwar, and (v) postwar buildings.

4.2. Dataset Annotation

The manual annotation of the dataset produced 11.111 building instances. Each building instance was annotated using axis-aligned bounding boxes and assigned to one of five architectural typologies using Supervisely online annotation tool [24]. In specific experimental configurations, an additional generic class (building class) was included for detection-oriented setups and for two-stage classification pipelines. The dataset reflects realistic urban variability, including differences in viewpoint, lighting, occlusion, and facade articulation. No synthetic data generation was applied during dataset construction. The annotation process was performed collaboratively by a group of three expert architects in Athenian architectural typology in the online platform. Each building instance was labeled based on established typological criteria, while the use of multiple expert annotators aimed to reduce subjective bias and ensure consistent classification across the dataset. Observed disagreements in annotation were resolved through discussion and consensus. Figure 1 illustrates example images with bounding box annotations for all the classes of the dataset. Usually, more than one class was assigned in every image. Supervisely is used solely as an annotation interface, while the typological taxonomy is defined independently based on architectural criteria.

Figure 1. Example image with bounding box annotations for all the classes of the dataset. From top to bottom and left to right: postwar, interwar, neoclassical eclectic, interwar-eclectic, neoclassical. The class colors as appear in Supervisely online annotation platform are shown in bottom right image.

The dataset is partitioned into training, validation, and test subsets, which are used consistently across all experiments involving model training and fine-tuning. It is partitioned into mutually exclusive training, validation, and test sets at the image level, ensuring that no image appears in more than one subset. Care was also taken to avoid placing visually similar views of the same building across different splits. The split follows the proportions defined in the original publication to ensure comparability of results. Specifically, 2.344 images are used for training, 502 images for validation, and 503 images for testing. The corresponding number of labeled building instances is 6.222 in the training set, 2.425 in the validation set, and 2.464 in the test set. The distribution of building instances across architectural classes is imbalanced, reflecting the actual composition of the Athenian built environment. Postwar buildings constitute the largest class, while Neoclassical buildings are the least frequent. This imbalance is preserved intentionally, as it represents a realistic urban typology distribution rather than an artificially balanced dataset, as reported in Table 1.

Table 1. Number of total images and number of labeled boxes for the different classes in the three sub-datasets used (training, validation and test sets). The total number of images in each set appears in bold.

Class	Training Set	Validation Set	Test Set
Total images	2344	502	503
Interwar-eclectic	553	198	200
Postwar	2547	1112	1119
Building	1395	623	629
Neoclassical-eclectic	767	239	249
Interwar	569	151	163
Neoclassical	391	102	104
Total labeled boxes	6222	2425	2464

Table 1 presents the dataset statistics reports for each architectural class, the number of annotated building instances in the training, validation, and test subsets, as well as the total number of images per split. The table clarifies that the reported counts correspond to bounding box instances, not images, since multiple buildings may appear in a single image. For all experiments involving model training or fine-tuning, the predefined training and validation splits are used. Hyperparameter tuning and early stopping decisions rely exclusively on the validation dataset. The test set is reserved strictly for final performance evaluation and is not accessed during training. This protocol is applied consistently across all YOLO-based approaches and two-stage pipelines to ensure methodological consistency and fair comparison. In the two-stage approach, the annotation workflow differs from that of the unified detection–classification setup. In the first stage, a YOLO model is trained to detect buildings as a single object class using the original bounding box annotations. In the second stage, a separate YOLO-based classifier assigns one architectural typology label to each detected building. Since the classification model operates at the image level and does not predict bounding boxes, a different dataset format is required. To construct this dataset, the original annotations are used to automatically crop individual building instances from the source images based on their bounding boxes. These cropped images are stored on disk as a separate classification dataset. Each cropped image contains a single building and is associated with a single architectural label. Corresponding label files are generated accordingly and used for training and evaluating the classification model.

4.3. Metrics

Model performance is evaluated using standard object detection and classification metrics. Metrics are computed on the validation set for monitoring and hyperparameter tuning, and on the test set to assess generalization. Evaluation distinguishes explicitly between detection performance (localization of buildings) and classification performance (architectural typology assignment), as well as their combined effect.

During training, YOLO optimizes a composite loss consisting of bounding box loss (localization accuracy), classification loss (typology discrimination), and Distribution Focal Loss (DFL) for refined box regression. These losses guide convergence but are not reported as final performance measures. Detection performance is assessed using precision, recall, F1-score and mean Average Precision (mAP). Precision measures the proportion of predicted building bounding boxes that correspond to true buildings, while recall reflects the proportion of annotated buildings successfully detected. The F1-score balances false positives and missed detections. mAP evaluates localization accuracy across Intersection over Union (IoU) thresholds; mAP@0.5 reflects moderate alignment, while mAP@0.5:0.95 provides a stricter assessment of bounding box quality. Classification performance is evaluated using class-level accuracy, precision, recall, and confusion matrices. The confusion matrix highlights systematic misclassification between visually similar architectural typologies and reveals the influence of class imbalance. The normalized confusion matrix enables relative comparison across classes. Background regions are treated as non-target areas during detection. False positives in the background class correspond to non-building elements (e.g., vegetation or facade fragments) incorrectly detected as buildings. Monitoring background false positives is particularly important in dense urban scenes where repetitive structural patterns may trigger false detections. Integrated detection–classification accuracy reflects realistic deployment conditions: a prediction is considered correct only if both the bounding box sufficiently overlaps the ground-truth instance and the architectural label is correct. Error analysis focuses on false positives, false negatives, and typology confusion matrices. Given the multiple objects per image and class imbalance, mAP and F1-score provide a more informative evaluation than accuracy alone. Inference time per image is reported to assess computational suitability for large-scale urban analysis.

5. Research Methodology

Building detection and architectural typology classification are investigated through a structured experimental framework that evaluates different computer vision approaches and detection–classification configurations, as shown in Figure 2. The methodology defines the model architectures used in the study, the training procedures applied to the annotated dataset, and the experimental setups used to compare alternative formulations of the task. In addition, the evaluation strategy and performance metrics are specified to ensure a consistent and reproducible experimental pipeline. This methodological framework forms the basis for the comparative analysis presented in the subsequent sections. All models are trained using independent training pipelines, and the annotation platform does not constrain the experimental design.

Figure 2. Overview of the experimental pipeline. The workflow includes dataset construction and annotation, followed by three alternative model configurations: closed-set YOLO detection, two-stage YOLO pipeline, and open-vocabulary Grounding DINO under different training settings (zero-shot, LoRA, full fine-tuning). All models are evaluated using consistent detection and classification metrics, enabling a systematic comparative analysis.

5.1. YOLO

YOLO is applied as a single-stage object detection architecture that jointly predicts bounding boxes and typology labels in a single forward pass. Each detected building is treated as an object-level instance and assigned one architectural class. The detector follows a backbone–head architecture, where convolutional feature maps are extracted by the backbone network, aggregated through a multi-scale feature pyramid, and processed by a detection head that produces dense predictions across the image. At each spatial location the model predicts candidate bounding boxes together with objective scores and class probabilities. Training optimizes a multi-component loss combining bounding-box regression, objective estimation, and classification loss, allowing the network to learn localization and typology discrimination simultaneously.

Although pretrained YOLO models are available, they are not trained to recognize architectural typologies in street-view imagery. Moreover, their original training datasets do not include architectural style sub-categories. For this reason, the detector is retrained using a newly annotated dataset containing building bounding boxes and typology labels corresponding to the defined architectural classes. A standard YOLO training framework is used to train the models under controlled experimental settings. The primary configuration in this study is a single-stage detection and classification model trained end-to-end on this domain-specific dataset. This setup preserves the full image context, including neighboring buildings and surrounding facade elements that may contribute to stylistic interpretation. Because architectural features are often distributed across the facade and surrounding structures, maintaining the full scene context supports more reliable typology classification. In addition to the unified model, a two-stage YOLO pipeline is also evaluated. In the first stage, YOLO is trained as a single-class building detector. In the second stage, a separate YOLO classification model assigns typology labels to cropped building bounding boxes extracted from the detection stage. This separation aims to simplify the localization and classification tasks independently. However, the two-stage configuration removes contextual information and introduces error propagation from the detection stage to the classification stage, which can negatively affect overall performance.

Training follows a fixed 70–15–15 dataset split for training, validation, and testing. Several model configurations are explored, including variations in model size (nano, small, medium), input resolution, learning rate, optimizer parameters, and training duration. Experiments with partial layer freezing are also conducted to investigate whether transfer learning from pretrained weights improves convergence. Data augmentation is applied during training to increase robustness to variations in scale, illumination, and viewpoint that occur in urban street-view imagery. Performance is evaluated using standard object detection and classification metrics, including precision, recall, F1-score, mAP@0.5, and mAP@0.5:0.95. Additional analysis is performed using confusion matrices and integrated detection–classification accuracy at the object level. Validation metrics are monitored during training, and final model performance is reported on the independent test set.

5.2. Grounding DINO

Grounding DINO is applied as a text-conditioned object detection architecture that predicts bounding boxes and typology labels through language-guided inference. In contrast to conventional object detectors that directly predict fixed class labels, Grounding DINO receives both an image and a textual query describing the target categories. Detection is performed by identifying image regions whose visual features align with the provided textual phrases. Each detected building region is associated with a phrase describing an architectural style, which is subsequently mapped to a predefined typology class. The initial configuration evaluates a zero-shot setting in which the pretrained model performs detection and classification from textual prompts only, without domain-specific architectural retraining.

To improve domain adaptation, the model is implemented through a custom training framework based on the Hugging Face implementation [25]. The implementation preserves the original phrase-grounded detection formulation of Grounding DINO instead of converting the model into a conventional closed-set classifier. In this formulation, each architectural class is represented by a descriptive phrase, for example “neoclassical column facade.” All class phrases are concatenated into a single caption and provided to the model together with the image. Internally, the caption is processed by a tokenizer that decomposes each phrase into tokens representing smaller text units. For each predicted image region, the model computes token-level alignment scores that measure the similarity between visual features and individual caption tokens. To obtain a score for a complete architectural class, the implementation identifies the token span corresponding to each phrase, defined by the start and end token indices of that phrase in the caption. Token scores within each span are then aggregated to produce a single phrase-level score representing the compatibility between the detected region and the architectural description. In this way, the model first evaluates visual alignment with individual tokens and then aggregates these scores to estimate the correspondence with the complete architectural phrase. This procedure preserves the phrase-grounded detection behavior of Grounding DINO while enabling supervised training for architectural typology recognition.

Next to the zero-shot experiments, two supervised fine-tuning strategies are evaluated. In the LoRA configuration, lightweight low-rank adapter layers are inserted into selected transformer attention modules while the visual backbone and text encoder remain frozen. This allows domain adaptation with a limited number of trainable parameters and reduced computational cost. In the full fine-tuning configuration, LoRA adapters are disabled and larger portions of the visual backbone are updated during training while the text encoder remains fixed. The text encoder remains frozen in both configurations in order to preserve the pretrained language-grounding capabilities of the model and improve training stability on the relatively limited architectural dataset. This design choice is important because updating both visual and textual backbones on a small domain-specific dataset can lead to loss of pretrained semantic structure (catastrophic forgetting). Freezing the text encoder avoids this and maintains the model’s open-vocabulary understanding. Instead, adaptation is restricted mainly to the visual backbone and cross-attention layers (via LoRA in the first setup), ensuring that learning happens in the visual space while the linguistic embedding space remains stable. Since both configurations share the same dataset pipeline, prompt structure, preprocessing procedures, and evaluation code, the comparison between LoRA and full fine-tuning remains controlled. The following configurations are not introduced as novel methods but are used to ensure a consistent and controlled comparison across model types and between detection paradigms.

Training follows the same 70–15–15 dataset split used in the YOLO experiments. To accelerate experimentation during early development, initial runs are performed on a reduced subset of the training dataset. The generic building class is removed in order to avoid dominance of trivial detections and to enforce discrimination between architectural typologies. Prompt design uses a single phrase per class together with negative terms describing common street-scene objects such as vehicles or pedestrians. Threshold sensitivity experiments are conducted to analyze the influence of detection confidence thresholds on the precision–recall trade-off. Training is performed using the Hugging Face Grounding DINO Tiny model. Training runs are typically scheduled for 100 epochs with batch sizes between 2 and 4 and learning rates between 2 × 10⁻⁴ and 2 × 10⁻⁵ depending on the configuration. To ensure stable optimization, a learning rate schedule with linear warmup (first 5–10% of steps) followed by cosine annealing decay is used. The warmup helps stabilize early training, especially in attention modules, while cosine decay gradually reduces the learning rate to improve convergence. In the LoRA setting, adapters are inserted into 18 transformer attention projection layers spanning the encoder text-enhancer attention, decoder self-attention, and decoder text cross-attention modules. Typical LoRA configurations use rank r from 8 up to 16, scaling parameter α from 16 to 32, and dropout 0.05. LoRA allows higher learning rates (around 1 × 10⁻⁴), while full fine-tuning uses lower rates (around 1 × 10⁻⁵) to avoid destabilizing pretrained representations.

The custom framework is designed with reproducibility and traceability in mind. Each experiment stores configuration files, model checkpoints, training logs, and validation outputs in dedicated run directories. This structure enables systematic comparison between experimental configurations and facilitates debugging and result verification. Given the risk of overfitting in this domain-specific and relatively small dataset, early stopping is applied as a key regularization strategy. Performance is evaluated on a validation set after each epoch, and training is stopped if no improvement is observed for 3–5 epochs. While validation loss (Hungarian matching, focal loss, L1, and GIoU) is monitored, the main stopping criteria are mAP and F1 score, as they better reflect real detection and classification quality than regression losses alone. In multiple runs the model achieved reasonable building localization while typology discrimination remained weaker, suggesting that architectural style recognition in this setting is not only a detection problem but also a challenging phrase-grounding task.

6. Experimental Results

6.1. YOLO Experiments and Results

This section presents the experimental evaluation of YOLO-based architectural typology classification from street-view imagery. The objective was to identify a stable and effective configuration for joint building detection and typology classification. Multiple model variants and training configurations were evaluated before selecting the final setup. Indicative results are depicted in Figure 3.

Figure 3. First row: Wrong classification with high probability score. Second row: classification with the highest score is wrong, however again a second detection with the correct classification with lower score is generated. Third row: While the correct class has the highest probability response, there is a similar detection with an adjacent class with the second-best score, showing the difficulty during classification of similar typologies. In all rows, the left image is the ground truth annotation and right one is the YOLO result. Different colors in the bounding boxes correspond to different architectural classes.

6.1.1. Model Selection and Training Configuration

The experiments were implemented using the Ultralytics YOLOv11 framework in Python (v 3.14). Multiple YOLOv11 model scales were evaluated, including Nano (YOLOv11n), Small (YOLOv11s), and Medium (YOLOv11m), in order to analyze the trade-off between detection performance and computational cost. All models were initialized with pretrained weights and retrained on the same dataset using identical training settings. The primary architecture is a single-stage YOLOv11 detection model that jointly performs building localization and architectural typology classification. Additional experiments use a two-stage pipeline consisting of a YOLOv11 single-class detector (single_cls = True option) followed by a YOLOv11 classification model (-cls option). Final model selection and hyperparameter configuration are based on validation performance.

Hyperparameter tuning focused on input resolution, learning rate, optimizer, and number of epochs. An input size of 640 pixels preserved sufficient facade detail without excessive computational cost. Training was performed using the AdamW optimizer with a decaying learning rate schedule and an initial learning rate of 0.001. Training was conducted for 500 epochs. This configuration resulted in stable convergence and no significant overfitting, as verified by validation loss and mAP curves.

Transfer learning with frozen backbone layers was also tested. Freezing early layers reduced adaptability to facade-specific patterns and decreased classification accuracy. Full fine-tuning of all layers consistently outperformed freezing strategies, indicating that architectural features require domain-specific adaptation even at low and mid-level representations.

6.1.2. Single-Stage Detection and Classification

In the single-stage setup, detection and classification are performed jointly within one YOLO model. On the test set, the model achieved a mean Average Precision of 0.97 at IoU 0.5 and 0.93 averaged over IoU thresholds from 0.5 to 0.95. Per-class precision and recall values were consistently high (Table 2), with strongest performance for Postwar and Neoclassical-Eclectic typologies. Confusion was primarily observed between Interwar and Interwar-Eclectic classes, reflecting architectural continuity rather than annotation errors (Table 3).

Table 2. Per-class detection and classification performance of the single-stage YOLO model on the test set. High precision, recall, and F1-scores are observed across all architectural typologies, indicating stable end-to-end behavior. Postwar and Neoclassical-Eclectic achieve the strongest performance, while Interwar and Interwar-Eclectic show slightly lower F1-scores due to confusion between transitional styles. The “Building” class exhibits high recall but lower precision, reflecting its role as a generic category in complex urban scenes. Mean Average Precision values confirm consistent localization accuracy under both moderate (IoU 0.5) and stricter overlap criteria (IoU 0.5–0.95).

Class	Precision	Recall	F1-Score	AP@0.5	AP@0.5:0.95
Interwar-eclectic	0.85	0.93	0.88	0.95	0.93
Postwar	0.96	0.97	0.97	0.99	0.95
Building	0.87	0.96	0.91	0.95	0.89
Neoclassical-eclectic	0.95	0.96	0.95	0.98	0.95
Interwar	0.88	0.89	0.89	0.94	0.91
Neoclassical	0.91	0.99	0.95	0.98	0.94

Table 3. Confusion matrix representing absolute counts of the single-stage YOLO model on the test set. Rows represent predicted classes and columns represent ground-truth labels. High values along the diagonal indicate strong class discrimination across most architectural typologies. Postwar and Neoclassical-Eclectic show particularly high correct classification rates. Misclassifications are primarily observed between transitional styles, especially Interwar and Interwar-Eclectic, reflecting architectural similarity rather than systematic detection failure. Background and generic building predictions mainly affect recall but do not dominate typology confusion.

Predicted/True	Interwar-Eclectic	Postwar	Building	Neoclassical Eclectic	Interwar	Neoclassical	Background
Interwar-Eclectic	160	0	2	20	0	18	0
Postwar	0	1053	12	0	0	51	0
Building	0	16	571	0	1	1	90
Neoclassical-Eclectic	2	0	0	236	0	3	7
Interwar	11	1	0	0	137	0	16
Neoclassical	0	0	1	4	0	98	9
Background	0	11	19	4	1	0	0

False negatives in detection occur mainly in scenes with strong occlusions, truncated facades, or extreme perspective distortion. False positives are associated with urban background elements that share geometric or textural similarities with building facades. Classification errors are dominated by confusion between historically adjacent typologies. These errors are consistent across the dataset and most probably correlate with overlapping architectural features such as facade rhythm, window alignment, and decorative density. The confusion matrix highlights these patterns clearly. Computationally, the single-stage model performs detection and classification in a single forward pass, making it suitable for large-scale or near-real-time analysis.

6.1.3. Two-Stage Detection and Classification

The two-stage configuration separates detection and typology classification into independent models. The single-class detector achieved high localization performance (Table 4). However, when combined with the classifier, integrated end-to-end performance decreased. The approximated integrated F1-score was 0.67, indicating significant error propagation between stages (Table 5).

Table 4. Performance of the YOLO single-class building detector on the test set. The model achieves high precision and recall (0.97 and 0.98), resulting in an F1-score of 0.97. Average Precision remains strong across IoU thresholds, with mAP@0.5 of 0.99 and mAP@0.5:0.95 of 0.94, indicating reliable and stable building localization.

Precision	Recall	F1-Score	AP@0.5	AP@0.75	mAP@0.5:0.95
0.97	0.98	0.97	0.99	0.98	0.94

Table 5. Integrated performance of the YOLO two-stage pipeline on the test set. Although detection accuracy is high, classification errors propagate through the pipeline, resulting in an end-to-end precision of approximately 0.66, recall of 0.68, and F1-score of 0.67. This confirms that typology classification is the main limiting factor in the two-stage setup.

Precision	Recall	F1-Score
0.66	0.68	0.67

The two-stage configuration uses two independent YOLOv11 networks: a single-class detection model for building localization and a separate YOLOv11 classification model trained on cropped building instances. The purpose is to investigate whether separating localization and classification into independent tasks improves typology recognition compared to the unified single-stage detector. In the first stage, the detector identifies building regions within the original urban street-view images without performing architectural typology discrimination. The detected bounding boxes are then used to automatically crop individual building instances from the original images. In the second stage, these cropped building images are provided as input to a separate YOLOv11 classification model that predicts the corresponding architectural typology.

Detection errors are similar to those observed in the single-stage setup. Classification errors increase due to the use of cropped images that lack surrounding architectural context. Transitional typologies are most affected. Bounding box cropping occasionally removes facade elements that are relevant for typology discrimination, such as upper-floor ornamentation or neighboring structural alignment. This leads to systematic misclassification patterns visible in the confusion matrix. These errors highlight the limitations of decoupling detection and classification for architectural typology analysis. The two-stage pipeline also increased computational cost due to repeated inference on cropped regions. Overall, the two-stage configuration did not provide performance advantages and introduced additional complexity (Figure 4).

Figure 4. Examples of correct and wrong classifications with the classification model of the two-stage method. Ground truth labeled images are displayed on the (left), while the results are on the (right). The class number is noted on the top left part of each image. Red squares indicate erroneous classification.

6.1.4. Central Building Evaluation

An additional evaluation was conducted focusing only on the central building in each image. Since central annotations are expert-verified and less affected by peripheral ambiguity, this experiment estimates upper-bound classification performance. Excluding the generic building class, the average precision, recall, and F1-score across typologies were approximately 0.95 (Table 6). These results confirm that architectural typology classification is feasible when the task is restricted to a single, clearly defined building instance. The performance gap between central-building evaluation and full-scene evaluation highlights the impact of annotation noise, contextual ambiguity, and multi-instance complexity in urban environments.

Table 6. Central-building classification performance derived from the single-stage confusion matrix. High precision and recall are observed across all typologies, with an average F1-score of 0.95. Postwar and Neoclassical-Eclectic show near-perfect discrimination, while lower performance for Interwar reflects confusion with adjacent transitional styles. These results represent an upper-bound estimate under minimal annotation ambiguity.

Class	Precision	Recall	F1-Score
Interwar-eclectic	0.88	0.92	0.90
Postwar	1.00	1.00	1.00
Building	0.98	0.98	0.98
Neoclassical-eclectic	0.92	0.87	0.89
Interwar	0.96	0.97	0.96
Neoclassical	0.95	0.95	0.95

6.2. Grounding DINO Experiments and Results

This section presents the experimental evaluation of Grounding DINO for architectural typology classification. Three settings were examined: zero-shot inference, LoRA-based fine-tuning, and full end-to-end fine-tuning. The objective was to assess whether an open-vocabulary vision–language detector can achieve reliable building localization and typology discrimination in street-view imagery.

6.2.1. Zero-Shot Inference

The zero-shot baseline was evaluated as an exploratory baseline experiment using Grounding DINO with textual prompts corresponding to the five architectural typologies. A predicted box was considered correct when IoU > 0.5 and the mapped text label matched the ground-truth class. Experiments on a subset of 50 test images demonstrated very limited performance, with almost no correct typology assignments despite several detected buildings. Out of 108 ground-truth objects, only one true positive satisfied both localization and classification criteria. A large number of wrong-class assignments and false positives were observed. It should be noted that, to avoid bias, the subset included images containing multiple architectural typologies and typical urban occlusions, providing a representative sample of the dataset conditions.

These results indicate that architectural typology classification is not reliable in a pure zero-shot setting. Although some buildings were localized with sufficient IoU, the assigned textual labels did not correspond to the correct typology. The main limitation is vocabulary mismatch. Architectural period names are rarely represented explicitly in large-scale pretraining data, leading to low confidence and unstable class assignment. Based on these findings, extending the evaluation to the full dataset was not expected to provide additional insight, and the analysis therefore focused on fine-tuning strategies.

6.2.2. LoRA Fine-Tuning

To improve adaptation, LoRA-based fine-tuning was applied while keeping the backbone and text encoder frozen. Training was conducted on a reduced subset of the dataset in order to evaluate convergence behavior under limited adaptation capacity. Despite stable training, performance remained low. The best checkpoint achieved macro-F1 of 0.07, mAP@0.5 of 0.05, and mAP@0.5:0.95 of 0.04. The number of false positives (FP = 5397) was substantially higher than true positives (TP = 116). Although localization improved compared to zero-shot inference, class discrimination remained weak. Confusion matrices showed bias toward a small subset of labels, and many detections corresponded to visually irrelevant regions. The limited adaptation capacity of few-layer LoRA, combined with frozen backbone and text encoder, was not sufficient to overcome the domain gap between general object detection pretraining and architectural typology recognition.

Visual inspection of the LoRA-adapted model outputs on the validation set further confirms the quantitative results. As illustrated in Figure 5, the model generates a large number of candidate detections with relatively low confidence scores. Many of these detections correspond to background regions or repeated overlapping boxes along building facades, indicating a tendency toward over-detection. While some building structures are localized correctly, the predictions often include multiple redundant bounding boxes that do not correspond to distinct architectural instances. This behavior is consistent with the high number of false positives observed in the quantitative evaluation and suggests that the LoRA adaptation was insufficient to enforce reliable architectural typology discrimination. The model appears to capture certain structural patterns associated with buildings, but the limited adaptation capacity of the few-layer LoRA configuration does not adequately constrain the detection process, resulting in unstable localization and weak class separability.

Figure 5. Visual examples of the LoRA-adapted Grounding DINO model on the validation set using the best checkpoint. The images illustrate the large number of low-confidence detections and overlapping bounding boxes produced by the model. While some building structures are localized, many predictions correspond to background regions or redundant detections, reflecting the high false-positive rate observed in the quantitative evaluation.

6.2.3. Full Fine-Tuning

Full fine-tuning was then performed using the entire training split. In this configuration, the visual backbone was trained while the text encoder remained frozen. LoRA was disabled to allow full weight updates. The generic building class was excluded to focus on typology discrimination. At the best validation epoch, performance improved substantially compared to LoRA. The model achieved macro-F1 of 0.65, mAP@0.5 of 0.67, and mAP@0.5:0.95 of 0.56. The confusion-matrix classification accuracy reached approximately 0.88. Four classes achieved F1-scores around 0.70 or higher, while the Postwar class remained lower (F1 ~ 0.48) due to false positives and missed detections (Table 7). The confusion matrix shows interpretable inter-class confusions, mainly between Interwar and Interwar-Eclectic, and between Neoclassical-Eclectic and Interwar-Eclectic. These errors reflect architectural similarity rather than random misclassification. However, background false positives remain significant. A notable proportion of background regions are assigned to the Postwar class, indicating that this category acts as a fallback prediction under uncertainty. Later training epochs showed increasing false positives and decreasing macro-F1, suggesting overfitting and confidence inflation for visually ambiguous regions.

Table 7. Confusion matrix of the full fine-tuned Grounding DINO model on the validation set. Rows represent predicted classes and columns represent ground-truth labels. The matrix shows strong diagonal values for most typologies, while significant background-to-Postwar and Interwar-Eclectic–Interwar confusions highlight remaining classification and background separation challenges.

Predicted/True	Interwar-Eclectic	Postwar	Neoclassical-Eclectic	Interwar	Neoclassical	Background
Interwar-Eclectic	35	0	8	3	0	0
Postwar	0	190	0	2	0	310
Neoclassical-Eclectic	4	1	70	0	4	4
Interwar	23	2	0	71	0	50
Neoclassical	3	0	6	0	43	9
Background	4	95	11	6	4	0

Visual inspection of the full fine-tuning results shows improved stability compared to the LoRA configuration as shown in Figure 6. In several validation images the model successfully localizes the main building instance and produces bounding boxes that better align with the ground-truth annotations. In some cases, the architectural typology is also correctly identified, indicating that the model has learned meaningful visual patterns related to the defined classes. However, the results still reveal several limitations. Additional detections with low confidence scores appear in background regions, and multiple partially overlapping boxes are sometimes generated around the same building structure. Furthermore, typology confusion between visually similar classes is occasionally observed, reflecting the inherent ambiguity of transitional architectural styles. These qualitative observations are consistent with the quantitative evaluation and indicate that while full fine-tuning improves localization stability and class discrimination, the task remains challenging due to visual similarity between architectural typologies and complex urban scene composition.

Figure 6. Visual examples of the fully fine-tuned Grounding DINO model on the validation set. The model shows improved localization of the main building structures and occasionally correct typology predictions. However, additional low-confidence detections and typology confusion between visually similar classes are still observed, highlighting the difficulty of fine-grained architectural classification in complex urban scenes.

It should be clearly noted that LoRA and fine-tuning configurations are evaluated as standard adaptation strategies to ensure a controlled comparison. They are not introduced as novel contributions but serve to assess model behavior under consistent training conditions.

6.3. Detection and Classification Comparison

The YOLO-based single-stage model provides stable and high detection performance, with mAP@0.5 close to 0.99 and strong per-class F1-scores. Localization is reliable across urban scenes, and most errors originate from confusion between transitional architectural styles rather than from missed detections. The two-stage YOLO pipeline achieves similarly high detection precision and recall in the first stage, but integrated end-to-end performance decreases due to classification error propagation. Overall, supervised closed-set training enables consistent class separation and controlled behavior.

Grounding DINO behaves differently. In the zero-shot setting, both detection and classification performance remain very low, mainly due to vocabulary mismatch between architectural typology terms and large-scale pretraining categories. LoRA fine-tuning improves localization slightly but fails to produce reliable class discrimination, with high false positive rates. Full fine-tuning significantly improves performance, reaching macro-F1 around 0.65 and validation classification accuracy close to 0.88. However, background false positives and class imbalance effects remain prominent, particularly for the Postwar class. Even under full adaptation, performance does not reach the level of the supervised YOLO models.

From a methodological perspective, YOLO operates as a closed-set detector optimized directly for the predefined architectural classes. This alignment between training objective and evaluation task explains its stable behavior. Grounding DINO, in contrast, is designed for open-vocabulary grounding. While flexible, it requires substantial adaptation to handle fine-grained architectural typologies and remains sensitive to prompt design and dataset characteristics.

Overall, for architectural typology classification in structured urban imagery, the supervised YOLO framework demonstrates higher robustness, stronger localization accuracy, and more stable end-to-end classification performance compared to the vision–language alternative (Table 8).

Table 8. Quantitative comparison of YOLO-based and Grounding DINO-based approaches under different training settings. The table highlights detection accuracy, classification performance, and main limitations of each configuration, demonstrating the robustness of supervised closed-set detection compared to open-vocabulary grounding methods for architectural typology classification.

Method	Setting	mAP@0.5	Macro F1	Classification Accuracy	Main Limitation
YOLO	Single-stage	~0.99	High (~0.9)	High	Transitional class confusion
YOLO	Two-stage	High detection	~0.67	Moderate	Error propagation
Grounding DINO	Zero-shot	Very low	Very low	Very low	Vocabulary mismatch
Grounding DINO	LoRA	~0.05	~0.07	Low	High false positives
Grounding DINO	Full fine-tuning	~0.67	~0.65	~0.88 (val)	Background false positives

Further comparison of the YOLO and Grounding DINO results shows consistent confusion patterns across both methods. Although the architecture differs, similar classification errors appear for specific architectural typologies. This suggests that part of the error is not model-specific but related to intrinsic visual similarity between classes. In both approaches, the strongest confusion occurs between Interwar and Interwar-Eclectic, and between Neoclassical-Eclectic and adjacent transitional styles. These typologies share facade composition patterns and structural characteristics. The class boundaries between them are gradual rather than visually clearly distinct.

Specifically, Interwar and Interwar-Eclectic typologies frequently exhibit simplified volumetric compositions and reduced ornamentation compared to earlier styles. However, Interwar-Eclectic buildings often retain residual decorative elements, such as modest facade articulation, ornamental balconies, or variations in window framing inherited from eclectic traditions. Additionally, similarities in window-to-wall ratios and facade rhythm further reduce visual separability. These overlapping features result in gradual stylistic transitions, making strict classification boundaries inherently ambiguous even for human experts.

Similarly, confusion between Neoclassical and Neoclassical-Eclectic typologies arises from the coexistence of classical compositional principles with increasingly complex decorative programs. While both categories maintain symmetry, proportional organization, and reference to classical orders, Neoclassical-Eclectic buildings often incorporate additional ornamental elements, such as elaborate cornices, decorative railings, or mixed stylistic motifs. In practice, these additions do not fundamentally alter the underlying structural logic of the facade, leading to high visual similarity. As a result, the distinction between the two typologies becomes dependent on subtle variations in decorative density and stylistic integration rather than clearly separable visual features.

In contrast, distinct defined styles such as Postwar and pure Neoclassical achieve higher precision and recall in both models. These classes appear more distinctive and have architectural features that are easier to separate in feature space. The similarity of confusion patterns indicates that the main limitation lies in the fine-grained separability of transitional architectural styles rather than in the detection mechanism itself. The dataset reflects real architectural characteristics, where stylistic categories do not form strictly separable clusters. Therefore, part of the classification error represents structural ambiguity rather than model instability. Overall, improving typology classification performance would likely require additional contextual metadata or refined labeling strategies, rather than solely modifying the detection architecture.

7. Discussion

This work evaluated two approaches for architectural typology classification from street-view imagery: supervised YOLO-based object detection and the open-vocabulary model Grounding DINO. For YOLO, multiple configurations were examined, including single-stage joint detection and classification and a two-stage pipeline. Detection performance was stable across all experiments, and buildings were reliably localized in complex urban scenes. Performance differences were mainly driven by classification rather than detection. Classification results showed consistent confusion between typologically adjacent classes, particularly the eclectic variants. These patterns remained stable across training, validation, and test sets, indicating that errors are primarily caused by architectural ambiguity rather than overfitting. The single-stage YOLO model achieved the best integrated performance, while the two-stage pipeline suffered from error propagation and loss of contextual information. Most misclassifications occurred under correct detection and were visually plausible, reflecting overlapping facade characteristics.

Grounding DINO was evaluated in both zero-shot and fine-tuned settings. Zero-shot inference was unreliable for fine-grained typology recognition. Fine-tuning improved validation performance but remained below the reliability required for final evaluation. LoRA-based adaptation did not perform satisfactorily. Grounding DINO shows stronger model-specific limitations related to vocabulary dependence and weak semantic separation of architectural styles.

YOLO outperforms Grounding DINO when the task involves clearly defined classes and requires stable, high-precision detection and classification. YOLO provided reliable building localization and consistent typology recognition under supervised training. It is particularly suitable when labeled data is available and classification is predictable. Grounding DINO may be preferred in exploration scenarios where categories are not fixed in advance and open-vocabulary flexibility is required. However, for fine-grained architectural typologies, zero-shot DINO was not sufficiently reliable, and even fine-tuning did not match YOLO’s stability.

YOLO offers high inference speed and efficient large-scale processing, making it suitable for urban-scale deployment. Its architecture is optimized for real-time detection and structured class prediction. Grounding DINO, in contrast, provides greater conceptual flexibility through text-based queries, allowing dynamic category definitions. This flexibility comes at the cost of reduced stability, higher computational complexity, and sensitivity to vocabulary design. In this study, the speed and robustness of YOLO outweighed the flexibility benefits of DINO for the specific task.

For practical deployment in urban analysis systems, supervised YOLO-based models are currently more suitable. They provide consistent performance, clear evaluation metrics, and scalable processing of large image collections. Grounding DINO would require careful prompt engineering, calibration, and stronger domain adaptation before reliable deployment. In operational environments where decisions depend on stable typology classification, YOLO represents the safer and more implementable solution under current technological conditions. Overall, supervised YOLO-based models provide a stable and effective solution for architectural typology classification, while open-vocabulary grounding approaches are currently not suitable for this task under the given constraints.

Finally, a final point that is important to emphasize, is that feature representations learned by the models align with real architectural ambiguities, as highlighted in Section 6.3, rather than artificial classification noise.

8. Conclusions and Future Directions

This study investigated the automatic detection and classification of Athenian architectural typologies from street-view imagery using deep learning-based computer vision models. Five typologies were considered: Neoclassical, Neoclassical-Eclectic, Interwar, Interwar-Eclectic, and Postwar. The evaluation followed a controlled experimental protocol with fixed training, validation, and test splits to ensure reproducibility and comparability.

The primary method was a YOLO-based object detection and classification framework. Detection-only, integrated end-to-end, and classification-oriented analyses were performed to separate localization performance from semantic recognition. Single-stage YOLO models demonstrated stable building detection and reliable typology classification. Most classification errors were associated with architectural ambiguity between transitional styles rather than localization failures or optimization instability. Two-stage pipelines showed reduced performance due to error propagation and loss of contextual information. Medium-scale YOLO models provided the most balanced trade-off between accuracy and computational efficiency. An additional central-building evaluation confirmed that classification performance improves when annotation noise and peripheral instances are reduced. These results represent an upper bound under controlled conditions and complement full-scene evaluation.

Grounding DINO was evaluated as an open-vocabulary alternative. Zero-shot inference was not reliable for fine-grained typology recognition. Although supervised fine-tuning improved validation metrics, performance remained sensitive to vocabulary design and class overlap and did not reach the stability achieved by supervised YOLO-based models. These findings indicate that open-vocabulary grounding models are currently not sufficient for architectural typology classification in facade-based RGB imagery without strong domain supervision.

A first conclusion is that architectural ambiguity defines a fundamental performance limit for all evaluated approaches. Transitional typologies share visual characteristics that cannot be fully separated using single-view RGB data. Within this constraint, supervised object detection frameworks currently provide the most effective and scalable solution for urban-scale typology mapping.

As a central conclusion, we may underline that this work could easily be positioned within the context of urban heritage documentation, where architectural typology classification serves as a structured methodological framework for recording, analyzing, and monitoring the built environment at scale. In this direction, the research presents a valuable approach to heritage documentation through street-level architectural recording, enabling systematic observation of urban fabric through widely available imagery sources. Rather than treating building categorization as a purely visual recognition task, the proposed approach integrates architectural knowledge and typological interpretation into a coherent classification process. This facilitates the organization of heterogeneous urban data into meaningful historical and morphological categories, supporting scalable and reproducible documentation workflows.

Future work can focus on improving both flexibility and scalability of architectural typology classification systems. Open-vocabulary cultural heritage systems could combine supervised training with language-based interaction, allowing users to search or define architectural styles in a more intuitive way. The use of multilingual prompts may help adapt models to different countries and architectural traditions. Cross-city transfer learning should be studied to understand how models trained in Athens perform in other urban environments and how much additional data is required for adaptation. In addition, scalability can be extended using UAV imagery and large public platforms such as Mapillary [26], which provide large volumes of urban images. Integrating the proposed methods into such image databases would enable automated and continuous mapping of building typologies and urban characteristics at neighborhood and city scales.

An additional direction for future work that can take advantage of the text-conditioned detection capabilities of grounding-based models by developing interactive applications that allow users to query architectural typologies through natural language. Because phrase-grounded detectors operate through text prompts, users could search for buildings using descriptive queries such as architectural features, stylistic elements, or historical references instead of selecting from a fixed list of classes. Such an interface could improve usability for historians, planners, and the general public, while also enabling exploratory analysis of architectural patterns that were not explicitly defined during model training.