A Multimodal AI and Visual-Programming Framework for Sketch-Based BIM: Formalization and Empirical Validation

Abstract

Early-stage architectural design is still dominated by hand-drawn 2D floor-plan sketches, and converting these sketches into object-based Building Information Modeling (BIM) models remains a manual bottleneck that delays downstream simulation, costing, and coordination. Recent advances in multimodal artificial intelligence (vision-language models, code-generating large language models) and visual programming for BIM (Dynamo, Grasshopper, Refinery) have not yet been unified. Objective: We present and empirically evaluate a semi-automatic framework in which a vision-language model (VLM) emits executable visual-programming code (Dynamo-compatible DesignScript) that is wrapped into a node graph and executed in Dynamo to drive Revit, producing native BIM elements; the IFC export is independently validated in BlenderBIM as a visual sanity check. This is a proof-of-concept evaluation (n = 5 case studies), not a claim of production-scale deployment. Methods: The seven-module pipeline (sketch input, preprocessing, VLM interpretation, code-to-graph translation, Dynamo + Revit execution, IFC export with a BlenderBIM sanity check, and human-in-the-loop review with a feedback loop) is formalized mathematically—a line-sharpness function, a probabilistic AI-success model, and a closed-form architect-effort expression—and evaluated on five case studies (a Medipol student sketch, a public RPLAN sample, a tablet-drawn two-story house, a CubiCasa5K sample, and a real Istanbul apartment) against three baselines: a CNN recognizer (MuraNet), a JSON-emitting LLM pipeline (Sketch2BIM), and a hand-coded visual-programming framework. The primary VLM is Qwen-VL 9B (open-weight, Apache 2.0); GPT-4V and Claude 3.5 Sonnet are ablated on Case A only as closed-weight sensitivity checks. Results: Across the five cases, the framework achieves wall/opening F1 of 0.84–0.91 (mean 0.88), IFC schema validity of 100% in every case, a 56–61% reduction in time-to-editable-BIM, and 58–74% fewer architect edits than fully manual modeling, outperforming all three baselines on the architect-edits-required metric. The closed-form model predicts a 72% effort reduction under nominal parameters but overestimates the observed 56–61% by 11–16 percentage points; once the empirically observed architect-edit time is substituted, the calibrated model matches the observed range. Numbers are means over three runs per case. Conclusions: Unifying multimodal AI code generation with visual programming for BIM produces a more robust, editable sketch-to-BIM pipeline than the JSON-schema approaches in the current literature. Caveat: the evidence is a proof-of-concept scale (n = 5); broader validation (≥20 cases, cross-practice) is a key objective for future research.

1. Introduction

Early-stage architectural design is dominated by hand-drawn 2D floor-plan sketches. In our pedagogical and professional experience at Istanbul Medipol University, we observe that even in highly digitized practices, architects still default to paper and pencil for rapid ideation; they communicate design ideas on A4 paper with an HB pencil, with informal annotations such as room names (“salon”, “WC”), dimension hints, and arrows pointing to design intentions [1,2]. The translation of these sketches into object-based Building Information Modeling (BIM) models is a manual, time-consuming bottleneck: a junior architect typically requires 6–12 h to convert a single A4 sketch into a Revit model with walls, doors, windows, and rooms [1,2]. The downstream pipeline—structural analysis, energy simulation, cost estimation, code compliance—all wait on the BIM model (Figure 1).

Figure 1. The problem this paper addresses (A) and the process it proposes (B). Manual conversion of a hand-drawn sketch into an object-based BIM model is a 6–12 h, error-prone, non-reproducible bottleneck (A); the seven-module framework detailed in Section 3.1 replaces it with AI-authored visual programming (Dynamo + Revit) plus a short human review (B). See Section 1 and Section 3.1.

1.1. The Opportunity Created by Multimodal AI

The 2023–2026 wave of large language models (LLMs) and vision-language models (VLMs) has produced a new generation of tools that can read images, understand spatial layouts, and emit structured text or code [3,4,5,6,7,8,9,10,11,12,13]. Of particular relevance to architecture is the emergence of:

• Multimodal LLMs (MLLMs) such as GPT-4V, Claude 3.5 Sonnet, and Qwen-VL that can read a floor-plan sketch and answer questions about its contents [3,4]. We use Qwen-VL 9B as the primary model in all reported runs (open-weight, Apache 2.0; reproducible weights listed in Appendix C) because it is the only candidate in this family that is open-weight at the time of writing; GPT-4V and Claude 3.5 Sonnet are ablated as closed-weight alternatives on Case A only and reported in Appendix E for sensitivity, not as primary runs.
• Code-generating LLMs that can emit executable code in response to a natural-language specification. Recent work has shown that LLMs can be embedded into BIM authoring tools (Dynamo, Revit, Vectorworks) as copilots [4,5], and that multi-agent MLLM pipelines can convert a hand-drawn sketch into a 3D BIM model [14].
• Domain-specific LLMs for BIM such as Qwen-BIM [8], fine-tuned on BIM corpora, and BIM-specific agents that use the Model Context Protocol (MCP) to query IFC files [9].
• LLM-driven code-compliance checking that uses retrieval-augmented generation (RAG) over building codes [10,13].

These advances (Table 1) suggest that the manual conversion of a sketch to a BIM model can be semi-automated. Prior work, however, has one of two limitations:

Table 1. Tools and model families referenced in Section 1.1, grouped by role in the pipeline.

Category	Examples	Role in This Paper
General-purpose MLLM/VLM	GPT-4V, Claude 3.5 Sonnet, Qwen-VL	Reads the sketch raster + SVG and emits DesignScript (Qwen-VL 9B is the primary model; GPT-4V/Claude 3.5 Sonnet are ablated on Case A only, Appendix E)
Domain-specific BIM LLM	Qwen-BIM [8]	Fine-tuned on BIM corpora; referenced as related work, not used in our pipeline
LLM + protocol agent	MCP4IFC [9]	Queries IFC files via the Model Context Protocol; referenced as related infrastructure
Code-compliance LLM	RAG-based checkers [10,13]	Checks generated BIM against building codes; motivates Section 5.5 future work
Visual-programming environment	Dynamo, Grasshopper, Refinery, Rhino.Inside	Dynamo is the execution environment for the AI-emitted DesignScript in this paper
Independent IFC viewer	BlenderBIM (IfcOpenShell)	Provides the second, Revit-independent visual sanity check (Section 3.1, Module 6)

1. The AI emits a JSON schema, not code. The schema is then post-processed by a separate tool (Revit API, IfcOpenShell). This separation has a known weakness: the AI is unaware of the visual-programming paradigm that BIM practitioners actually use [5,8,14].

2. The visual-programming graph is hand-coded. Practitioners write Dynamo or Grasshopper scripts to drive Revit, but no AI assistance is provided for the script-writing step itself [15].

1.2. The Opportunity Created by Visual Programming

In parallel, the BIM authoring community has adopted visual programming as the dominant paradigm for parametric and generative design. Visual-programming tools—Dynamo, Grasshopper, Refinery, Rhino.Inside—allow architects to build complex BIM workflows by connecting nodes on a canvas, rather than by writing imperative code [15]. Visual programming has also been examined as a medium for teaching computational design concepts [15], while point-cloud-based scan-to-BIM workflows have automated the creation of BIM elements from as-built data [16,17].

A key insight is that the intermediate representation matters. Unlike Ratul et al. [14] whose sketch-to-BIM pipeline we initially intended to replicate in our lab, we found that direct VLM-to-JSON generation is too brittle for production workflows, forcing the AI to invent a schema no BIM tool understands natively. In contrast, code—specifically, the visual-programming language (DesignScript) of the target tool—maps to a representation the practitioner already uses. The architect sees a node graph, can edit it, and can re-run it. The same insight has been articulated in the broader LLM-for-code literature: the LLM should emit code in a language the human already speaks.

1.3. Research Question and Contribution

The research question that motivates this paper is: Can a multimodal AI system, paired with a visual-programming bridge for BIM and an independent IFC viewer, semi-automatically convert hand-drawn 2D sketches into object-based BIM models with a measurable reduction in architect effort?

We answer this question with the following primary and secondary contributions:

1. Primary contribution: The visual-programming bridge—A new framework (Section 3) in which a VLM emits executable visual-programming code (DesignScript) that is wrapped into a node graph and executed in Dynamo + Revit. This intermediate representation serves as an editable bridge, replacing direct JSON-schema generation. The output is validated independently in BlenderBIM.

2. Supporting mathematical formalization (Section 3.2) of the pipeline: (i) a line-sharpness function that quantifies the preprocessing gain, (ii) a probabilistic model of the AI’s success as a function of sketch complexity, and (iii) a closed-form expression for the expected reduction in architect effort.

3. A prompt-engineering protocol for the VLM, with concrete prompt templates and worked examples for the Dynamo/Revit/BlenderBIM workflow.

4. An empirical evaluation on five case studies (Section 4) with five metrics: wall/opening F1, IFC schema validity, time to editable BIM, architect edits required, and visual-programming execution time.

Novelty relative to the closest prior work. To make the differentiation explicit: unlike Ratul et al. [14], whose Sketch2BIM pipeline emits a JSON schema that a separate tool must post-process, our framework emits executable visual-programming code directly, so the intermediate artifact is one that BIM practitioners can inspect, edit, and re-run. Unlike Jeong et al. [18,19], whose visual-programming graphs are hand-coded with no AI assistance, our VLM authors the Dynamo graph automatically from a hand-drawn sketch. And unlike CNN-based recognizers such as MuraNet [20], which output a wall/door/window classification that still requires a separate, hand-engineered conversion step to reach BIM, our pipeline produces native Revit elements directly. This combination—AI-authored (not hand-coded) executable code (not a JSON schema) that drives native BIM authoring (not classification alone)—is, to our knowledge, not previously demonstrated in the sketch-to-BIM literature (Section 2.6 details the three specific gaps in the literature that this closes).

The paper is structured as follows. Section 2 reviews the literature, presents the PRISMA 2020 systematic search that grounds the study, and identifies the gap our work addresses. Section 3 presents the framework with its mathematical formalization, the prompt template, and a worked example of a Dynamo graph produced by the AI. Section 4 reports the case-study evaluation, the prompt-and-output examples, and the comparison with baselines. Section 5 discusses threats to validity and future directions. Section 6 concludes.

The framework, all code, and all case studies are released under Apache 2.0 (code) and CC-BY 4.0 (data), available from the corresponding author upon reasonable request, to encourage reproduction and extension (see Appendix C for code, weights, dataset, and Docker details).

2. Related Work

2.1. Systematic Search Protocol (PRISMA 2020)

We conducted a PRISMA 2020 systematic search of the literature on sketch-to-BIM conversion, with an explicit focus on the intersection of (a) sketch recognition, (b) visual programming, and (c) BIM authoring. The search was performed on 2026-06-28 across three open-access databases—OpenAlex (primary), Semantic Scholar (cross-check), and arXiv (pre-prints)—using six query families:

• Q1: (“sketch-to-BIM” OR “sketch to BIM” OR “hand-drawn BIM” OR “sketch modeling”) AND (“Dynamo” OR “DesignScript” OR “visual programming” OR “Refinery” OR “Grasshopper” OR “Rhino.Inside”)
• Q2: (“Dynamo” OR “DesignScript”) AND (“Revit” OR “BIM” OR “IFC” OR “BIM authoring”) AND (“automatic” OR “generative” OR “AI” OR “machine learning”)
• Q3: (“BlenderBIM” OR “IfcOpenShell”) AND (“sketch” OR “floor plan” OR “BIM authoring” OR “IFC generation”)
• Q4: (“Grasshopper” OR “Rhino.Inside.Revit”) AND (“floor plan” OR “BIM” OR “generative design”) AND (“AI” OR “LLM” OR “neural”)
• Q5: (“LLM” OR “GPT” OR “Copilot”) AND (“Revit API” OR “Dynamo” OR “DesignScript” OR “IFC” OR “BIM authoring tool”)
• Q6: (“code generation” OR “program synthesis”) AND (“BIM” OR “Revit” OR “IFC” OR “Building Information Modeling”)

PRISMA 2020 funnel (Figure 2; Table 2); Appendix D. Records were retrieved in two waves—a broader v0.1 baseline (n = 102, 8 June 2025) and a Dynamo-aware wave (n = 561 unique OpenAlex records, 28 June 2026, Q1–Q6)—and combined without duplication, since the two waves target distinct, non-overlapping DOI populations (see Table 2 for the full stage-by-stage rationale).

Figure 2. PRISMA 2020 funnel. 561 records identified across Q1–Q6; 17 seed + 90 carryover = 124 corpus. See Section 2.1.

Table 2. PRISMA 2020 stage counts (Section 2.1).

Stage	n	Rationale
Records identified (Q1–Q6, OpenAlex)	561	Dynamo-aware queries only; the v0.1 baseline used different (broader, sketch-focused) queries
Records retained after cross-database deduplication (OpenAlex ∩ Semantic Scholar ∩ arXiv)	561	The Dynamo-aware queries are narrowly scoped to visual-programming + BIM intersections, which retrieve a fundamentally different DOI population than the v0.1 sketch-focused queries; Semantic Scholar reconciliation confirmed 0 true duplicates across the two waves. No overlapping DOIs were found due to the distinct categorical separation between the two query waves.
Title/abstract screened	561	Two independent screeners (authors), Cohen’s κ = 0.91
Excluded at title/abstract (out of scope: pure synthetic-floor-plan reconstruction, pure point-cloud scan-to-BIM, pure CAD-to-BIM)	505	Standard scope exclusion
Full-text assessed for eligibility	56
Excluded at full text—wrong modality (point-cloud only)	22	No sketch or 2D input
Excluded at full text—no BIM authoring (sketch recognition only)	17	Recognition without BIM element creation
Full text retained after exclusion	17
Plus: manually seeded foundational papers	17	Seeding criteria: (1) software-artifact releases not indexed as empirical studies (Dynamo/DesignScript language reference, IfcOpenShell/BlenderBIM toolkit, Qwen-BIM model card); (2) foundational visual-programming references (Grasshopper, Rhino.Inside); (3) protocol/infrastructure papers (MCP4IFC). All 17 are disclosed in the bibliography with a † prefix.
Carried over from the v0.1 baseline (sketch + BIM, 8 June 2025)	90	These 90 papers were full-text assessed and included in the v0.1 wave; they are carried forward without re-screening because their eligibility was established under the same inclusion criteria.
Total corpus (n)	124	17 (new wave) + 17 (seeded) + 90 (carry-over)

Reading. The Q1–Q6 queries are narrowly scoped to the sketch + visual-programming/BIM intersection; papers on sketch-to-BIM alone are re-introduced via the v0.1 carry-over, and visual-programming-only papers are excluded at full text (Table 2 gives the full per-stage rationale). Seventeen foundational software-artefact papers (Dynamo/DesignScript, IfcOpenShell/BlenderBIM, Qwen-BIM) are seeded manually and disclosed with a † prefix in the bibliography. The resulting 124-paper corpus is well above the PRISMA 2020 minimum of 30 and supports the bibliometric snapshot in Figure 3: CNN 28 (22.6%), LLM 25 (20.2%), Visual Programming 14 (11.3%), Transformer 12 (9.7%), Diffusion 11 (8.9%), GNN 8 (6.5%), Hybrid 6 (4.8%), GAN 5 (4.0%), Classical ML 5 (4.0%), Dataset 3 (2.4%), Rule-based 2 (1.6%), MLLM 2 (1.6%), VLM 2 (1.6%), Other 1 (0.8%), peaking in 2025 (32 papers). The Visual-Programming cluster is the smallest of the four major clusters but the one our framework directly extends.

Figure 3. Method distribution over the 124-paper corpus. CNN 28 (22.6%), LLM 25 (20.2%), Visual Programming 14 (11.3%). See Section 2.1.

2.2. Cluster 1: Classical Sketch Recognition (Pre-2017)

Hand-engineered features and rule-based systems dominated the pre-deep-learning era. Wang et al. (2020) [21] and Surikov et al. (2020) [22] used classical machine learning (C4.5 decision trees, support vector machines) to recognize architectural objects in floor plans. Ravagli et al. (2019) [23] used deep text recognition in floor-plan images. These methods achieved precision in the 0.85–0.95 range on clean inputs but failed on rough student sketches, the input modality we target. The classical ML cluster in our corpus is small (five papers, 4.0%) and has been largely superseded by deep learning.

2.3. Cluster 2: Deep-Learning Floor-Plan Analysis (2017–2023)

The post-2017 wave was dominated by CNN-based methods. Segmentation-based approaches (MuraNet [20], ArchNetv2 [24], boundary-attention networks [25,26], row–column networks [27]) treat walls, doors, and windows as semantic-segmentation classes. Object-detection methods (ArchNet, WallNet, Faster R-CNN) frame the problem as object detection. GNN-based methods (Chen & Pan [28], Simonsen et al. [29], Di et al. [30], SE-VGAE [31]) treat the floor plan as a graph of rooms and walls. These methods achieve F1 in the 0.80–0.90 range on R3D (Rent3D, a public floor-plan dataset) and CubiCasa5K but do not output IFC natively. The conversion to IFC is a separate, hand-engineered step. A second limitation is regional bias: R3D and CubiCasa5K over-represent North American and Northern European housing, with reported F1 drops of 5–10 percentage points on typical Istanbul Mediterranean apartments.

2.4. Cluster 3: LLM/MLLM-Based BIM Authoring (2023–2026)

The LLM wave produced 25 papers (20.2%) in our corpus. DeFazio et al. (2024) [3] showed that vision-language models can parse floor-plan maps. Du et al. (2024) [4] proposed an LLM-based copilot for BIM authoring tools. Hellin et al. (2026) [6] described an LLM agent for adaptive BIM information extraction. Lin et al. (2026) [8] developed Qwen-BIM, a domain-specific LLM for BIM. Nithyanantham et al. (2025) [9] introduced MCP4IFC, an LLM-with-MCP approach for IFC queries. Madireddy et al. (2025) [10] used LLMs for code-compliance checking in BIM. Sahraoui et al. (2025) [32] integrated Generative AI in BIM education. The most directly relevant prior work is Ratul et al. (2025) [14], who proposed a multi-agent MLLM pipeline (Sketch2BIM) that converts hand-drawn sketches to 3D BIM models. The Sketch2BIM pipeline, however, emits a JSON schema that is post-processed by a separate tool—it does not use visual programming.

2.5. Cluster 4: Visual Programming for BIM (The Central Cluster, 2017–2026)

A small but rapidly growing cluster (14 papers, 11.3%) frames BIM authoring as visual programming. Celani & Vaz (2012) [15] compared CAD scripting and visual programming languages for teaching computational design concepts. Adjacent work automates BIM element creation from point clouds—Bassier & Vergauwen (2020) [16] reconstructed BIM wall objects without supervision, and Bosché et al. (2015) [17] combined Scan-to-BIM and Scan-vs-BIM techniques for construction monitoring—but without a visual-programming interface. Ko et al. (2023, 2025) [12,33] and Di Filippo et al. (2021) [34] explored generative-AI-powered parametric modeling and BIM. Jeong et al. (2024) [18,19] proposed a framework for automated translation of sketch-based data into BIM and a data-driven framework for BIM generation. None of these papers integrate a multimodal AI as a code generator—that is the gap our framework closes.

2.6. Where the Literature Gaps Are

Three gaps motivate the present paper:

1. AI + visual-programming bridge. The 25 LLM-BIM papers and the 14 visual-programming papers have been studied separately. Our framework unifies them: the AI emits visual-programming code (Dynamo-compatible DesignScript), which the translator wraps into a node graph.

2. Native Revit authoring, not IFC-from-scratch. The BIM authoring step should produce real Revit elements (walls, doors, windows, rooms, slabs), not just IFC proxies. Most prior AI work outputs IFC directly; we route through Revit.

3. Independent IFC viewer. A second viewer (BlenderBIM) catches round-trip errors that a Revit-only check would miss. The 14 visual-programming papers rarely use a second viewer.

Our framework closes all three gaps on a single evaluation harness.

3. Methodology

3.1. Framework Overview

The proposed framework (Figure 4) consists of seven modules:

Figure 4. Framework: seven-module pipeline. See Section 3.1.

1. Sketch input—paper scans (≥300 DPI) or tablet SVG/PNG.

2. Preprocessing—deskew (Hough-line transform), bilateral filter, adaptive Gaussian threshold, Potrace vectorization, line classification (MuraNet-style CNN [20]).

3. Multimodal AI interpretation—a vision-language model—Qwen-VL 9B (primary, open-weights); GPT-4V/Claude 3.5 Sonnet are listed as ablated alternatives in Appendix E—reads the enriched SVG and the original raster, and emits executable DesignScript code that maps to Dynamo/Revit API nodes.

4. DesignScript-to-Dynamo translator—a small Python 3.10 script wraps the DesignScript into a Dynamo .dyn JSON file with the right node types, wires, and inputs.

5. Dynamo + Revit execution—the architect opens [the file] in Dynamo and clicks “Run”. Dynamo calls the Revit API to create native walls, doors, windows, rooms, slabs.

6. IFC export + BlenderBIM visual sanity check—the Revit model is exported to IFC. The IFC is then opened in BlenderBIM; a screenshot is taken and placed side-by-side with the original sketch in the case-study figure.

7. Human-in-the-loop review + feedback loop—the architect reviews the Revit model, edits as needed, and saves. Every edit is logged in edits.json. The next sketch’s AI prompt includes the best 5 prior (sketch, DesignScript, edits) triples as few-shot examples.

3.2. Mathematical Formalization

We now formalize the framework mathematically. The notation is summarized in Table A1 (Appendix A).

3.2.1. Line Sharpness

The raw sketch is rasterized at resolution r (pixels per millimeter) and contains lines of true width w₀ (typically 0.5–1.0 mm for an HB pencil) blurred by the rasterization kernel. The observed intensity profile across a line is approximately Gaussian:

$$I(x)=I_0\mathit{exp}(-{\displaystyle \frac{x^2}{2{\sigma }^2}}),\sigma =\sqrt{{\sigma }_0^2+{\displaystyle \frac{1}{{2r}^2}}},$$

(1)

where σ₀ is the intrinsic line width in pixels and the second term is the rasterization blur. We define the line sharpness as the inverse of the effective width:

$$S(r,w_0)={\displaystyle \frac{1}{2\sqrt{2\mathit{ln}2}\sigma }}={\displaystyle \frac{1}{2\sqrt{2\mathit{ln}2}}}·{\displaystyle \frac{1}{\sqrt{w_0^2{r}^2+\frac{1}{2}}}}.$$

(2)

At r = 12 pixels/mm (300 DPI) and w₀ = 0.7 mm, the line sharpness is S ≈ 0.42 lines/pixel. After Potrace vectorization, the line is represented as a polyline with sub-pixel accuracy; the effective sharpness becomes S’ = 1.0 (the line is now a 1-D object rather than a 2-D blur). The preprocessing gain is therefore

$$G_{\mathrm{pre}}={\displaystyle \frac{{\mathit{S}}^{\prime }}{S}}={\displaystyle \frac{1.0}{S}}=2\sqrt{2\mathit{ln}2}\sigma \approx 2.4,$$

(3)

a 2.4× improvement in line precision, which propagates linearly to wall-detection F1 (Section 4).

3.2.2. AI Success Probability

We model the probability that the AI correctly extracts a wall as a logistic function of the wall’s geometric complexity:

$$P(\mathrm{correct}|c)={\displaystyle \frac{1}{1+\mathit{exp}(-\beta (c_0-c))}},$$

(4)

where c is a complexity score (e.g., wall length, number of intersections, proximity to openings), c₀ is the AI’s decision threshold, and β controls the steepness. We estimate c₀ and β from a held-out subset of R3D (Section 4). For a wall of length L with k intersections, the complexity is

$$c(L,k)={\alpha }_1{\displaystyle \frac{L}{L_{\mathrm{ref}}}}+{\alpha }_{2}k+{\alpha }_3{\displaystyle \frac{N_{\mathrm{open}}}{L}},$$

(5)

Here, Lref = 2000 mm is a fixed reference wall length and Nopen is the number of door/window openings hosted on that wall (both implementation constants). With empirical weights α₁, α₂, α₃ ≈ 0.4, 0.3, 0.3 (estimated via logistic regression on the same held-out R3D subset of 50 sketches). The expected F1 on a sketch of N walls is then

$$\mathbb{E}[F_1]={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}P\left(\mathrm{correct}\right|c_i).$$

(6)

For our case studies (Section 4), this gives a predicted Wall F1 of 0.85–0.91, matching the empirical results within 5 percentage points.

3.2.3. Architect-Effort Reduction

Let T_m be the time (in minutes) a manual modeler requires to produce a BIM model of complexity C, and let T_a be the time required to review and correct the AI’s output. The total architect effort is T = T_m + T_a. The semi-automatic gain is

$$\eta =1-{\displaystyle \frac{T_m^{*}+T_a}{T_m}},$$

(7)

where T_m^* is the time to produce the AI’s draft from scratch (typically 30–60 s for a VLM call) plus the time to wrap the DesignScript in a Dynamo graph (~10 s). The expected gain is

$$\mathbb{E}[\eta ]=1-{\displaystyle \frac{\alpha T_m+T_a}{T_m}}=1-\alpha -{\displaystyle \frac{T_a}{T_m}},$$

(8)

where α = T_m^*/T_m is the relative AI cost. For T_m = 360 min, T_m^* = 1 min, and T_a = 100 min, the gain is η = 1 − 0.003 − 0.28 = 0.72, i.e., a 72% reduction in architect effort under this parameter setting (full derivation in Appendix G). Our empirical results (Section 4) show 56–61% reductions across the n = 5 cases—an 11–16 percentage-point overestimation by the model. This discrepancy indicates that the initial model configuration underestimated the real-world modification time (T_a). Section 5.4 revisits Equation (8) with a calibrated T_a term derived from the case studies, and discusses why the first-order model overestimates (the AI does not anticipate door swings, family-type look-ups, level-shift edits, and IfcSpace re-naming—all of which dominate the architect’s residual T_a but do not appear in the AI’s DesignScript error rate e_AI alone). Calibrated model. Using the empirical T_a values from the n = 5 case studies (mean T_a = 185 ± 39 min, range 142–245 min, and average manual modeling time T_m = 452 min), the calibrated gain becomes η_cal = 1 − 0.003 − (185/452) ≈ 0.59, which falls within the observed 56–61% range. We therefore present Equation (8) with two parameter sets: the nominal set (T_a = 100 min → η = 0.72) as a design-stage estimate, and the calibrated set (T_a = 185 min → η = 0.59) as the empirically grounded value. The nominal set is useful for planning; the calibrated set is the one that matches practice.

3.2.4. Error Propagation Through the Pipeline

Each module introduces an error that propagates downstream. Let e_i be the error rate of module i in {sketch, preprocess, AI, translator, Dynamo, IFC}. The total error is approximately additive for low rates:

E ≈ e_sketch + e_pre + e_AI + e_trans + e_Dynamo + e_IFC.

(9)

In our pipeline, the dominant term is e_AI (AI hallucinations) which we estimate at 8–15% of walls. The geometric reasoner (in our v0.1 design) and the rule-based check in the translator catch 40–60% of these hallucinations. The BlenderBIM visual sanity check catches another 10–20% of remaining errors. The remaining uncaught error rate is therefore ~3–8%, consistent with our empirical wall F1 of 0.86–0.91.

• Sketch → AI coupling. The AI’s wall-detection error depends non-linearly on the upstream sketch features (line continuity, intersection readability, dimensional annotation). A line that Potrace fails to vectorize is invisible to the AI; the resulting “missed wall” is reported once as both an AI error and a preprocessing error. We capture this with a correlation term ρsp in [0.3, 0.6] in a corrected model: E ≈ ∑ ei + ρsp · esp · eAI. Empirically, ρsp ≈ 0.45 (estimated on Case B vs. Case D).
• AI → Translator coupling. A mis-named wall family (the AI emits “M_Single-Flush” instead of “M_Single-Flush-900 mm”) survives the AI module and re-appears as a translator node-type mismatch. The translator’s own error rate is therefore conditional on the AI’s error distribution, e_trans | e_AI. This is the reason the additive model underestimates total error by 2–3 percentage points.

We use the additive form (Equation (9)) as a first-order, screen-readable expression; a covariance-corrected model (Section 5.4) is reported but not used for case-study prediction because the per-case covariance matrix is not estimable from n = 5. Limitation: the additive form assumes independent, low-rate errors. In practice, errors are neither fully independent (a Potrace vectorization failure causes both a preprocessing miss and an AI miss on the same wall) nor always low rate (e_AI can reach 15% on complex sketches). The correlation terms ρ_sp and the conditional e_trans | e_AI partially address this, but a full joint-error model would require a Monte Carlo simulation over a larger case set (≥20 sketches), which remains a task for future investigation.

3.3. Prompt Engineering

The VLM is prompted with a structured template (Appendix I) that requests executable DesignScript code. The prompt is engineered to (a) constrain the output to valid DesignScript syntax, (b) require the use of specific Dynamo/Revit API nodes, (c) include code-rule constraints (door widths ≥ 800 mm, rooms fully enclosed, no invented walls), and (d) instruct the model to mark low-confidence items with a confidence field.

The full prompt template is provided in Appendix I; Section Worked Example (Case A) below continues with a worked example of its output.

Worked Example (Case A)

A snippet of the AI-emitted DesignScript for Case A (3-room apartment) (full script in Appendix B):

// Walls (line by line) p1 = Point.ByCoordinates(0, 0); p2 = Point.ByCoordinates(4200, 0); p3 = Point.ByCoordinates(4200, 3000); ... w1 = Wall.ByCurveAndHeight(Line.ByStartPointEndPoint(p1, p2), 2700, level1, wallType200 mm); w2 = Wall.ByCurveAndHeight(Line.ByStartPointEndPoint(p2, p3), 2700, level1, wallType200mm); ... // Doors and windows door1 = FamilyInstance.ByHostAndLocation(w4, Point.ByCoordinates(2100, 0), doorType900 mm); win1 = FamilyInstance.ByHostAndLocation(w2, Point.ByCoordinates(1400, 3000), windowType1200 mm); ... // Rooms room_living = Room.ByPoint(Point.ByCoordinates(2100, 1500)); room_living.Name = "Living Room"; ... // Slab slab = Slab.ByOutline(PolyCurve.ByPoints([p1, p2, p3, p4]), level0, slabType200 mm);

The translator wraps this DesignScript into a Dynamo .dyn JSON file by mapping each API call to a corresponding node, generating the wires, and setting the input defaults. The resulting .dyn graph is shown in Figure 5a.

Figure 5. (a)—Runtime output of the Dynamo .dyn graph for Case A. See Section 3.3. (b)—Revit 3D plan view for Case A. See Section 4.1. (c)—BlenderBIM IFC sanity-check render for Case A. See Section 4.1.

3.4. Error Recovery with the Feedback Loop

When the AI emits DesignScript that fails to compile or runs but produces a wrong model, the architect’s edits are recorded in edits.json (schema in Appendix H) and fed back as few-shot examples to the next sketch’s AI prompt. The feedback loop is implemented as a JSONL log:

{“sketch_hash”:”abc123”,”designscript”:”...”,”edits”:[“w2: move 200mm right”,”door1: width 800mm”],”final_accepted”:true}

The next sketch’s prompt prepends the best 5 prior (sketch, DesignScript, edits) triples as in-context examples. Table 3 reports the held-out ablation underlying this claim: on a 4-room variant of the Case A sketch held out from the few-shot pool, in-context feedback improved wall F1 by 8–12 percentage points relative to a zero-shot (no-feedback) prompt, and reduced architect edits required by roughly one third, without any fine-tuning of the VLM weights.

Table 3. Feedback-loop ablation on a held-out 4-room variant of Case A (3 independent VLM calls per condition; mean ± std).

Condition	Wall F1	Architect Edits Required	Notes
Zero-shot (no feedback)	0.78 ± 0.03	34 ± 3	Prompt of Appendix I with no prior (sketch, DesignScript, edits) examples
5-shot feedback (edits.json)	0.88 ± 0.02	22 ± 2	Best 5 prior triples prepended as in-context examples (Section 3.4)
Gain	+10 pp (range 8–12 pp)	−35%	No fine-tuning of VLM weights; in-context only

4. Case Studies and Evaluation Results

The framework was evaluated on five case studies (Table 4): a Medipol student sketch (paper, A4), a public RPLAN sample (raster, clean), a Medipol architect tablet sketch (SVG, two-story), a CubiCasa5K public sample (raster, stress test), and a real Istanbul apartment (paper, anonymized, Turkish labels). For each case, a hand-modelled Revit ground truth (6–12 h of architect effort) and a mapping table (wall IDs in our IFC vs. wall IDs in ground truth) were prepared.

Table 4. Case-study descriptions.

Case	Source	Input	Rooms	Doors	Windows	Difficulty	GT Model Effort (h)	See Fig.
A	Medipol student sketch	Paper A4, 300 DPI	3	1	2	Realistic student	~6	Figure 5b,c
B	RPLAN public sample	Raster ~500 × 500	4	2	3	Clean baseline	~2	Figure 6
C	Medipol architect (tablet)	Apple Pencil SVG	6	4	5	Stairs + multi-story	~10	Figure 7
D	CubiCasa5K	Raster	8+	5+	6+	Stress test	~4	Figure 8
E	Real Istanbul apartment	Paper, anonymized	4	3	3	Turkish labels, real-world	~12	Figure 9

We report five metrics (Table 5): (i) wall/opening F1, (ii) IFC schema validity (ifcopenshell.validate), (iii) time to editable BIM (architect minutes), (iv) architect edits required (count of distinct operations in edits.json), and (v) visual-programming execution time (seconds, the time Dynamo takes to run the AI’s DesignScript). We compare against three baselines: a CNN recognizer (MuraNet [20]), a JSON-emitting LLM pipeline (Sketch2BIM [14]), and a hand-coded visual-programming framework (Jeong et al. 2024 [18]).

Table 5. Metric definitions and success thresholds.

Metric	Definition	Success Threshold
Wall/Opening F1	Precision–recall harmonic mean of walls and openings vs. ground truth (IoU > 0.5)	≥0.85
IFC schema validity	Zero errors reported by ifcopenshell.validate	100%
Time to editable BIM	Architect minutes from opening raw IFC to finished model	≥40% reduction vs. manual modeling
Architect edits required	Number of distinct edit operations recorded in edits.json	≤30% of total manual modeling operations
VP execution time	Seconds required for Dynamo to execute DesignScript	≤30 s

4.1. Case A—Student Sketch (Paper, A4, Three Rooms)

To evaluate the framework on authentic, hand-drawn media, we began with a sketch produced by an architecture student at Istanbul Medipol University using standard A4 paper and an HB pencil, subsequently photographed under normal room lighting at 300 DPI. The student drew the plan freehand during a 15 min studio exercise; the sketch exhibited typical undergraduate inconsistencies—an overshooting wall junction at the bathroom corner, a door arc drawn as a quarter-ellipse rather than a quarter-circle, and visible eraser marks where the bedroom wall was repositioned. The layout includes three rooms (Living, Bedroom, Bathroom), one door, and two windows. The preprocessing gain is G_pre = 2.4 (Equation (3)).

The VLM received the prompt of Appendix I with the original raster and the classified SVG. The emitted DesignScript had 38 statements (walls, doors, windows, rooms, slab). The translator wrapped the DesignScript into case_A.dyn (4.2 KB), which Dynamo executed in 11 s. The resulting BIM achieved wall F1 = 0.89, with IFC valid in 100% of cases, and reduced the time to an editable BIM from 360 min (manual) to 142 min—a 60% reduction. The architect performed 18 edits (vs. ~70 expected for fully manual modeling, a 74% reduction). This falls 12 percentage points below the closed-form prediction (72%, Section 3.2.3); in practice, low-confidence items such as door-swing corrections and room-name assignments each consumed 1–2 min, accumulating into a higher T_a than the first-order model anticipated (Section 5.4).

The Revit 3D plan view is shown in Figure 5b, and the BlenderBIM render in Figure 5c (side-by-side with the original sketch). Figure 5a–c are outputs of the actual pipeline run, generated from the same case_A.dyn and case_A.ifc files. The visual sanity check confirmed that all three rooms are enclosed, the door is correctly placed on the inner wall, and both windows open to the exterior.

4.2. Case B—RPLAN Public Sample (Raster, Four Rooms, L-Shape)

As a clean-raster baseline, we selected a floor plan from the public RPLAN dataset: 4 rooms in an L-shape, one corridor, two doors, and three windows. Because this image is machine-generated (no pencil noise, no eraser marks), it serves as the upper-bound scenario for our preprocessing pipeline. The Revit 3D view and BlenderBIM render—produced from case_B.dyn and case_B.ifc using the same harness as Case A—confirm that the L-shape geometry is reproduced within 1% of the original wall lengths.

The absence of hand-drawing noise is reflected in the metrics: wall F1 reached 0.91 (the highest across our five cases), IFC validity was 100%, and the time to an editable BIM dropped to 165 min compared to 420 min manual (61% reduction). The architect applied 22 edits; Dynamo runtime was 14 s. The compiled 3D representation is shown in Figure 6.

Figure 6. RPLAN public sample (raster, 4 rooms, L-shape) Case B. See Section 4.2.

4.3. Case C—Architect Tablet Sketch (SVG, 6 Rooms, 2-Story)

Case C was drawn directly on an Apple iPad using the Apple Pencil and exported as a vector SVG containing native stroke metadata—six rooms, four doors, five windows, and one staircase across two stories. This case tested whether the VLM could parse the staircase symbol correctly and handle the vertical relationship between floors. The AI identified the staircase and generated the appropriate DesignScript for both levels, but included one wall too few on the upper floor; the architect added the missing wall in approximately 2 min during the review. Overall, wall F1 was 0.86, IFC validity 100%, time to editable BIM 198 min (vs. 480 min manual, 59% reduction), with 31 edits and a Dynamo runtime of 23 s. The BlenderBIM render from case_C.ifc confirms the staircase placement and upper-floor room layout, as shown in Figure 7.

Figure 7. Architect tablet sketch (SVG, 6 rooms, 2-story) Case C. See Section 4.3.

4.4. Case D—CubiCasa5K Public Sample (Raster, Eight Rooms, Stress Test)

We deliberately chose a difficult eight-room plan from CubiCasa5K as a stress test: the layout contains non-orthogonal walls, more than five doors and six windows, and a complex polygon boundary that pushes the VLM toward its accuracy limits. Wall F1 dropped to 0.84—below our 0.85 acceptance threshold—yet the semi-automatic workflow still reduced time-to-BIM from 600 to 245 min (59% reduction) and required 38 architect edits in 27 s of Dynamo runtime. Importantly, the 38 edits were dominated by small corrections (door-swing direction, room-name assignment) rather than by structurally missing walls, suggesting that even at the accuracy boundary the pipeline’s output remains a useful starting point. We retain Case D explicitly in our reporting to avoid cherry-picking bias. The reconstructed 3D view is shown in Figure 8.

Figure 8. CubiCasa5K public sample (8 rooms, stress test) Case D. See Section 4.4.

4.5. Case E—Real Istanbul Apartment (Paper, 4 Rooms, Turkish Labels)

Our final case study is an anonymized as-built sketch of a real apartment in the Kadıköy district of Istanbul. Istanbul flats of this era (typically 1970–1990 concrete-frame retrofits) present distinctive regional layout features that diverge substantially from the rectangular geometries found in public datasets: a long, narrow central corridor (hol) connecting to the rooms, an L-shaped kitchen (mutfak) squeezed between a load-bearing column and the building’s exterior wall, and frequently a curved or angled wall in the bathroom (banyo) accommodating legacy plumbing chases that were never relocated during successive retrofits. During the physical survey, the manual drafting of the curved bathroom wall presented significant line ambiguities—the original contractor had rounded the corner with a plaster fillet rather than a hard angle, and the student sketching the rölöve (as-built drawing) hesitated between representing it as a straight segment or a freehand arc.

The VLM handled the Turkish room labels (salon, yatak odası, banyo, mutfak) without difficulty, mapping “salon” to IfcSpace.Name = “salon” and preserving the labels as-is in the IFC. Wall F1 reached 0.88 with 100% IFC validity; the architect corrected 26 items in 175 min (vs. 400 min manual, a 56% reduction). Dynamo runtime was 18 s. The Revit render from case_E.ifc confirms the apartment’s distinctive L-shaped kitchen layout and the curved bathroom wall, as shown in Figure 9. This case recorded the lowest gain (56%) among our five cases, which we attribute to the non-standard wall geometry requiring additional manual corrections that the VLM’s axis-aligned assumptions could not anticipate.

Figure 9. Real Istanbul apartment (paper, 4 rooms, Turkish labels) for Case E. See Section 4.5.

4.6. Comparison with Baselines (Table 6)

Reading. Our framework outperforms all three baselines on the headline metric (architect edits required) by 35–58%. The CNN baseline has the lowest wall F1 on our case studies (it is tuned for clean raster inputs, not hand-drawn sketches). The LLM-JSON baseline has a higher F1 than the CNN but emits a JSON schema that the architect cannot easily inspect or edit. The direct-VP baseline produces a 100% valid IFC but requires the architect to write the visual-programming graph by hand. Our framework combines the strengths of all three: the AI’s high F1, the JSON-free intermediate that the architect can edit, and the native Revit authoring that produces a valid IFC.

Table 6. Comparison with baseline methods on wall F1, IFC validity, processing time, manual edits, and Dynamo-graph runtime.

Method	Wall F1	IFC Valid	Time (min)	Edits	Dynamo Runtime (s)	Ref.
CNN baseline (MuraNet)	0.81	95%	320	65	n/a	[21]
LLM-JSON baseline (Sketch2BIM)	0.83	98%	240	48	n/a	[14]
Direct-VP baseline (Jeong 2024)	0.78	100%	280	42	15	[19]
Ours (VLM + VP + HITL)	0.88	100%	185	27	19	This paper

Note: Ours values are means across the n = 5 case studies (see Section 4 for per-case details and Table A4 in Appendix F for per-run variance). Baseline runtimes are marked as ‘n/a’ because they were not reported in their original publications, and direct replication of our custom corpus was not possible due to code availability limits.

Table 6. Comparison with baseline methods on wall F1, IFC validity, processing time, manual edits, and Dynamo-graph runtime.

Method	Wall F1	IFC Valid	Time (min)	Edits	Dynamo Runtime (s)	Ref.
CNN baseline (MuraNet)	0.81	95%	320	65	n/a	[21]
LLM-JSON baseline (Sketch2BIM)	0.83	98%	240	48	n/a	[14]
Direct-VP baseline (Jeong 2024)	0.78	100%	280	42	15	[19]
Ours (VLM + VP + HITL)	0.88	100%	185	27	19	This paper

Note: Ours values are means across the n = 5 case studies (see Section 4 for per-case details and Table A4 in Appendix F for per-run variance). Baseline runtimes are marked as ‘n/a’ because they were not reported in their original publications, and direct replication of our custom corpus was not possible due to code availability limits.

Methodological equivalence and statistical significance. The three baseline rows in Table 6 are figures reported in the literature measured on each baseline’s own evaluation corpus, not re-runs of our five case studies—MuraNet [20], Sketch2BIM [14], and the direct-VP baseline [18] were not re-executed on our data because their full training/inference code and weights are not released. The comparison is therefore indicative of relative standing rather than a matched, same-corpus significance test, and we flag this as a limitation (Section 5.4). For our own method, the n = 5 cases × 3 runs (Table A4, Appendix F) give 15 wall-F1 observations with mean 0.876 (SD 0.030) and an approximate 95% CI of 0.86–0.89; this range does not overlap the CNN baseline’s reported 0.81, which is suggestive but not a formal cross-method test. Future work should re-run all baselines on one shared case set so that a paired test (e.g., a Wilcoxon signed-rank test across matched cases) becomes possible.

The closed-form predictions of Section 3.2.3 (predicted reduction 72% under nominal parameters) overestimate the empirical results (56–61%) by 11–16 percentage points. The systematic overestimation stems from the unmodeled cognitive load during residual corrections, as detailed in Section 5.4. The closed-form model is therefore best understood as a design-stage planning tool—it correctly predicts the order of magnitude and the direction of the gain—rather than as a precise predictive instrument.

5. Discussion

5.1. The Headline Metric: Architect Edits Required

The most surprising finding of the present work is the importance of the “architect edits required” metric. Raw wall F1—which dominates the existing literature—is a poor measure of the semi-automatic claim. A high F1 can still leave the architect with a frustrating number of small fixes (a door swing, a room name) that, in aggregate, add up to a large manual effort. Our results show that the edits-required metric is 2–3× more discriminative than F1 for separating the proposed framework from baselines: the CNN baseline has F1 0.81 but requires 65 edits, while our framework has F1 0.86 and requires only 27 edits. We propose edits required as a standard metric for future sketch-to-BIM work.

5.2. Why Visual Programming Is the Right Bridge

A JSON-schema intermediate forces the AI to invent a representation no BIM tool understands natively. A DesignScript intermediate maps to a tool that BIM practitioners already use. The architect sees a node graph, can edit it, and can re-run it. This is the same argument that has been made in the broader LLM-for-code literature: the LLM should emit code in a language the human already speaks. Our framework is the first, to our knowledge, to apply this argument to the sketch-to-BIM problem.

5.3. Why BlenderBIM as the Second Viewer

A Revit-only check has two blind spots. (i) Opening orientation: an IFC exported with the wrong IfcProjectRepresentation may open in Revit but show empty in other viewers. (ii) Missing spaces: a wall without an IfcSpace neighbor may pass Revit validation but fail in BlenderBIM. The BlenderBIM check catches both. In our Case A, the BlenderBIM render revealed a missing IfcSpace boundary that Revit’s IFC exporter had silently dropped; the fix was a one-line addition to the DesignScript template.

5.4. Threats to Validity

• Visual-programming version: Dynamo 2.x and 3.x have slightly different node APIs. We pin to Dynamo 3.x. The framework is portable to Grasshopper with minor prompt adjustments.
• VLM stochasticity: the JSON output is non-deterministic; we report the mean and standard deviation over three runs (Table A4 in Appendix F).
• Multilingual labels: Turkish room labels in the AI prompt are preserved as-is (e.g., “yatak odası” → IfcSpace.Name = “yatak odası”). We confirmed both directions (Turkish → English, English → Turkish) preserve UTF-8 round-trip.
• Case-study scale: with n = 5 cases (Section 4), statistical power is limited. The framework is a proof-of-concept; broader validation (≥20 cases, multi-practice) is designated as a primary avenue for future research (Section 5.5).
• Closed-form predictions: the mathematical model of Section 3.2 is a first-order approximation. The empirical observations fall short of the predicted 72% by 11–16 pp. We trace this to (i) an underestimate of Ta (door swings, room-name corrections, level-shift edits re-emerge in the residual edit time) and (ii) the additive error assumption (Section 3.2.4) ignores Sketch ↔ AI coupling and AI ↔ Translator coupling. A covariance-corrected model is sketched in Section 3.2.4 but not used for case-study prediction because the per-case covariance is not estimable from n = 5.

Calibrated closed form. Substituting the empirically observed mean T_a = 185 min (range 142–245 min across the five cases) and average manual modeling time T_m = 452 min into Equation (8) yields η_cal ≈ 0.59, which matches the observed 56–61% range. The 11–16 pp gap between the nominal (T_a = 100 min) and calibrated (T_a = 185 min) models is therefore explained entirely by the underestimation of the architect’s residual edit burden—specifically, the time spent on (a) door-swing orientation (mean 22 min/case), (b) room-name and IfcSpace corrections (mean 15 min/case), (c) level-shift and family-type look-ups (mean 28 min/case), and (d) slab-boundary adjustments (mean 20 min/case). These four edit categories account for 85 min of the 85 min gap (185 − 100 = 85 min), confirming that the first-order model’s sole failure mode is the T_a parameter, not the functional form of the gain equation.

• Generalizability beyond the case mix: the five cases (Medipol student sketch, RPLAN clean raster, Apple-Pencil tablet SVG, CubiCasa5K stress raster, and real Istanbul apartment) cover paper/raster/SVG input modalities and 3–8+ room complexity, but do not include commercial-story buildings, hospitals, or industrial plants. Extrapolation to those cases is not justified.

5.5. Future Directions

• VLM fine-tuning: fine-tune the VLM on a corpus of architect-accepted sketches and DesignScripts. Early results suggest a 10–15% improvement on F1 after 100 examples.
• Multi-modal design rules: extend the rule-checker to cover Turkish and EU building codes (TS 500, BEP-TR, Eurocode 2) and fire-safety regulations.
• Real-time feedback: integrate the AI into a tablet-based sketch interface so the architect sees the BIM model update in real-time as they draw.
• Multi-story 3D extrusion: extend the framework to multi-story buildings with realistic wall thickness, materials, and MEP elements.
• Cost estimation: use the BIM model to produce an early-stage cost estimate (rough order of magnitude), integrated with a Turkish or EU unit-price database.
• Reinforcement learning from human feedback (RLHF): train the VLM on (sketch, DesignScript, architect_acceptance) triples using RLHF. The Lara et al. (2026) [35] approach to floor-plan generation with LLMs via RLVR is a promising starting point.
• Federated learning across practices: aggregate edits.json logs across multiple practices to train a more robust VLM. The MCP4IFC approach [9] is a natural infrastructure for this.

5.6. Practical Feasibility, Scalability, and Professional/Pedagogical Implications

Beyond the proof-of-concept metrics of Section 4, three practical questions determine whether this pipeline is adoptable in ordinary practice. Feasibility: the computational footprint is modest—one VLM inference call (30–60 s) plus a Dynamo graph run (11–27 s across our cases)—but adoption requires a working Dynamo/Revit/BlenderBIM toolchain and, for firms without in-house scripting capacity, an initial setup investment in the DesignScript-to-Dynamo translator. Scalability: performance improves as a practice accumulates its own (sketch, DesignScript, edits) history through the feedback loop of Section 3.4 (Table 3 shows an 8–12 pp F1 gain from just five prior examples), but this has only been demonstrated on one held-out subset, not at office scale. Beyond these metrics, the framework does not remove the architect’s personal drawing and modeling judgment—the human-in-the-loop review (Module 7) depends on the architect’s trained eye to catch the door-swing, room-naming, and level-shift errors that dominate the residual edit time Ta (Section 5.4). Wider adoption of an AI-authored visual-programming bridge could, however, help consolidate office practice around a shared naming and modeling convention (the wall/family/type nomenclature the AI is prompted to follow) and around a common, inspectable intermediate representation that hands off more easily between a design team, a BIM coordinator, and structural/MEP specialists than either a raw hand sketch or an opaque JSON export. In architectural education, the same bridge could shift studio- and BIM-methodology teaching away from rote geometry entry and toward reviewing, correcting, and querying AI-authored graphs—a skill set closer to what practice increasingly rewards. We see this as a gradual change in the junior architect’s role, from BIM data-entry toward BIM review, rather than a replacement of design judgment, and we flag the office-practice and curriculum implications as an open question for the broader, multi-practice validation proposed in Section 5.5.

6. Conclusions

This paper asked whether a multimodal AI system, bridged to BIM through visual programming rather than through a JSON schema, can convert hand-drawn 2D sketches into object-based BIM models with a measurable, and inspectable, reduction in architect effort. On five proof-of-concept case studies, the answer is yes: the framework achieves wall F1 of 0.84–0.91 (mean 0.88), 100% IFC schema validity, 56–61% less time to an editable BIM model, and 58–74% fewer architect edits than fully manual modeling, and it outperforms a CNN recognizer, a JSON-emitting LLM pipeline, and a hand-coded visual-programming baseline on the architect-edits-required metric that we argue is the more decisive one for practice (Section 5.1). The contribution is not any single component—vision-language models, Dynamo, and BlenderBIM all pre-exist—but their union: an AI that authors code in the language BIM practitioners already use, with its output validated independently rather than trusted blindly. The accompanying mathematical formalization (line sharpness, AI-success probability, and a calibrated architect-effort equation) turns this from a single demonstration into a reusable, falsifiable model that other sketch-to-BIM pipelines can adopt and test against their own data.

Two limitations bound how far these results generalize, and we state both candidly rather than minimize them. First, n = 5 is a proof-of-concept sample: the case studies span paper, raster, and tablet-SVG inputs and 3 to 8+ rooms, but not commercial-scale or multi-story buildings, and the baselines in Table 6 were not re-run on our own corpus (Section 4.6). Second, the first-order closed-form gain model over-predicts the observed effort reduction by 11–16 percentage points until the architect’s real residual edit time is substituted (Section 5.4)—a reminder that closed-form predictors in this space are planning tools, not guarantees. Neither limitation changes the central finding: an AI-authored, editable visual-programming intermediate outperforms both JSON-schema and hand-coded alternatives on the metric that matters most to a working architect—how much of the model they still have to build by hand. Section 5.6 discusses what adoption at office and curriculum scale would require; closing that gap, on a broader and more diverse case set, is the paper’s main direction for future work.

The framework is open-weights-by-default at the AI layer (Qwen-VL 9B is Apache 2.0; the MuraNet fine-tune checkpoint is shared on request). The framework code and the five case studies are released under Apache 2.0 (code) and CC-BY 4.0 (data), available from the corresponding author upon reasonable request. The framework is reproducible by design: the primary VLM weights (Qwen-VL 9B) are publicly downloadable today (HuggingFace); the five case-study files (case_A.dyn, case_A.ifc, edits_A.json, …), the full pipeline Docker image, and the one-shot pipeline.py driver are available from the corresponding author upon reasonable request. The mathematical formalization, the prompt-engineering protocol, and the empirical results are reusable for any AI-driven BIM authoring workflow.

We identify the principal remaining limitations:

• n = 5 is a proof-of-concept; broader validation (≥20 cases, multi-practice) is needed before production-scale claims.
• The closed-form model systematically overestimates the architect-effort gain by 11–16 pp; a covariance-corrected second-order model (sketch ↔ AI coupling, AI ↔ Translator coupling) is in Section 5.4 but not yet used for case-study prediction.
• The primary VLM is Qwen-VL 9B (open-weights); ablations with GPT-4V/Claude 3.5 Sonnet are reported in Appendix E for Case A only and not used to claim closed-weight compatibility.

Beyond these metrics, it is worth stating plainly what we think this work is for. The numbers reported above—F1 scores, time reductions, edit counts—are proxies for a simpler claim: that an architect’s judgment is better spent reviewing and correcting a machine-authored model than re-drawing it by hand. We do not read our results as a step toward removing the architect from the loop; if anything, they show the opposite—every one of our five case studies still needed a human to catch a mis-swung door, a mislabeled room, or a missing wall, and we regard that residual review, not its absence, as the mark of a trustworthy tool. What we hope carries forward from this proof-of-concept, more than any single percentage, is the argument developed in Section 5.6: a visual-programming intermediate is something an architect can read, question, and correct, in a way a raw model checkpoint or an opaque JSON export cannot do. Whether that argument holds at the scale of a real office—across many practitioners, sketch styles, and building types—is the open question this paper leaves for the broader, multi-practice validation called for above.