OpenAsset: A Pipeline for Open-World Asset Integration into Indoor Scene Synthesis

Abstract

Text-driven 3D indoor scene synthesis has witnessed significant progress through Large Language Model (LLM)-based frameworks like ReSpace. However, current paradigms heavily rely on retrieving objects from pre-defined, static 3D asset libraries, which fundamentally constrains the diversity and personalization of generated scenes due to the closed-set nature of existing databases. Conversely, recent breakthroughs in promptable segmentation (SAM 3) and single-image 3D reconstruction (SAM 3D) have empowered the extraction of high-fidelity 3D geometry and texture from in-the-wild images. In this paper, we bridge the gap between text-driven scene layout generation and single-view object reconstruction by proposing OpenAsset. This novel framework converts single images of real-world objects into reusable, canonicalized 3D assets that seamlessly integrate into the scene synthesis workflow. Specifically, given a user concept prompt or target region, OpenAsset leverages SAM3 for precise instance isolation and SAM3D for geometry and texture recovery. To ensure compatibility with structured scene representations (SSR), we introduce an automated canonicalization module that normalizes the scale, orientation, and coordinate systems of the reconstructed meshes. By transforming “wild” visual percepts into standardized assets, OpenAsset effectively expands the controllable vocabulary of indoor scene synthesis beyond curated datasets, offering a practical pathway toward user-sourced open 3D scene generation that supports custom objects outside fixed predefined asset libraries. It is worth noting that the satisfactory performance of OpenAsset critically depends on effective segmentation and reconstruction results, which serve as essential prerequisites for our method.

1. Introduction

The democratization of 3D content creation is a long-standing goal in computer graphics and vision. Constructing high-quality indoor scenes traditionally requires extensive manual effort and expertise with professional modeling tools, posing a significant barrier for non-expert users. This challenge is particularly restrictive for applications such as interior design, embodied AI simulation, virtual and augmented reality, digital twins, and game development, where users typically care more about semantic intent than low-level geometric modeling. Natural language therefore provides an attractive interface: instead of manually specifying meshes, coordinates, and transformations, users can describe desired environments in terms of functionality, style, and object relationships. Recent advances in Large Language Models (LLMs) have accelerated this vision, enabling text-driven 3D scene synthesis systems that translate free-form descriptions into plausible spatial layouts. Models like LayoutGPT, Holodeck, InstructScene and ReSpace demonstrate LLMs can interpret object semantics, spatial relations and scene composition.

A common strategy in these approaches is to decompose scene generation into planning and realization. Rather than directly producing meshes, the system first predicts an intermediate structural representation of the scene—including room boundaries, object categories, sizes, positions, and relations—before instantiating concrete 3D assets. This design improves controllability and editability, making it suitable for interactive scene creation. ReSpace, for example, formulates indoor scene synthesis as autoregressive generation over a structured scene representation (SSR), enabling both scene generation and natural-language-driven editing such as adding, removing, or replacing objects.

Despite these advances, a key bottleneck remains: the availability of suitable 3D assets. Most pipelines rely on retrieval-based instantiation, where predicted objects are matched with assets from fixed repositories such as 3D-FUTURE or Objaverse. Although large, these repositories are inherently finite, creating a mismatch between the open-ended nature of language and the closed set of available assets. As a result, systems often retrieve category-correct but instance-inaccurate objects when users request personalized items, rare categories, or fine-grained stylistic attributes.

This limitation affects the practical usefulness of text-driven scene synthesis. In many indoor scenes, object identity matters at the instance level—for example, a user’s own lamp or coffee machine may have distinctive geometry or appearance. Retrieval-based proxies often fail to preserve these visual identities, preventing current systems from supporting truly personalized scene creation.

Recent advances in vision offer a promising alternative. In 2D perception, promptable segmentation has evolved toward open-vocabulary grounding. SAM 3 enables precise localization of arbitrary objects using concept prompts. Meanwhile, single-image 3D reconstruction methods such as SAM 3D can recover object geometry and texture from a single photograph. Together, these techniques make it possible to generate user-specific 3D assets directly from reference images instead of relying on database retrieval.

However, integrating such reconstructed assets into LLM-based scene synthesis is non-trivial. Reconstructions from real-world photos typically lack canonical coordinate frames, consistent scales, or well-defined orientations, and may contain noise or background artifacts. In contrast, structured scene representations demand assets with unified dimensions, poses and spatial boundaries, which guarantees stable object placement and editable operations in scenes.

In this paper, we present OpenAsset, a unified pipeline that harmonizes single-image asset reconstruction with text-driven indoor scene synthesis. OpenAsset can be viewed as an asset-agnostic extension to the ReSpace framework. Given a user-provided reference image and a concept prompt, we first employ SAM 3 to isolate the target object from cluttered backgrounds, and then use SAM 3D to lift the segmented observation into a textured 3D representation. Crucially, because raw reconstructions are not immediately compatible with structured scene synthesis, we introduce a dedicated canonicalization module that estimates a consistent object-centric frame, normalizes scale and orientation, and aligns the resulting asset with the conventions required by the target structured scene representation. Through this design, objects captured from ordinary photographs can participate in the same placement, rotation, and editing operations as native database assets, effectively integrating personalized real-world content into an LLM-guided 3D scene.

The resulting framework enables a new form of personalized, open-vocabulary scene generation. A user can describe a room in natural language while also supplying reference photos of specific objects they wish to preserve in the final scene. OpenAsset then grounds these user-specific assets within a coherent 3D environment, combining the structural reasoning ability of LLM-based layout planners with the visual specificity afforded by image-grounded reconstruction. We validate our approach through extensive qualitative and quantitative experiments, showing that OpenAsset preserves the layout coherence and editability of ReSpace while substantially improving scene realism, instance-level faithfulness, and personalization compared with retrieval-only baselines. More broadly, our work takes a step toward lifting text-driven 3D scene synthesis from a closed asset-selection problem to a more practical open-world content creation paradigm. It should be emphasized that the promising performance of our OpenAsset method relies on accurate segmentation and high-quality single-image reconstruction as essential prerequisites.

In summary, our contributions are as follows:

• We introduce OpenAsset, an end-to-end framework that integrates concept-prompted segmentation and single-image 3D reconstruction into text-driven indoor scene synthesis, thereby relaxing the closed-set assumption imposed by fixed 3D asset libraries.
• We propose an automated asset canonicalization strategy that converts raw reconstructions from in-the-wild images into scene-ready assets compatible with structured scene representations, enabling seamless interoperability with LLM-based scene planners.
• We demonstrate personalized, open-vocabulary scene generation and editing from user-provided reference photos, offering a practical path toward more faithful, realistic, and user-centric 3D indoor content creation.

2. Related Work

2.1. 3D Indoor Scene Synthesis

Early approaches to indoor scene synthesis predominantly relied on heuristic rules or procedural modeling [1,2,3,4,5,6]. With the advent of deep learning, research has progressively shifted towards learning-based methodologies [7,8,9,10,11,12], extensively leveraging Transformers and diffusion models. Specifically, Deep Priors [13] introduced CNN-based attribute prediction, while Fast&Flexible [7] developed a conditional cascaded CNN pipeline. SceneFormer [8] and ATISS [9] modeled scenes via autoregressive Transformers and unordered object sets, respectively, whereas FOREST2SEQ [14] optimized object ordering strategies. Diffusion-based approaches, such as DiffuScene [10], Mi-Diff [11], PhyScene [15], and LEGO-Net [12], have significantly enhanced generation quality and controllability. Methods like Text2Room [16], DreamScene [17], and Set-the-Scene [18] improved texture and layout generation through unified scene representations; however, their handling of small objects remains limited. Human-centric and scene graph-based methods, such as MIME [19], InstructScene [20], and EchoScene [21], support functional constraints by modeling object relationships. Nevertheless, most existing approaches lack capabilities for fine-grained editing or handling open layouts. Recently, text-based methods, including LLPlace [22], CasaGPT [23], and ReSpace [24], leverage Large Language Models (LLMs) for end-to-end prediction of object semantics and layouts, while related studies explore LLM-driven agents and synthetic data generation [25,26,27]. These approaches unify the generation of furniture of all sizes, support interactive text editing and break the constraints of fixed asset libraries, creating new opportunities for open-world indoor scene synthesis. Related research has also investigated the construction of geometrically coherent and physically meaningful synthetic environments. The two-view reconstruction method proposed in [28] reconstructs spatially aligned 3D structural models by combining generative models with epipolar geometric constraints from paired images. The PBGM (physics-based graphics model) framework presented in [29] builds physically accurate 3D synthetic simulation scenes to serve as test platforms for autonomous visual detection of civil engineering structures. These studies demonstrate the importance of geometric consistency and physically meaningful 3D content. In contrast, OpenAsset focuses on converting an object depicted in a single user-provided image into a canonicalized asset that can be inserted into an existing structured indoor scene synthesis pipeline.

2.2. Image Segmentation

Early image segmentation relied on hand-crafted features and heuristics, often struggling with complex semantics. The advent of deep learning shifted focus to CNNs and, subsequently, foundation models enabling zero-shot generalization. While early interactive methods were limited by closed-set vocabularies, the Segment Anything Model (SAM) family established a new promptable paradigm. SAM 2 [30] extended this to video, enabling efficient temporal tracking via point and box prompts, while SEEM [31] enhanced static image segmentation through flexible multi-modal prompting (points, boxes, and text). Recent advancements have evolved these frameworks into open-vocabulary, concept-driven systems. GeoSceneGraph [32] builds a geometry-aware scene graph diffusion pipeline for text-guided indoor scene synthesis, explicitly modeling object geometries and pairwise spatial relations to generate physically plausible indoor layouts. Pushing further, SAM 3 [33] introduces Promptable Concept Segmentation (PCS), allowing the detection, segmentation, and tracking of all instances matching natural language phrases or visual exemplars, thereby overcoming single-instance limitations. Concurrently, BREPS [34] establishes rigorous benchmarks for bounding-box robustness, addressing critical reliability gaps. Weakly supervised semantic segmentation has further advanced via augmentation and prompt clustering paradigms [26,35]. Collectively, these works advance promptable segmentation from geometric interaction to semantic-driven, open-vocabulary extraction, providing a robust foundation for precise object isolation and multi-instance extraction in complex scenes.

2.3. 3D Reconstruction

Traditional methodologies for recovering 3D geometry from single images were largely constrained by handcrafted priors [36] established direct 3D regression, GECO [37] proposes a two-stage image-to-3D generation pipeline, and Jiang et al. [38] enhanced geometric completeness using consistency priors and Schrödinger bridges. Concurrently, semantic integration has advanced via native 3D grounding in vision-language models [39], complemented by emerging open-vocabulary efforts [40]. Drawing on multimodal fusion techniques, we address the challenge of inconsistent scales and orientations by introducing standardized normalization. Most existing methods prioritize individual object fidelity over scene-level compatibility, lacking the standardized normalization required for structured scene representations. Our work addresses this gap by introducing OpenAsset, a framework that canonicalizes wild images into reusable assets, enabling seamless integration into text-driven scene synthesis workflows beyond the limits of static libraries.

3. Framework

Given a scene-level text prompt $P_{\mathrm{scene}}$, a room-floor polygon $\mathcal{R}\subset {\mathbb{R}}^2$, and an optional set of K user references

$$\mathcal{X}={\left\{(I_k,P_k,{\ell }_k)\right\}}_{k=1}^K,$$

(1)

where $I_k$ is a reference image, $P_k$ is a concept prompt describing the target object, and ${\ell }_k$ is its object category, our goal is to synthesize a 3D indoor scene

$$\mathcal{S}={\left\{({\ell }_i,a_i,{\mathbf{p}}_i^w,{\theta }_i)\right\}}_{i=1}^N,$$

(2)

where ${\ell }_i$ denotes the category of instance i, $a_i$ is the instantiated 3D asset, ${\mathbf{p}}_i^w={[x_i,y_i,z_i]}^{\top }\in {\mathbb{R}}^3$ is its translation in the world frame, and ${\theta }_i\in (-\pi ,\pi ]$ is the yaw angle around the global up-axis. Each asset may come either from a pre-built 3D database or from a user-provided in-the-wild image. For clarity, we use superscripts c, o, and w to denote the camera frame, the canonical object frame, and the world frame, respectively.

Our framework extends text-driven scene synthesis with an asset-side pipeline that converts open-vocabulary image references into scene-ready 3D assets. The pipeline consists of three stages: (1) open-vocabulary 3D asset extraction, (2) automated asset canonicalization, and (3) layout reasoning within Structured Scene Representation (SSR). The first stage converts a language-specified object in an image into a raw 3D reconstruction in the camera frame. The second stage removes view-dependent ambiguity by transforming the raw reconstruction into a canonical, metric, and grounded asset representation in the object frame. The third stage inserts the canonicalized asset into the same SSR interface used by database assets, enabling unified downstream reasoning and placement in the world frame. The overview of the proposed framework is shown in Figure 1.

Figure 1. Overview of the proposed framework. The pipeline consists of three main stages. (1) Open-vocabulary 3D asset extraction: Given a reference image $I_k$ and a text concept prompt $P_k$, a segmentor ${\varphi }_{seg}$ extracts the target object mask $M_k$, which is subsequently processed by a reconstructor ${\varphi }_{rec}$ to generate a raw 3D asset in the camera frame. (2) Automated asset canonicalization: The raw 3D model undergoes orientation rectification (utilizing RANSAC and LLM-based semantic priors) and scaling/grounding via a semantic knowledge base to produce a standardized, canonicalized asset. (3) Scene-aware Integration: Guided by an integration text prompt, an integration and synthesis model performs layout reasoning and pose optimization to seamlessly insert the canonicalized asset into a source 3D scene, resulting in the final augmented 3D scene.

3.1. Open-Vocabulary 3D Asset Extraction

The first stage bridges high-level semantic specification and low-level 3D reconstruction. Existing single-image 3D reconstruction methods typically require an accurate 2D instance mask as input, which is inconvenient for end users and breaks the natural interaction loop of text-driven creation. We therefore adopt a Prompt-to-Reconstruct strategy that predicts the object mask directly from a concept prompt. For each reference pair $(I_k,P_k)$, we first obtain a binary mask:

$$M_k={\mathsf{\Phi }}_{\mathrm{seg}}(I_k,P_k),M_k\in {\{0,1\}}^{H_k\times W_k},$$

(3)

where ${\mathsf{\Phi }}_{\mathrm{seg}}(·)$ denotes a promptable open-vocabulary segmentor, and $M_k$ identifies the image region corresponding to the target object. This design allows the user to specify an object instance using natural language, such as “a silver refrigerator” or “the wooden side chair near the window”, without requiring manual polygon annotation.

The predicted mask is then passed, together with the original image, to a mask-conditioned single-image reconstructor:

$${\mathcal{A}}_{k,\mathrm{raw}}^c={\mathsf{\Phi }}_{\mathrm{rec}}(I_k,M_k)=({\mathbf{V}}_k^c,{\mathbf{F}}_k,{\mathbf{U}}_k,{\ell }_k),$$

(4)

where ${\mathbf{V}}_k^c={\left\{{\mathbf{v}}_{k,n}^c\right\}}_{n=1}^{N_k}$ is the set of reconstructed mesh vertices in the camera frame, ${\mathbf{F}}_k$ is the face connectivity, and ${\mathbf{U}}_k$ denotes the appearance representation (e.g., texture maps or vertex colors). The output is instance-specific: it corresponds to the particular physical object depicted in the image rather than to a generic category prototype.

This stage converts a semantically grounded image region into a 3D object representation, but the resulting asset is not yet scene-ready. In particular, the reconstruction is expressed in a camera-dependent coordinate frame, its metric scale is unknown due to single-view ambiguity, and its orientation is generally inconsistent with the conventions required by SSR. These issues motivate the canonicalization stage described next.

3.2. Automated Asset Canonicalization

To bridge the representation gap between raw reconstruction and structured scene synthesis, we could canonicalize each reconstructed asset by estimating a similarity transform [41,42,43]

$${\mathcal{T}}_k=({\mathbf{R}}_k,s_k,{\mathbf{t}}_k),$$

(5)

where ${\mathbf{R}}_k\in SO\left(3\right)$ is a rotation matrix, $s_k\in {\mathbb{R}}_{+}$ is a global isotropic scaling factor, and ${\mathbf{t}}_k\in {\mathbb{R}}^3$ is a translation vector.

Applying ${\mathcal{T}}_k$ maps each raw vertex from the camera frame to the canonical object frame:

$${\mathbf{v}}_{k,n}^o=s_k{\mathbf{R}}_k{\mathbf{v}}_{k,n}^c+{\mathbf{t}}_k,n=1,\dots ,N_k.$$

(6)

Accordingly, the canonicalized asset is written as:

$${\mathcal{A}}_{k,\mathrm{can}}^o=({\mathbf{V}}_k^o,{\mathbf{F}}_k,{\mathbf{U}}_k,{\ell }_k),{\mathbf{V}}_k^o={\left\{{\mathbf{v}}_{k,n}^o\right\}}_{n=1}^{N_k}.$$

(7)

By construction, ${\mathcal{A}}_{k,\mathrm{can}}^o$ is upright, canonically scaled based on a category-level prior, and grounded at the origin of its local canonical frame, making it directly compatible with the SSR interface.

3.2.1. Gravity-Aware Orientation Rectification

Raw single-view reconstructions often inherit camera-induced tilt and arbitrary heading. To establish a consistent up direction, we first isolate the lowermost 20% of the mesh vertices along the camera’s Y-axis (the axis pointing upward in the camera frame) and designate this subset as the base. We first estimate the supporting surface normal ${\mathbf{n}}_{k,\mathrm{base}}^c\in {\mathbb{R}}^3$ by fitting a plane to the lower portion of the reconstructed mesh using RANSAC with a distance threshold of 0.01 and 10,000 iterations to ensure robust fitting.For the front direction, we project the asset’s geometry onto the horizontal plane (the X-Z plane in the camera frame) and perform Principal Component Analysis (PCA). The first principal component, which captures the dominant horizontal axis of elongation, is extracted as the candidate front-back axis. To resolve the front-direction ambiguity, we extract a dominant horizontal axis from the asset geometry and use an LLM-based semantic prior conditioned on $(P_k,{\ell }_k)$ to determine which direction along this axis corresponds to the functional front.Specifically, we adopt the Mimo-embodied [44] and query it with the prompt template: “For a bed, which end typically faces the user or serves as the primary functional front? The object is symmetric along its z-axis.” The LLM’s response, which we validate through a consistency check, is used to assign the sign of the axis, yielding the final semantic front direction ${\mathbf{v}}_{k,\mathrm{front}}^c\in {\mathbb{R}}^3$. For objects with severe front-back symmetry (e.g., a round table), we default to the positive direction of the principal axis.

We then estimate the rotation ${\mathbf{R}}_k$ by solving a constrained optimization:

$$\underset{{\mathbf{R}}_k\in SO\left(3\right)}{\min }\left({∥{\mathbf{R}}_k{\mathbf{n}}_{k,\mathrm{base}}^c-{\mathbf{u}}^o∥}_2^2+\lambda {∥{\mathbf{R}}_k{\mathbf{v}}_{k,\mathrm{front}}^c-{\mathbf{f}}^o∥}_2^2\right),$$

(8)

where ${\mathbf{u}}^o={[0,0,1]}^{\top }$ and ${\mathbf{f}}^o={[1,0,0]}^{\top }$ are the canonical up and forward axes in the object frame, respectively, and $\lambda =0.1$ balances gravity alignment against functional orientation consistency. This optimization is solved using the Kabsch-Umeyama algorithm [45] on the union of two weighted point sets, where the first set comprises base normal vectors and the second set comprises front-direction vectors, thereby converting the problem to a standard Procrustes alignment. This optimization ensures that the asset is placed upright and assigned a consistent semantic heading.

3.2.2. Category-Level Scale Restoration and Grounding

A key challenge in single-view reconstruction is the inherent scale-depth ambiguity, which leaves the absolute physical size of ${\mathcal{A}}_{k,\mathrm{raw}}^c$ unconstrained. To address this issue, we introduce a category-level semantic-to-physical mapping based on object category priors. Specifically, given the category label ${\ell }_k$, we retrieve a reference physical height $H_{\mathrm{ref}}\left({\ell }_k\right)$ from a semantic knowledge base.This knowledge base is constructed by aggregating the mean height statistics for each object category from the COCO [46] and SUN RGB-D [47] datasets, followed by manual curation to remove statistical outliers. For categories not present in these datasets, we use standard anthropometric and furniture dimensions from online design guidelines.

Let:

$${\tilde{\mathbf{v}}}_{k,n}={\mathbf{R}}_k{\mathbf{v}}_{k,n}^c$$

(9)

denote the rotated vertices after orientation rectification. We estimate the object height in the rectified frame as:

$$h_k=\underset{n}{\max }{\left({\tilde{\mathbf{v}}}_{k,n}\right)}_z-\underset{n}{\min }{\left({\tilde{\mathbf{v}}}_{k,n}\right)}_z,$$

(10)

and compute the global scale factor by:

$$s_k={\displaystyle \frac{H_{\mathrm{ref}}\left({\ell }_k\right)}{h_k}}.$$

(11)

As the scale is derived from a category-level height prior, this procedure yields a canonical scale that is plausible at the category level rather than an absolute metric scale for the specific instance.

After rotation and scaling, we define:

$${\widehat{\mathbf{v}}}_{k,n}=s_k{\mathbf{R}}_k{\mathbf{v}}_{k,n}^c.$$

(12)

To standardize the asset for downstream placement, we further ground it at the origin of the canonical frame. Specifically, we compute the bottom center of the axis-aligned bounding box in the rectified frame:

$${\mathbf{c}}_{k,\mathrm{base}}=\left[\begin{array}{c}{\displaystyle \frac{x_{k,\min }+x_{k,\max }}{2}}\\ {\displaystyle \frac{y_{k,\min }+y_{k,\max }}{2}}\\ z_{k,\min }\end{array}\right],$$

(13)

where $x_{k,\min },x_{k,\max },y_{k,\min },y_{k,\max },z_{k,\min }$ are computed from the set ${\left\{{\widehat{\mathbf{v}}}_{k,n}\right\}}_{n=1}^{N_k}$. The translation vector is then defined as:

$${\mathbf{t}}_k=-{\mathbf{c}}_{k,\mathrm{base}}.$$

(14)

Therefore, the final canonicalized vertices satisfy:

$${\mathbf{v}}_{k,n}^o=s_k{\mathbf{R}}_k{\mathbf{v}}_{k,n}^c+{\mathbf{t}}_k,$$

(15)

which grounds the asset at $(0,0,0)$ in its local canonical frame and makes it ready for layout integration.

3.3. Layout Reasoning Within Structured Scene Representation

Once canonicalized, ${\mathcal{A}}_{k,\mathrm{can}}^o$ is inserted into the Structured Scene Representation (SSR) used by ReSpace. SSR serves as a layout abstraction layer that encodes room topology, object poses, and relational constraints.

Unlike conventional Manhattan-world formulations, ReSpace explicitly supports non-orthogonal floor plans. For each object instance i, the layout planner predicts a world-space pose $({\mathbf{p}}_i^w,{\theta }_i)$, where ${\mathbf{p}}_i^w={[x_i,y_i,z_i]}^{\top }$ is the translation and ${\theta }_i$ is the yaw angle around the world up-axis. Given the canonical asset vertices $\left\{{\mathbf{v}}_{i,n}^o\right\}$ associated with $a_i$, the corresponding world-space geometry is instantiated as

$${\mathbf{v}}_{i,n}^w={\mathbf{R}}_z\left({\theta }_i\right){\mathbf{v}}_{i,n}^o+{\mathbf{p}}_i^w,$$

(16)

where ${\mathbf{R}}_z\left({\theta }_i\right)$ denotes rotation about the global vertical axis. The horizontal position must satisfy ${[x_i,y_i]}^{\top }\in \mathcal{R}$, while $z_i=0$ for floor-supported objects.The current formulation and quantitative experiments focus on floor-supported objects. Wall-mounted and ceiling-mounted objects would require additional parameters, including support-surface normals, attachment anchors, and contact constraints. Such support-specific placement mechanisms are not quantitatively evaluated in the present work.

The LLM-based layout planner explicitly reasons over:

• Spatial affordances: ensuring accessibility and sufficient functional clearance, such as maintaining unobstructed walkable paths;
• Functional priors: modeling semantic relationships among objects, such as placing a refrigerator adjacent to a kitchen counter;
• Geometric constraints: avoiding collisions and respecting the boundaries induced by irregular floor geometry.

By treating ${\mathcal{A}}_{k,\mathrm{can}}^o$ as a first-class entity in SSR, our framework unifies user-captured in-the-wild assets and pre-modeled database assets under a common reasoning interface. This design enables consistent manipulation and compositional scene synthesis across heterogeneous asset sources.

4. Experiments

4.1. Experimental Setup

• Implementation Details. All experiments are conducted on a high-performance server equipped with an Intel(R) Xeon(R) Platinum 8470Q CPU (16 vCPUs) and a single NVIDIA GPU with 32 GB VRAM (configured as vGPU-32GB). The experimental environment is deployed as a virtualized GPU instance, where one dedicated vGPU with 32 GB of visible GPU memory is allocated to each experimental process. All reported experiments are executed using this single vGPU instance, without multi-GPU parallelism or distributed inference. The vGPU memory allocation remains fixed throughout all experiments, ensuring that all evaluated methods are tested under the same GPU resource constraint.

The server runs Ubuntu 22.04 LTS with Python 3.10 and PyTorch 2.5.1. The CUDA runtime used for neural-network inference is CUDA 12.4, together with the corresponding NVIDIA vGPU driver environment provided by the server. The main geometry-processing dependencies include NumPy 1.26.4, Open3D 0.18.0, and Trimesh 4.4.9. Unless otherwise specified, neural-network inference is performed using mixed-precision computation, whereas geometric post-processing and metric computation are performed in FP32.

All segmentation and reconstruction models are evaluated using their publicly released pretrained checkpoints without additional fine-tuning on our evaluation data. Specifically, the official pretrained checkpoints corresponding to SAM 3 and SAM3D are used in our pipeline. The comparison methods, including CLIPSeg, SEEM, Grounded-SAM, TripoSR, InstantMesh, and SF3D, are also evaluated using their officially released pretrained checkpoints and default inference configurations unless otherwise stated. The same checkpoint and configuration are retained for all repeated evaluations.

For reproducibility, Python, NumPy, and PyTorch random number generators are initialized with a fixed seed of 42. For stochastic scene-level experiments, we additionally repeat the evaluation using three random seeds (42, 123, and 3407) and report the averaged results. The same seed set is applied to all compared configurations whenever stochastic sampling is involved.

The entire pipeline, including open-vocabulary segmentation via SAM 3, single-image reconstruction via SAM3D, and the subsequent canonicalization and layout synthesis modules, operates efficiently within this resource constraint. The peak GPU memory consumption during the most intensive phase (3D reconstruction and refinement) is 18.3 GB. GPU memory usage is measured from the allocated 32 GB vGPU memory space during single-process inference, and no additional GPU devices are used during the measurement.

Runtime is measured using wall-clock time with CUDA synchronization before and after each inference. Model initialization, checkpoint loading, and offline data preparation are excluded from the reported runtime, whereas all online processing stages required to transform an input text–image pair into a layout-ready scene are included. Specifically, the timing starts after the input image and text prompt have been loaded and covers text-conditioned segmentation, foreground preprocessing, single-image 3D reconstruction, mesh refinement, asset canonicalization, semantic scaling, and final scene placement. The timer stops when the complete scene representation has been produced. Offline GPT-4o-based prompt-bank construction and visualization rendering are excluded because they are not part of online inference.

Before timing, 20 warm-up runs are performed to eliminate one-time initialization and CUDA-kernel overhead. Runtime statistics are then collected over 200 evaluation scenes on the same single vGPU-32GB instance. Each scene is processed independently, and no batch-level or multi-GPU acceleration is used. In terms of efficiency, the average inference time to generate a complete indoor scene from scratch is 8.7 s per scene. Across the measured scenes, the runtime is $8.7\pm 0.6$ s per scene (mean ± standard deviation).

For the closed-set evaluation, all 500 test scenes in each of the bed, liv, and all splits are evaluated. For the open-set evaluation, all 1000 collected text–image pairs are processed for scene-level assessment. Deterministic component-level experiments are repeated to verify numerical consistency, while stochastic scene-level evaluations are repeated over the three aforementioned random seeds. Unless otherwise specified, quantitative results are reported as the average over repeated runs.

• Datasets. To evaluate the effectiveness of OpenAsset and to provide an initial asset pool for 3D indoor scene synthesis, we adopt the Scene-Prompt dataset built upon 3D-FRONT [48], following the same preprocessing pipeline and data splits as prior work [24]. The dataset is divided into three categories: bed (bedrooms), liv (living rooms and dining rooms), and all. After filtering noisy samples with a voxelization-based strategy, the resulting training/validation/test splits contain 6328/500/500, 3830/500/500, and 12055/500/500 scenes for bed, liv, and all, respectively.

Since 3D-FRONT does not provide textual descriptions for objects, we generate object-level captions using GPT-4o based on rendered images of 3D-FUTURE assets.

For each object, the model receives rendered mesh images captured from single and multiple viewing perspectives and is instructed to produce ten concise descriptions covering object category, material, color, shape, and distinctive appearance attributes without introducing unsupported spatial context. The generated descriptions were manually audited, screened for invalid textual content and deduplicated for redundant samples before being added to the prompt bank. GPT-4o is used only for offline prompt construction and is not used to generate the evaluated object geometry.

• Baselines. We evaluate OpenAsset at both the component and system levels. For open-vocabulary segmentation, we compare SAM 3 against representative text-promptable segmentation methods, including CLIPSeg, SEEM, and Grounded-SAM. For single-image 3D reconstruction, we compare SAM3D with TripoSR, InstantMesh, and SF3D. To ensure a fair comparison, all reconstruction methods take the same foreground object crop as input and are followed by the same canonicalization and downstream layout modules unless otherwise specified.

For scene synthesis, we compare OpenAsset against the original ReSpace pipeline with closed-set asset retrieval, which serves as the most direct system-level baseline. We additionally report results of ATISS and MiDiffusion, two representative floor-conditioned 3D indoor scene synthesis methods. Note that ATISS and MiDiffusion are scene-layout baselines rather than photo-conditioned personalization baselines, as they cannot reconstruct user-specified assets from reference images. At the system level, we further analyze oracle and ablated variants, including GT-mask, GT-mesh, and w/o-canonicalization settings, to isolate the contributions of segmentation, reconstruction, and canonicalization.

• Evaluation Metrics. We evaluate OpenAsset from four aspects: open-vocabulary segmentation, single-image 3D reconstruction, asset canonicalization, and scene-level synthesis.

For open-vocabulary segmentation, we report mean Intersection over Union (mIoU), Dice coefficient, and Boundary F-score (BFScore). mIoU and Dice measure the overlap between the predicted and ground-truth object masks, while BFScore evaluates boundary accuracy. Higher values indicate better segmentation performance.

For single-image 3D reconstruction, we uniformly sample point clouds from the reconstructed and ground-truth meshes and report Chamfer Distance (CD), 3D F-score, and Normal Consistency. Let $\mathcal{P}$ and $\mathcal{G}$ denote the point clouds sampled from the reconstructed and ground-truth meshes, respectively. CD is defined as

$$\mathrm{CD}(\mathcal{P},\mathcal{G})={\displaystyle \frac{1}{\left|\mathcal{P}\right|}}\sum _{p\in \mathcal{P}}\underset{g\in \mathcal{G}}{\min }{\parallel p-g\parallel }_2+{\displaystyle \frac{1}{\left|\mathcal{G}\right|}}\sum _{g\in \mathcal{G}}\underset{p\in \mathcal{P}}{\min }{\parallel g-p\parallel }_2.$$

The 3D F-score measures geometric completeness based on point-level precision and recall under a fixed distance threshold, while Normal Consistency evaluates the agreement between the surface normals of matched points. Lower CD and higher 3D F-score and Normal Consistency indicate better reconstruction quality.

For asset canonicalization and open-set integration, we use Angular Error, Scale Error, and Layout Success Rate. Angular Error measures the deviation between the estimated and reference canonical orientations. Scale Error measures the relative difference between the predicted physical height and the independently obtained ground-truth height, which is distinct from the category-level height prior used during semantic scaling. Layout Success Rate is the percentage of generated scenes in which all evaluated objects are properly grounded, remain within the room boundary, and do not collide with other objects. Lower Angular Error and Scale Error and a higher Layout Success Rate indicate better performance.

For scene-level synthesis, following ReSpace [24], we report FID, ${\mathrm{FID}}_{\mathrm{CLIP}}$, and KID to measure the feature-distribution discrepancy between generated and reference scene renderings. FID uses Inception features, whereas ${\mathrm{FID}}_{\mathrm{CLIP}}$ uses CLIP image features. Lower values indicate that the generated scenes are closer to the reference distribution.

We additionally evaluate geometric layout violations using Out-of-Bounds loss (OOB), Mesh Boundary Loss (MBL), and Voxelization-Based Loss (VBL). Following ReSpace [24], each scene is voxelized using a voxel size of $0.05\mathrm{m}$. Let $V_j\left(q\right)\in \{0,1\}$ indicate whether voxel q is occupied by the j-th object, and let $V_R\left(q\right)\in \{0,1\}$ indicate whether voxel q lies within the valid room region. OOB is computed as

$$\mathrm{OOB}=\sum _j\sum _q{V}_j\left(q\right)\left(1-V_R\left(q\right)\right),$$

which measures the amount of object geometry extending beyond the room boundary. MBL is defined as

$$\mathrm{MBL}=\sum _{m<n}\sum _q{V}_m\left(q\right)V_n\left(q\right),$$

where each unordered object pair is counted only once. MBL measures inter-object collisions, while VBL combines boundary violations and inter-object collisions into an overall layout violation metric. Lower OOB, MBL, and VBL indicate better geometric validity. All layout-related metrics are evaluated only on floor-supported objects.

For the user study, participants evaluate the generated scenes in terms of Realism, Consistency, and Plausibility using a five-point scale, with higher scores indicating better perceptual quality.

4.2. Open-Vocabulary Segmentation

We first evaluate the object extraction stage by comparing SAM 3 with state-of-the-art open-vocabulary segmentation methods, including CLIPSeg, SEEM, and Grounded-SAM, under text-conditioned settings. The quantitative results are summarized in Table 1. SAM 3 achieves the best performance on all three metrics, with an mIoU of 0.86, a Dice score of 0.92, and a BFScore of 0.84. Relative to the strongest baseline, Grounded-SAM, SAM 3 improves mIoU from 0.83 to 0.86, Dice from 0.90 to 0.92, and BFScore from 0.80 to 0.84. The larger gain in BFScore suggests that SAM 3 produces more precise object boundaries, which is particularly important for downstream geometry reconstruction.

Table 1. Quantitative comparison of open-vocabulary segmentation methods under text-conditioned object extraction. Higher is better for all metrics.

Method	mIoU ↑	Dice ↑	BFScore ↑
CLIPSeg [49]	0.71	0.82	0.69
SEEM [31]	0.78	0.87	0.75
Grounded-SAM [50]	0.83	0.90	0.80
SAM 3 [33]	0.86	0.92	0.84

We further examine how segmentation quality affects downstream 3D reconstruction. To isolate this factor, we fix SAM3D as the reconstruction backbone and vary only the input foreground masks. Table 2 shows a clear monotonic trend: better masks lead to better reconstructed geometry. Using SAM 3 masks yields a CD of 0.48, an F-score of 0.87, and a Normal Consistency of 0.94, outperforming all automatic mask providers and reducing the gap to the GT-mask upper bound (0.32/0.91/0.96). These results indicate that accurate object boundaries are critical for preserving geometric details and suppressing reconstruction artifacts.

Table 2. Impact of segmentation quality on downstream 3D reconstruction. All methods use SAM3D as the reconstruction model and differ only in the provided foreground masks. Lower is better for CD; higher is better for F-score and Normal Consistency.

Mask Provider	CD ↓	F-Score ↑	Normal Cons. ↑
CLIPSeg [49]	0.92	0.72	0.86
SEEM [31]	0.71	0.79	0.89
Grounded-SAM [50]	0.55	0.84	0.92
SAM 3 [33]	0.48	0.87	0.94
GT Mask	0.32	0.91	0.96

4.3. 3D Reconstruction and Canonicalization

We next evaluate the quality of reconstructed 3D assets. To ensure a fair comparison, SAM 3 is used as the default segmentation module to provide the same foreground object for all reconstruction baselines. Under this controlled setting, we compare SAM3D with TripoSR, InstantMesh, and SF3D.

As reported in Table 3, SAM3D performs best on all reconstruction metrics, achieving the lowest CD (0.48), the highest F-score (0.87), and the highest Normal Consistency (0.94). Compared with the strongest baseline SF3D, SAM3D reduces CD from 0.52 to 0.48, improves F-score from 0.85 to 0.87, and improves Normal Consistency from 0.92 to 0.94. These results indicate that SAM3D recovers more faithful geometry and produces smoother, more consistent surfaces, which is especially beneficial for objects with fine-grained structural details.

Table 3. Quantitative comparison of single-image 3D reconstruction methods. All methods take the same foreground object segmented by SAM 3 as input. Lower is better for CD; higher is better for F-score and Normal Consistency.

Method	CD ↓	F-Score ↑	Normal Cons. ↑
TripoSR [51]	0.65	0.80	0.88
InstantMesh [52]	0.58	0.83	0.90
SF3D [53]	0.52	0.85	0.92
SAM3D [54]	0.48	0.87	0.94

We further study whether the proposed canonicalization module improves the usability of reconstructed assets for downstream scene synthesis. Quantitative results in Table 4 show that the full canonicalization pipeline substantially reduces both angular error and scale error while dramatically improving layout success rate. This confirms that canonicalization is essential for transforming reconstructed meshes into layout-ready assets that can be inserted into indoor scenes without manual adjustment.

Table 4. Ablation study of the canonicalization module. Both orientation rectification and semantic scaling are critical for producing layout-ready assets.

Configuration	Angular Error (°) ↓	Scale Error (%) ↓	Layout Success Rate (%) ↑
Raw Output	21.4	33.2	41.3
w/o Gravity Alignment	22.8	14.1	62.0
w/o Semantic Scaling	4.2	29.8	74.7
Ours (Full)	3.1	9.8	84.2

4.4. Personalized Scene Editing and Synthesis

We first evaluate OpenAsset on the 3D-FRONT [48] test split, where all objects can be retrieved from the asset library. This closed-set setting verifies whether introducing OpenAsset compromises the original retrieval-based modeling capability. As shown in Table 5, OpenAsset consistently outperforms ReSpace across all reported metrics on all three splits. For example, on the all split, OpenAsset improves FID from 44.85 to 43.21, ${\mathrm{FID}}_{\mathrm{CLIP}}$ from 2.43 to 2.20, KID from 2.44 to 1.96, OOB from 160.2 to 142.5, MBL from 181.6 to 139.8, and VBL from 341.8 to 298.4. These results show that incorporating personalized asset modeling does not weaken closed-set scene generation; instead, it yields consistent gains over the original ReSpace pipeline. The visualization results are presented in Figure 2.

Table 5. Closed-set evaluation on the 3D-FRONT test split. Lower is better for FID, FID_CLIP, KID, OOB, MBL, and VBL.

Split	Method	FID ↓	FIDCLIP ↓	KID ↓	OOB ↓	MBL ↓	VBL ↓
bed	ATISS [9]	43.51	2.34	2.51	414.3	99.7	514.1
	MiDiffusion [11]	43.18	2.23	1.34	360.1	66.6	427.0
	ReSpace [24]	44.77	2.70	2.17	67.4	140.7	208.1
	OpenAsset (Ours)	44.12	2.32	2.08	65.2	122.6	191.7
liv	ATISS [9]	44.14	2.26	8.06	506.6	135.1	641.6
	MiDiffusion [11]	40.76	2.11	4.29	361.5	117.1	478.7
	ReSpace [24]	46.17	2.42	8.05	254.4	310.4	564.8
	OpenAsset (Ours)	42.38	2.18	5.12	231.6	188.7	420.3
all	ATISS [9]	45.58	2.37	3.87	631.4	108.5	739.8
	MiDiffusion [11]	42.57	2.14	1.27	327.4	87.1	414.5
	ReSpace [24]	44.85	2.43	2.44	160.2	181.6	341.8
	OpenAsset (Ours)	43.21	2.20	1.96	142.5	139.8	298.4

Figure 2. The visualization results reflect the overall execution flow of our framework, including mask extraction from input raw images, subsequent 3D object reconstruction and final scene arrangement deployment. Given an RGB image and a text description specifying the target object (leftmost), we show from left to right: the segmentation mask obtained via the text prompt, the 3D reconstruction of the target furniture, and the final composition of the reconstructed object placed in a novel indoor scene. The first column corresponds to original captured images paired with matched text prompts, the second column displays prompt-guided object segmentation masks extracted from input visuals, the third column renders the recovered 3D model outcomes, and the last column exhibits the practical application of our extracted real objects within 3D indoor scene layout construction.

Compared with ATISS and MiDiffusion, OpenAsset exhibits a different trade-off. While MiDiffusion achieves the best image-distribution metrics such as FID and KID on several splits, OpenAsset delivers substantially better physical plausibility metrics, especially OOB and VBL, and also improves over ReSpace in both fidelity-oriented and physical-layout metrics. This behavior is expected because ATISS and MiDiffusion are optimized for layout generation under closed-set assumptions, whereas OpenAsset is designed to support personalized asset creation and scene integration.

4.5. Evaluation in the Open-Set Setting

We further evaluate OpenAsset in a more challenging open-set setting, where target objects are absent from the asset library. This evaluation is fundamentally different from directly testing on the 3D-FRONT dataset, which highlights the advantage of our approach. In this setting, the input consists of around 1000 groups of user prompts and corresponding reference images (illustrated in Figure 2), of which 900 are images sourced from the internet and 100 are photos taken in real-world environments.The evaluation set covers ten common indoor-object categories, including chairs, tables, sofas, beds, cabinets, shelves, desks, stools, lamps, and miscellaneous decorative or functional objects. To avoid a strong bias toward any single category, the test samples are approximately balanced across the major furniture categories, while a smaller proportion is assigned to less frequently occurring objects such as lamps and decorative items. Specifically, the 1000 test instances contain 150 chairs, 130 tables, 120 sofas, 100 beds, 120 cabinets, 90 shelves, 90 desks, 70 stools, 60 lamps, and 70 miscellaneous objects. Samples are selected according to three criteria: (1) the target object is clearly identifiable from the reference image; (2) a sufficiently large portion of its geometry and appearance is visible to support reliable reconstruction; and (3) the object is not subject to severe occlusion, extreme image degradation, or ambiguity that prevents its category and geometric structure from being determined.

To ensure that this evaluation genuinely reflects an open-set scenario, we define an object as an out-of-library instance when the exact target instance is not represented by any 3D model in the asset library, although semantically similar objects belonging to the same category may still exist. For each collected test image, we first retrieve the most similar assets from the complete library using category labels and visual-semantic similarity and subsequently perform manual inspection of the retrieved 3D models and their rendered views. If an asset with the same instance-level geometry and appearance is found, the sample is removed from the open-set evaluation. In contrast, category-level similarity alone does not lead to exclusion; for example, a test chair remains an open-set instance as long as the same chair model is unavailable in the library, even if other visually similar chairs are present. This instance-level verification procedure prevents the evaluation from degenerating into conventional in-library retrieval.

It is worth noting that, for our pipeline, there is no practical distinction between these two sources: as long as the textures in an image contain sufficient 3D information that can be reconstructed, the image is directly usable in our workflow. In contrast to methods like ReSpace, which retrieve suitable 3D assets from a library based on semantic similarity, our system follows a pipeline of segmentation, 3D reconstruction, and asset placement, allowing it to extract 3D assets from any text–image pair without relying on an existing 3D asset library. This directly addresses a practical scenario that ReSpace cannot support: when users simply want to use a piece of furniture found online or an object from their physical surroundings for scene layout. In our evaluation, the system first generates 3D asset models from the given inputs, then combines these newly created assets with the existing library to produce the final scenes.

For quantitative evaluation, the reference annotations are obtained independently from the predictions generated by OpenAsset. Ground-truth object masks are produced by manually refining the automatically generated segmentation masks at the pixel level, particularly around thin structures, object boundaries, and partially occluded regions. Each refined mask is subsequently inspected by a second annotator to remove obvious segmentation errors. These manually verified masks are used only for evaluation and are not provided to the reconstruction pipeline during inference.

Because metrically accurate ground-truth meshes are generally unavailable for arbitrary objects collected from Internet images, we do not treat the reconstructed geometry generated by OpenAsset as ground truth. Instead, mesh-based geometric inspection is performed only on the subset of real-world objects for which an independently acquired reference model is available. For these objects, the reference meshes are obtained through multi-view reconstruction or 3D scanning performed independently of the OpenAsset pipeline. Internet-sourced samples without independently verifiable reference geometry are retained for open-set reconstruction and scene-layout evaluation but are not used for mesh-dependent quantitative measurements. This separation avoids circular evaluation in which the model output itself would implicitly define the reference geometry.

The canonical orientation of each object is manually annotated according to its semantically identifiable front-facing direction. For objects with an obvious functional front, such as chairs, sofas, cabinets, and desks, the direction in which a user would normally face or interact with the object is defined as the canonical front. For approximately rotationally symmetric objects or objects without a visually meaningful front direction, orientation ambiguity is explicitly considered and such instances are excluded from the angular-error calculation when a unique canonical direction cannot be reliably determined. Two annotators independently inspect the reference image together with the corresponding object geometry to determine the canonical direction, and disagreements are resolved through joint inspection.

The physical height used for scale evaluation is obtained from independently verifiable sources rather than from the reconstructed output. For real-world photographs collected by us, the actual object height is measured directly during data acquisition. For Internet-sourced objects, metric dimensions are obtained only when reliable product specifications or manufacturer-provided dimensions can be associated with the exact object instance. Samples for which no trustworthy metric dimension can be established are excluded from scale-error computation. Consequently, angular and scale errors are evaluated on the subsets for which reliable orientation and metric-size ground truth are available, whereas all 1000 test cases are retained for evaluating reconstruction applicability and scene-layout success.

A generated scene is regarded as successful if all objects have no mutual collision or overlap and are placed firmly on the ground without floating; otherwise, the scene is deemed a failure due to object collision, overlap, or floating issues.

To avoid conflating asset fidelity, canonicalization accuracy, and scene-layout validity, we clarify the meaning of each metric: angular error and scale error reflect the pose and size consistency of instantiated assets, while layout success rate judges the geometric rationality of generated scenes. Accordingly, angular error is computed only for instances with an unambiguous manually verified canonical orientation, and scale error is computed only for instances with independently verified metric dimensions. The layout success rate, in contrast, is evaluated over the complete set of 1000 open-set test cases. This metric-specific evaluation protocol ensures that each quantitative measure is calculated only when the corresponding ground truth is reliably available.

ReSpace acts as our core comparison baseline, as it shares the same structured scene synthesis framework but depends on pre-stored asset retrieval; it can only match similar substitute objects when target instances are missing from the library. As shown in Table 6, OpenAsset substantially outperforms ReSpace across all metrics, reducing angular error to ${3.1}^{\circ }$ (vs. ${18.4}^{\circ }$) and scale error to 9.8% (vs. 23.2%), while achieving a layout success rate of 84.2% compared to ReSpace’s 52.5%. These results highlight the key advantage of OpenAsset in open-set personalized scene synthesis, with representative qualitative comparisons presented in Figure 3.

Table 6. Open-set evaluation on scenes containing unseen target objects. Lower is better for angular and scale errors, and higher is better for layout success rate.

Method	Angular Error (°) ↓	Scale Error (%) ↓	Layout Success Rate (%) ↑
ReSpace	18.4	23.2	52.5
OpenAsset (Ours)	3.1	9.8	84.2

Figure 3. Visualization Results. The visualization results demonstrate how our framework compares with the retrieval-based baseline ReSpace on challenging open-set test samples sharing identical input prompts and reference images. The figure is organized into two rows to highlight the contrast in performance: the first row shows diverse indoor scenes generated by OpenAsset using user-provided photo assets, demonstrating high fidelity in preserving geometric and textural details as well as physically plausible and semantically consistent layouts; the second row presents typical flawed outputs of ReSpace on these hard cases, including object collision, misalignment, abrupt placement, and geometric distortion. The comparison highlights that OpenAsset produces more realistic and personalized results by leveraging canonicalized photo-derived assets.

4.6. Ablation Studies

We conduct ablation studies to isolate the contribution of the canonicalization module. Specifically, we compare the raw reconstructed output, a variant without gravity alignment, a variant without semantic scaling, and the full OpenAsset pipeline.

• Analysis.

Effect of gravity alignment. Comparing Raw Output and w/o Semantic Scaling shows that correcting object orientation is essential for downstream placement. Enabling gravity alignment reduces the angular error from ${21.4}^{\circ }$ to ${4.2}^{\circ }$, while the scale error remains relatively high at $29.8\%$. Although the layout success rate improves to $74.7\%$, the remaining scale mismatch still limits reliable object placement. This indicates that orientation rectification alone is insufficient for producing layout-ready assets.

Effect of semantic scaling. Comparing Raw Output and w/o Gravity Alignment shows that semantic scaling effectively reduces scale mismatch, lowering the scale error from $33.2\%$ to $14.1\%$. However, without gravity alignment, the angular error remains high at ${22.8}^{\circ }$, resulting in a layout success rate of only $62.0\%$. This demonstrates that correcting scale alone cannot fully resolve geometric inconsistencies.

Combined effect. The full model achieves the best performance on all three metrics, reducing angular error to ${3.1}^{\circ }$ and scale error to $9.8\%$, while increasing the layout success rate to 84.2%. Compared with the raw output, this corresponds to a reduction of 18.3° in angular error, a reduction of 23.4 percentage points in scale error, and an improvement of 42.9 percentage points in layout success rate. These results demonstrate that gravity alignment and semantic scaling are complementary, and both are necessary for producing canonicalized assets that can be reliably integrated into indoor scene layouts.

• User Study. To comprehensively assess the perceptual quality of OpenAsset, we expanded our user study to include 150 participants, including 60 3D design professionals and 90 general users, as reported in Table 7. Each participant evaluated 20 paired scene results, with each pair consisting of an OpenAsset result and a retrieval-based baseline (ReSpace) result, resulting in 20 paired ratings per evaluation dimension for each participant. Each participant evaluated generated scenes across three critical dimensions: Realism (photorealistic texture and geometry), Consistency (semantic alignment with text prompts), and Plausibility (physical correctness of object interactions). Participants were presented with paired results from OpenAsset and the retrieval-based baseline (ReSpace) in a randomized order without revealing the source method.
  Table 7. User Study Results with 150 Participants. Participants score each method on a 5-point scale (1–5). Higher scores indicate better performance. Values are reported as mean ± standard deviation. p-values are obtained using paired-sample t-tests, and Cohen’s d denotes the effect size.

  Method	Realism	Consistency	Plausibility	Average Score
  ReSpace (baseline)	$3.42\pm 0.81$	$3.35\pm 0.84$	$3.51\pm 0.79$	$3.43$
  OpenAsset (Ours)	4.26 ± 0.62	4.38 ± 0.58	4.41 ± 0.55	4.35
  p-value	<0.001	<0.001	<0.001	–
  Cohen’s d	$1.05$	$1.22$	$1.27$	–

To account for the repeated evaluation of multiple scenes by the same participant, we averaged the 20 scene-level ratings separately for each participant, method, and evaluation dimension before statistical testing. This procedure yielded one aggregated OpenAsset score and one aggregated ReSpace score per participant for each dimension. Therefore, the participant was treated as the statistical unit of analysis ($n=150$), and the individual scene-level ratings were not treated as independent observations.

For each dimension, the reported mean and standard deviation were calculated across the 150 participant-level aggregated scores, thereby characterizing the variability between participants. As shown in Table 7, OpenAsset achieves higher average ratings than ReSpace across all three dimensions. Specifically, OpenAsset improves the scores from 3.42 to 4.26 for Realism, from 3.35 to 4.38 for Consistency, and from 3.51 to 4.41 for Plausibility. To further assess the paired differences, we performed paired-sample t-tests on the participant-level ratings. The improvements of OpenAsset over ReSpace were statistically significant for Realism ($p<0.001$, Cohen’s $d=1.05$), Consistency ($p<0.001$, Cohen’s $d=1.22$), and Plausibility ($p<0.001$, Cohen’s $d=1.27$), indicating large effect sizes across all three evaluation dimensions. These results demonstrate a statistically significant group-level preference for OpenAsset across all three evaluation dimensions.

Qualitative feedback highlighted that OpenAsset’s ability to incorporate specific user photos results in more natural and engaging scenes compared to the generic assets retrieved by ReSpace. Furthermore, we supplemented the study with an analysis of typical failure cases of ReSpace, including severe occlusion, thin structures, and transparent objects. These complex visual inputs remain challenging for the upstream segmentation and reconstruction modules of our pipeline, and we do not make generalized claims of universal robustness against these hard cases; the observed visual advantages only apply to samples that can be reliably segmented and reconstructed.

5. Conclusions

In this paper, we proposed OpenAsset, a unified framework for personalized and open-vocabulary 3D indoor scene synthesis from natural language and user-provided reference images. The proposed method integrates concept-prompted object segmentation, single-image 3D reconstruction, and asset canonicalization into an LLM-guided scene synthesis pipeline, thereby addressing the closed-set limitation of traditional retrieval-based methods.

The main advantage of OpenAsset is that it enables the use of user-specific real-world objects in structured indoor scene generation. By introducing a canonicalization module, the reconstructed assets can be transformed into scene-ready representations with consistent orientation, scale, and alignment, making them compatible with structured scene representations and subsequent layout operations. In this way, OpenAsset preserves the controllability and editability of existing language-driven scene synthesis frameworks while extending their ability to handle personalized and previously unseen objects.

The experimental results demonstrate that OpenAsset achieves strong performance in open-vocabulary segmentation and single-image 3D reconstruction, while the proposed canonicalization strategy substantially improves the usability of reconstructed assets in downstream scene synthesis. In addition, OpenAsset maintains competitive performance in closed-set settings and shows clear advantages in open-set scenarios, where retrieval-based approaches are inherently limited. These results confirm the effectiveness of the proposed framework for personalized 3D indoor scene generation.

Although the current study demonstrates promising results, several limitations remain. The performance of OpenAsset relies on effective segmentation and single-image reconstruction as critical prerequisites. The quality of the final scene still depends on the robustness of object reconstruction from a single image, and complex object geometries or severe occlusions may affect the final asset quality. In future work, we plan to incorporate multi-view cues, improve reconstruction robustness for challenging objects, and further enhance the interaction between user-provided assets and language-based scene planning. We believe that OpenAsset provides a practical step toward more flexible, realistic, and user-centered 3D content creation.