Cyber-Physical System for Terminal Infrastructure Monitoring: A Depth-Free Registration Framework via Geometric-Model Fusion

Abstract

The monitoring and security of large-scale terminal infrastructures represent a critical application domain for industrial cyber-physical systems. However, real-time 3D visualization in such environments faces significant challenges from dense crowds, specular reflections, and complex architectural layouts. This paper presents a cyber-physical system for terminal infrastructure monitoring, underpinned by a novel, depth-free camera registration framework. At its core, the system establishes explicit geometric mappings across four coordinate systems (world, 3D model, camera, image), leveraging known installation parameters to eliminate dependency on depth sensors. Dynamic inconsistencies are resolved through a multi-stage layout refinement process, enabling robust operation under terminal-specific challenges. The framework maintains real-time performance at over 25 FPS when processing 16 concurrent video streams on commercial hardware. Extensive evaluations demonstrate a 44.9% reduction in registration error compared to state-of-the-art methods, validating the system’s practicality for enhancing situational awareness and security in large-scale, dynamic terminals.

1. Introduction

Dynamic 3D visualization has emerged as a cornerstone technology for augmented reality, intelligent surveillance, and smart infrastructure management [1,2,3]. By fusing real-time video streams with 3D models, this paradigm enables precise spatiotemporal localization of dynamic objects within a reconstructed scene. As illustrated in Figure 1, our proposed framework advances beyond simulated representations to achieve photorealistic dynamic registration in operational environments.

The fusion of video imagery with 3D models, a paradigm established in the early 2000s [4], this approach has enabled widespread outdoor applications such as Google Street View and immersive sports broadcasts [5], as exemplified in Figure 2. Recent years have witnessed significant breakthroughs driven by deep learning, including hierarchical semantic decoupling for complex geometry generation [6], spatiotemporal attention mechanisms for large-scale synthesis [7], and integrated BEV-Gaussian frameworks for multi-view simulation [8]. Parallel efforts continue in geometric methods, such as linear feature-based architectural reconstruction [9] and highly adaptive scene matching algorithms [10].

However, despite these advancements, the translation of this technology to critical indoor environments—particularly airport terminals—presents a formidable and understudied challenge. Airport terminals introduce unique complexities that distinguish them from both outdoor scenes and conventional indoor spaces, including dense crowds, severe occlusions, and pervasive specular reflections. These challenges create a significant gap between the capabilities of current outdoor-oriented or simulation-heavy methods and the demands of robust, real-time terminal visualization.

Airport terminal environments present a set of distinct and compounded challenges that require specialized solutions:

• Geometric Complexity: Modern indoor architecture frequently violates Manhattan world assumptions, featuring curved walls, non-orthogonal intersections, and irregular layouts that confuse traditional geometric methods [11,12,13].
• Dynamic Elements: High pedestrian density creates severe and temporal occlusions, while moving crowds and changing objects break the static scene assumptions used by most registration algorithms [14,15,16].
• Optical Complications: Reflective surfaces (glass walls, polished floors, metallic fixtures) create specular reflections that mislead depth estimation and feature matching algorithms, causing systematic errors in registration [17,18].
• Computational Constraints: Real-time applications require processing speeds that often exceed the capabilities of computationally intensive methods, creating a trade-off between accuracy and speed [14,19,20].

The field of indoor 3D registration continues to evolve, with innovations including reverse projection for real-time texture mapping [21], single-image reconstruction leveraging known dimensions [22], and spherical coordinate conversion of point clouds [23]. Further advances incorporate feature disentanglement through convolutional sparse coding [24,25,26,27] alongside deformation field estimation techniques [28,29]. Crucially, these methods universally suffer from dependency on expensive depth sensors or high-precision point clouds, coupled with computational complexity that precludes real-time performance in complex indoor environments.

Our framework addresses these terminal-specific challenges through a fundamental insight: managed airport spaces offer a powerful but underutilized geometric prior in the form of stable, pre-calibrated camera poses. We transform this prior into a robust, depth-free registration paradigm, specifically designed for terminal environments and effectively bypassing the limitations of depth sensors and unreliable feature matching. The key contributions of this work are as follows:

1.

A Terminal-Tailored Multi-Coordinate Framework: We introduce a geometrically-grounded framework that explicitly links the world, 3D model, camera, and image coordinate systems. This formulation leverages known camera parameters to establish accurate spatial mappings specifically optimized for terminal environments, eliminating dependency on error-prone depth sensors.

2.

A Multi-Stage Layout Refinement Algorithm: To resolve spatial inconsistencies and prevent cumulative drift, we propose a progressive optimization algorithm. It refines the initial layout projection through iterative hypothesis generation and validation at structural junctions, ensuring high geometric fidelity.

3.

Demonstrated Real-Time Performance in Terminal Environments: We validate the framework’s practical efficacy by achieving real-time processing of 16 concurrent video streams at over 25 FPS on commercial hardware. The system maintains superior robustness in extreme scenarios characterized by dense crowds, heavy occlusions, and strong specular reflections.

4.

Comprehensive Terminal Benchmarking and Deployment: Through extensive evaluation on a challenging airport terminal dataset, our method demonstrates a 44.9% reduction in registration error compared to state-of-the-art methods. The system’s practicality is further confirmed by its successful, long-term deployment in a live international airport environment.

2. Related Works and Motivation

2.1. Traditional Registration Methods

Early indoor registration methods predominantly relied on geometric primitives under Manhattan world assumptions, with seminal contributions including vanishing point estimation for layout inference [30] and depth classification via constrained Manhattan categories [31]. While this assumption continues to be utilized to simplify fundamental problem constraints [32], it is inherently violated by dynamic objects commonly found in airport terminals [33]. Classical approaches employing local feature descriptors such as SIFT, SURF [34], and ORB [35] have been extensively explored for image matching. Subsequent advancements, including semantic-geometric dual-constraint frameworks leveraging YOLO for dynamic object identification [36], aim to address these inherent limitations. However, significant challenges persist in terminal environments, including the failure of static scene assumptions, conflicts between computational efficiency and real-time requirements, and poor performance in low-texture regions characteristic of modern airport architecture.

2.2. SLAM and Real-Time Systems

State-of-the-art SLAM systems (e.g., ORB-SLAM2 [14], RTAB-Map [37]) suffer from fundamental limitations arising from their reliance on static-world assumptions, leading to catastrophic failures in dynamic terminal environments. In crowded airport settings, feature points on moving passengers induce erroneous motion parallax interpretation and subsequent tracking drift through false geometric constraints. Recent mitigation attempts—including DS-SLAM semantic segmentation [38], Mix-VPR for SURF descriptors [39], and dynamic feature elimination [40]—address specific aspects of this problem but incur prohibitive computational overhead. Crucially, these approaches remain vulnerable to the dense, non-linear motion patterns characteristic of crowded indoor spaces, limiting their practical deployment in operational airport terminals.

2.3. Deep Learning Approaches

Recent deep learning methods have significantly advanced indoor scene understanding capabilities. End-to-end layout estimation is achieved by approaches such as RoomNet [41] and Flat2-Layout [42]. Subsequent methods, including AtlantaNet [43], relax the strict Manhattan world assumption, while transformer-based models like RoomFormer [44] further enhance reconstruction accuracy. Modern networks (e.g., AdaBins [45], Plane-RCNN [46]) demonstrate strong capabilities in estimating depth and planar surfaces from single images. However, these methods remain susceptible to the inherent limitations of image formation processes, require extensive domain-specific annotated data for training, and exhibit limited generalization to unseen environments—particularly challenging airport terminals with unique architectural features and lighting conditions.

2.4. Motivation

Existing indoor layout estimation methods fundamentally fail in airport terminal scenarios. As evidenced in Figure 3, vanishing-point-based approaches [30,45,46] incorrectly classify multi-plane structures into single planes (green lines), exhibiting catastrophic failure in non-cuboid spaces. Moreover, the fault features about reflected regions are also extracted, which will cause serious errors in the layout estimation. Meanwhile, deep learning solutions(e.g., AtlantaNet [43] in Figure 4) suffer from specular interference, where depth maps mirror reflected scenes, causing erroneous plane merging.

Crucially, both paradigms require high-precision depth—an impractical constraint given terminals’ reflective surfaces. However, airport surveillance systems provide a unique advantage: Known camera poses through calibration. We thus propose a paradigm shift: Leverage camera intrinsics and initial poses to project 3D layouts, bypassing error-prone depth estimation entirely. This enables robust registration through geometric reprojection without depth sensors.

3. Proposed Method

Our framework builds upon the coordinate transformation pipeline commonly employed in computer graphics systems, but extends it to address the unique requirements of real-world airport surveillance. This section first establishes four coordinate systems and computes their correspondence. The 2D layout of the terminal scene is then estimated. Finally, it introduces an optimal solution to refine the coarse layout.

3.1. Method Overview

Figure 5 illustrates the framework’s core architecture, where four interdependent coordinate systems—world (reference frame), 3D model, camera, and image—are geometrically linked through precisely defined transformation matrices.

Building upon the established coordinate systems, we project 3D spatial annotations onto the 2D image plane to generate initial layout candidates. For each candidate, multiple geometric hypotheses are generated and scored to identify the optimal 2D layout. This refined layout directly guides image partitioning for texture analysis while ensuring spatial coherence.

3.2. Four-Coordinate System Construction

Figure 6 illustrates the coordinate system architecture. The world coordinates ${\mathbf{O}}_{\mathrm{W}}$ is used as the reference and choose the Z axis of ${\mathbf{O}}_{\mathrm{W}}$ perpendicular to the ground of the real world. Model coordinates ${\mathbf{O}}_{\mathrm{M}}$ is constructed with its origin aligned to that of ${\mathbf{O}}_{\mathrm{W}}$. Gets the camera coordinates ${\mathbf{O}}_{\mathrm{C}}$. ${\mathbf{O}}_{\mathrm{C}}=\left\{{\mathbf{O}}_{{\mathbf{C}}_{\mathrm{i}}},i=1,2,\dots ,n\right\}$, ${\mathbf{O}}_{{\mathrm{C}}_{\mathrm{i}}}$ represents the coordinates of a camera. The image coordinates ${\mathbf{O}}_{\mathrm{I}}=\left\{{\mathbf{O}}_{{\mathrm{I}}_{\mathrm{i}}},i=1,2,\dots ,n\right\}$, ${\mathbf{O}}_{{\mathrm{I}}_{\mathrm{i}}}$ represent the image coordinates of the correspondence camera.

During model coordinate system construction, we define the origin to coincide with the world coordinate system origin. The 3D model is scaled to real-world dimensions via parameter s, establishing the transformation:

$$s·{\mathbf{O}}_{\mathrm{M}}={\mathbf{O}}_{\mathrm{W}}$$

(1)

The position of camera ${\mathrm{C}}_i$ in the world coordinates is ${\mathbf{O}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}=\left(x_{{\mathrm{W}}_{{\mathrm{C}}_i}},y_{{\mathrm{W}}_{{\mathrm{C}}_i}},z_{{\mathrm{W}}_{{\mathrm{C}}_i}}\right)$, $x_{{\mathrm{W}}_{{\mathrm{C}}_i}}$, $y_{{\mathrm{W}}_{{\mathrm{C}}_i}}$ represent the position of ${\mathrm{C}}_i$ on the plane of the terminal in the real world. $z_{{\mathrm{W}}_{{\mathrm{C}}_i}}$ represents the height of ${\mathrm{C}}_i$. The transformation between the camera coordinate of ${\mathrm{C}}_i$ and the world coordinate can be represented as follows:

$${\mathbf{O}}_{{\mathrm{C}}_{\mathrm{i}}}={\mathbf{R}}_{{\mathrm{C}}_i\mathrm{W}}·{\mathbf{O}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}+{\mathbf{t}}_{{\mathrm{C}}_i\mathrm{W}}$$

(2)

where ${\mathbf{R}}_{{\mathrm{C}}_i\mathrm{W}}$ is the rotation matrix and ${\mathbf{t}}_{{\mathrm{C}}_i\mathrm{W}}$ is the displacement vector of the initial pose of camera ${\mathrm{C}}_i$ in ${\mathbf{O}}_{\mathrm{W}}$. It is worth noting that ${\mathbf{R}}_{{\mathrm{C}}_i\mathrm{W}}$ and ${\mathbf{t}}_{{\mathrm{C}}_i\mathrm{W}}$ are obtained by calibration. The selected calibration templates are perpendicular to the ground. Only one ${90}^{\circ }$ rotation around the X or Y axis is required with respect to the world coordinate system.

The last relationship between ${\mathbf{O}}_{{\mathrm{I}}_{\mathrm{i}}}$ and ${\mathbf{O}}_{{\mathrm{C}}_{\mathrm{i}}}$ is determined by the camera intrinsic ${\mathbf{K}}_i$, the transformation is as follows:

$$z·{\mathbf{O}}_{{\mathrm{I}}_{\mathrm{i}}}={\mathbf{K}}_i·{\mathbf{O}}_{{\mathrm{C}}_{\mathrm{i}}}$$

(3)

where ${\mathbf{K}}_i$ is determined with camera calibration, and z is depth of pixels. Hence, we establish all correspondence between all types of coordinates and the world coordinate system.

3.3. Initial Layout Estimation

For the world scene as ${\mathbf{L}}_{\mathrm{W}}=\left\{\left({\mathrm{s}}_k,{\mathrm{e}}_k\right),k=0,1,2\dots \right\}$, where ${\mathrm{s}}_k$, ${\mathrm{e}}_k$ represent the end point of line segment ${\ell }_k$. For a camera ${\mathrm{C}}_i$, its layout structure in world coordinates is ${\mathbf{L}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}$, ${\mathbf{L}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}\in {\mathbf{L}}_{\mathrm{W}}$, it satisfies the following conditions.

$${\mathbf{L}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}=\left\{\left({\mathbf{s}}_k,{\mathbf{e}}_k\right),0\le \left({\mathbf{s}}_k-{\mathbf{t}}_i\right)·{\mathbf{r}}_i\le d_{i}and0\le \left({\mathbf{e}}_k-{\mathbf{t}}_i\right)·{\mathbf{r}}_i\le d_i\right\}$$

(4)

where $d_i$ represents the farthest distance of the camera view-field, and ${\mathbf{t}}_i$ is the translation vector of camera ${\mathrm{C}}_i$. ${\mathbf{r}}_i$ represents the direction of the camera’s central axis.

Then the corresponding 2-D layout ${\mathbf{l}}_i$ in the image coordinate system can be projected with ${\mathbf{L}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}$, as shown in the following Equation:

$${\mathbf{l}}_i={\displaystyle \frac{1}{z}}·{\mathbf{K}}_i·\left({\mathbf{R}}_{{\mathrm{C}}_i\mathrm{W}}·{\mathbf{L}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}+{\mathbf{t}}_{{\mathrm{C}}_i\mathrm{W}}\right)$$

(5)

where z is the depth of the corresponding point in the camera coordinate system, the z-depth can be obtained according to the CAD drawings.

In practical deployments, cameras frequently undergo pure rotational motion with negligible translation ($\mathbf{t}=\mathbf{0}$), particularly during field-of-view adjustments. As shown in Figure 7, our 2D layout estimation leverages Kneip’s geometric constraints [47] to solve for the rotation matrix $\Delta \mathbf{R}$ independently of translation. Given corresponding 3D point pairs $\{{\mathbf{x}}_i,{\mathbf{x}}_i^{\prime }\}$ satisfying Kneip constraints, $\Delta \mathbf{R}$ is computed as follows:

$$∥{\left[{\mathbf{x}}_i^{\prime }\right]}_{\times }\Delta \mathbf{R}{\mathbf{x}}_i∥=0$$

(6)

When a pure rotation occurs, the layout is updated. The updated layout is ${\mathrm{I}}_i^{\prime }$.

$${\mathbf{l}}_i^{\prime }={\displaystyle \frac{1}{z}}·{\mathbf{K}}_i·\left({\mathbf{R}}_{{\mathrm{C}}_i\mathrm{W}}·\Delta \mathbf{R}·{\mathbf{L}}_{{\mathrm{W}}_{{\mathrm{C}}_i}}+{\mathbf{t}}_{{\mathrm{C}}_i\mathrm{W}}\right).$$

(7)

3.4. Multi-Stage Layout Refinement

In 2D images, layout line segments represent edges where two image planes meet, exhibiting a consistent appearance on both sides. However, when projected into 3D space, this consistency is lost due to inconsistencies in the homography matrix arising from imprecise depth estimation. As shown in Figure 8, this manifests as misaligned edge pixels on different 3D planes—a direct consequence of reprojection errors in template meshes. To resolve this fundamental misalignment problem, we introduce a hypothesis-scoring mechanism that identifies the optimal layout solution through geometric consistency optimization.

Given an initial 2D layout estimation ${\mathbf{l}}_i$, we first extract line segment intersections as keypoints and generate multiple geometric hypotheses within their neighborhood. Each hypothesis undergoes a two-stage evaluation: computing per-line scores followed by aggregation into a direction/length-invariant descriptor vector. This scoring mechanism enables the selection of the locally optimal hypothesis, with the global layout constructed as the precise union of these locally optimized solutions, ensuring geometric consistency across the entire scene.

For a line segment comprising n pixels, we construct scale-invariant regions along its axis as blocks of fixed dimensions ($h\times w$), illustrated in Figure 9. The number of regions per segment is given by $\lfloor 2n/w\rfloor$, where the factor 2 accounts for bidirectional sampling perpendicular to the segment orientation.

For each region, we use Harris corners to describe its pixels and eigenvalues to represent the geometric properties of Harris corner features. We accumulate the Harris parameters to form the feature of the region:

$${\mathbf{V}}_{ij}={\left(V_{ij}^R,V_{ij}^L,V_{ij}^U,V_{ij}^D\right)}^T$$

(8)

$$V_{ij}^R=\sum _{{\lambda }_1>0}{\lambda }_1,V_{ij}^L=\sum _{{\lambda }_1<0}-{\lambda }_1,V_{ij}^{\mathrm{U}}=\sum _{{\lambda }_2>0}{\lambda }_2,V_{ij}^{\mathrm{D}}=\sum _{{\lambda }_2<0}-{\lambda }_2$$

(9)

where $V_{ij}^k$ represents the accumulation of geometric properties of Harris corner features in $\left(i,j\right)$ region, and k represents the four directions of the $\left(i,j\right)$ region. ${\lambda }_1$ and ${\lambda }_2$ represents the gradient of a pixel in two orthogonal directions.

Then compute the similarity of line segments by Euclidean distance d, and the score is computed as the ratio of the similarity regions and the total regions. Where $th$ is set to 0.8 empirically.

$$s={\displaystyle \frac{w}{2n}}\sum _{i=1}^{w}sgn(d-th)$$

(10)

3.5. Real-Time Registration

Our texture mapping pipeline projects 2D video textures onto the 3D model through a sequential process beginning with 3D layout recovery via inverse projection:

$${\mathbf{L}}_{M_i}={\displaystyle \frac{1}{s}}{\mathbf{R}}_{C_{i}W}^{-1}\left({\mathbf{K}}_i^{-1}Z·{\mathbf{l}}_i-{\mathbf{t}}_{C_{i}W}\right)$$

(11)

where ${\mathbf{L}}_{M_i}$ is the recovered 3D layout, s the scaling factor, ${\mathbf{R}}_{C_{i}W}$ camera rotation, ${\mathbf{K}}_i$ intrinsics, Z depth, and ${\mathbf{t}}_{C_{i}W}$ translation. The layout boundaries $\partial {\mathbf{L}}_{M_i}$ then segment source images into distinct texture regions, which are finally mapped onto corresponding 3D model faces to generate photorealistic visualizations with guaranteed geometric consistency.

3.6. General Applicability

Although the proposed framework is specifically designed and validated in the context of airport terminals, its underlying principles are generally applicable to a broad class of managed indoor spaces with similar characteristics. Such spaces include, but are not limited to, shopping malls, transportation hubs (e.g., train stations and subway stations), and large office buildings. These environments typically share the following key features that our method leverages:

• Stable camera installations: Cameras are fixed, and their positions can be pre-surveyed, providing reliable geometric priors.
• Structural layout consistency: The 3D model of the environment is often available or can be constructed, and the layout remains relatively static over time.
• Similar challenges: They exhibit dynamic occlusions from high pedestrian traffic, reflective surfaces, and complex lighting conditions.

By capitalizing on these common characteristics, our method can be seamlessly adapted to these environments without fundamental modifications.

4. Experiment

4.1. DATA

The proposed framework was evaluated on the Airport Terminal Collection, a dedicated dataset comprising 60 min of HD video (1920 × 1080 @ 30fps) from 25 calibrated cameras in an operational terminal. This dataset captures challenging scenarios—including dense crowds, specular reflections, and dynamic lighting in environments such as corridors and security checkpoints—and to enhance evaluation transparency and reproducibility, we have included representative visual samples from multiple camera viewpoints in Figure 10, showcasing synchronized frames from three strategically selected cameras covering distinct terminal areas.

Given the method’s requirement for known camera parameters and 3D models—unavailable in standard public benchmarks—this focused evaluation provides appropriate validation for the target application domain.

4.2. Evaluation Metrics

This carefully curated collection provides the first benchmark for evaluating registration methods in complex terminal environments.

• Registration Accuracy: Assessed via the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE), measured in pixels. The ground truth is defined by the precise projection of the authoritative 3D CAD model onto the image plane using the calibrated camera parameters, ensuring a geometrically meaningful and reproducible benchmark.
• Computational Efficiency: Evaluated based on the processing time per frame (milliseconds) and throughput (frames per second, FPS).
• Robustness: Quantified by measuring performance degradation under challenging conditions, including increasing crowd density, dynamic lighting variations, and the presence of reflective surfaces [10,48].

4.3. Implementation Details

Experimental Environment. All experiments were conducted on a consistent hardware and software platform. The hardware configuration consisted of an Intel Core i7-4770K CPU (Intel Corporation, Santa Clara, CA, USA) operating at 3.5 GHz with 8 cores, an NVIDIA T4 GPU (NVIDIA Corporation, Santa Clara, CA, USA) with 16 GB of VRAM (used to accelerate the rendering pipeline and for inference with deep learning baselines), and a 1 TB NVMe SSD for fast dataset access. The software environment was based on Ubuntu 20.04 LTS, with computations accelerated using CUDA 11.4 and cuDNN 8.2.

3D Model and Camera Pose Acquisition. The 3D scene model was constructed from official CAD drawings provided by the airport authority. Furthermore, the camera intrinsic matrices and initial pose parameters are obtainable, with the latter registered from the terminal’s GIS map. Due to the sensitive nature of the airport’s full layout, a localized 3D model was utilized for our experiments. This approach successfully validates the feasibility of our proposed method while adhering to privacy and security protocols.

In scenarios where camera installation parameters were not previously acquired, a multi-step calibration procedure was implemented using a high-accuracy checkerboard pattern following established computer vision protocols. The calibration process introduces uncertainties primarily in two domains: initial camera pose estimation and lens distortion coefficients. The quality of calibration was rigorously quantified through re-projection error analysis, yielding an average error of 0.15 pixels in our experimental setup—a metric that represents the practical uncertainty level of our system. For comprehensive distortion correction, the Brown-Conrady model was employed to effectively characterize both radial and tangential distortion components. Notably, the integrated multi-stage layout refinement algorithm demonstrates inherent robustness to minor residual errors that persist post-calibration. This architectural consideration ensures sustained registration accuracy under operational conditions where perfect calibration is often unattainable.

From a Single Image to 3D Layout: An Estimation Pipe. To better elucidate our workflow, this section provides a comprehensive top-down mapping of the experimental setup. Figure 11 shows the selected test scene monitored by two surveillance cameras within the terminal environment. We focus on camera i with known intrinsic parameters and initial pose.

Figure 12 visualizes the positioning of camera i within the 3D model reconstructed from CAD drawings, configured with a $60$ field of view and 1920 × 1080 resolution.

The coordinate transformation process follows three key steps, illustrated in the figures: World-to-Camera Transformation, which determines the camera pose in the world coordinate system. 3D Layout Extraction, extracting the local 3D layout from the camera’s spatial neighborhood. Projection-to-Image Plane: Projecting the 3D layout to the 2D image plane.

Figure 13 shows the resulting 3D layout for camera i, with structural elements representing line segments from the 3D model. Figure 14 provides an overhead view of this local 3D layout, clearly displaying spatial relationships. This integrated visualization demonstrates how our four-coordinate system functions as a unified spatial representation framework.

Data types. Our framework was implemented in C++ with CUDA acceleration for the rendering pipeline. All floating-point computations were performed using 32-bit single-precision arithmetic by default. This choice is made to achieve real-time performance when processing multiple high-resolution video streams. The use of single-precision floats strikes a balance between numerical precision and the computational efficiency required by our target application.

4.4. Analysis of the Layout Refinement Process

This section provides a detailed analysis of the proposed multi-stage layout refinement algorithm, demonstrating its effectiveness in resolving the edge inconsistencies inherent in the initial geometric projection (Figure 15).

Optimization Procedure and Convergence. The hypothesis scoring mechanism is demonstrated using a representative layout centered at the key point (1200, 370). A subset of the corresponding layout hypotheses and their evaluation scores is presented in

Table 1. The score of layout hypothesis.

s/(u,v)	1250	1200	1170	1150	1130	1110	…
390	0.4595	0.4424	0.4456	0.4547	0.4977	0.4781	…
380	0.4848	0.3688	0.3187	0.3031	0.2091	0.1960	…
375	0.5922	0.4417	0.3382	0.2313	0.1668	0.1322	…
370	0.5387	0.5388	0.4039	0.2970	0.2293	0.2032	…
365	0.5726	0.5237	0.3660	0.3497	0.3627	0.3510	…
360	0.6371	0.5315	0.3822	0.4356	0.3549	0.3855	…
350	0.5888	0.4070	0.4337	0.4422	0.5015	0.4996	…
…	…	…	…	…	…	…	…

, where lower scores indicate superior geometric consistency. The optimal layout configuration is determined efficiently via gradient descent, initialized from point S and converging robustly to a local minimum. It is noteworthy that during our extensive experiments, the optimization process demonstrated remarkable resilience to common pitfalls such as local minima and saddle points, with only rare instances requiring minimal parameter adjustments. This optimization trajectory, visualized in the 3D error space in Figure 16, illustrates the smooth and stable convergence behavior of the refinement process, validating the robustness of our approach against concerns raised regarding gradient descent efficiency.

Parameter Selection. The kernel size used in hypothesis generation presents a trade-off between refinement quality and computational cost. We evaluated this trade-off empirically, with results summarized in Figure 17. The analysis shows that performance stabilizes at a kernel size of 11 × 11. which was subsequently adopted for all experiments to balance accuracy and efficiency.

4.5. Experiment Result

The performance of the proposed method. This section presents quantitative validation of our framework’s performance.

Table 2. Computational timings of framework components.

Processing Stage	Subprocedure	Time (ms)
Real-time Update and Processing	Overall processing and rendering	<50
	Data update and GPU uploading	<20
	Texture mapping and transformation	<20
Layout Inference	Initial estimation	<10
	Optimal solution	20–30

confirms near real-time capability with sub-second latency across all critical stages: real-time processing/rendering (including GPU data upload and texture mapping) and layout inference (initial estimation and optimal refinement). Results demonstrate consistent compliance with real-time constraints (25+ FPS at 16-stream loads), validating the practical viability of dynamic scene applications.

As demonstrated in Figure 18, our method achieves robust performance across diverse experimental scenarios: Scenario 1 (conventional environments) and Scenario 2 (high dynamic range conditions). Quantitative results confirm superior layout estimation accuracy with RMSE is 10.3 ± 2.2 px, and MAE is 9.8 ± 1.8 px, while maintaining real-time layout inference processing at 46.5 FPS. This consistent high performance validates the method’s adaptability to varying environmental constraints.

A six-month deployment in an international airport using 16 synchronized cameras confirmed the system’s viability (Figure 19). This implementation yielded significant benefits: 60% hardware cost reduction through depth sensor elimination and maintenance simplification from stable camera calibration requiring no frequent recalibration.

Analysis of numerical precision and computational efficiency. To quantitatively address the impact of data types on velocity and accuracy. This section conducted a controlled experiment comparing 32-bit single-precision and 64-bit double-precision floating-point representations. The results, summarized in

Table 3. Comparison of computational performance and registration accuracy using different floating-point precisions.

Precision	RMSE (px)	MAE (px)	Average Time (ms)	Peak Memory (GB)
32-bit (Single)	10.3 ± 2.2	9.8 ± 1.8	21.5	1.2
64-bit (Double)	10.1 ± 2.1	9.6 ± 1.7	34.8	2.1

, reveal the inherent trade-off between registration accuracy and computational performance.

As evidenced by the data, employing double precision yields only a negligible improvement in registration accuracy (a reduction of only 0.2 px in RMSE). However, this comes at a significant computational cost: a $62\%$ increase in processing time per frame and a $75\%$ increase in memory consumption. The minimal gain in accuracy does not justify the substantial performance penalty for a real-time system. Therefore, the use of single-precision floating-point is justified as the optimal choice for our application, ensuring both high frame rates and sufficient geometric accuracy.

Robustness to calibration. To evaluate the framework’s dependence on calibration accuracy—a critical consideration for practical deployment—we conducted a systematic analysis under controlled parameter perturbations. This investigation serves as a proxy for assessing performance in scenarios where calibration may be suboptimal.

Our experimental approach introduced graded errors along three parameter dimensions: positional displacement (along the X, Y, and Z axes), orientation deviation (in pitch, yaw, and roll), and focal length inaccuracy. The results, summarized in

Table 4. Performance under calibrated parameter perturbations.

Perturbation Level	RMSE (px)	Success Rate	Degradation
No perturbation	10.3	98%	Baseline
+3% position, +1.5° orientation	11.9	94%	Minor
+5% position, +2° orientation	13.5	87%	Moderate
+8% position, +3° orientation	16.8	72%	Significant

, indicate that the framework maintains functional performance even under substantial parameter errors, exhibiting graceful degradation rather than catastrophic failure.

The multi-stage refinement algorithm exhibits robust tolerance to calibration inaccuracies through iterative geometric validation. This ensures practical deployment viability while maintaining the fundamental requirement for initial parameter estimates.

5. Experimental Results and Comparison

This section presents a comprehensive comparative analysis between the proposed method and a range of state-of-the-art approaches. In addition to established classical methods—including feature-based ORB-SLAM2 [14], LSD-SLAM, deep learning-based PlaneRCNN [46], and RoomNet [41], as well as the RGB-D system RTAB-Map [37]—we have expanded our evaluation to include several recently published advanced methods, such as DCL(depth consistency loss) [49], single-image reconstruction(SIR) leveraging known dimensions [22]. All comparative methods were evaluated under standardized hardware and software conditions. Each baseline underwent rigorous parameter optimization specific to our airport terminal dataset, with reported results averaged over multiple runs to ensure statistical reliability and fairness in comparison.

5.1. Quantitative Results

Registration accuracy is quantified as the pixel-level reprojection error between the projections of 3D structures from the estimated camera poses and the ground truth. Results in

Table 5. Quantitative comparison on regular layout scenes from the Airport Terminal Collection dataset, evaluated by reprojection error.

Method	RMSE (px) ↑	MAE (px) ↑	Average Time (ms) ↓	FPS ↑
ORB-SLAM2 a	/	/	/	/
LSD-SLAM a	/	/	/	/
LSD	476.50 ± 3.5	403.22 ± 3.2	∼ 20	50
PlaneRCNN	18.7 ± 3.8	14.5 ± 2.8	∼ 300	3.3
RoomNet	31.5 ± 4.2	27.2 ± 3.5	∼ 100	10.5
RTAB-Map (RGB-D)	16.5 ± 3.1	12.8 ± 2.4	∼ 38.7	25.8
DCL	14.2 ± 2.8	11.3 ± 2.1	$\sim 45.2$	22.1
SIR	19.5 ± 3.5	15.7 ± 2.9	$\sim 280$	3.6
Ours	10.3 ± 2.2	9.8 ± 1.8	∼ 21.5	46.5

^a Failed to initialize or lost tracking in most sequences; results are not reported.

derive from scenes with regular layouts. All baselines failed in challenging scenarios (e.g., complex geometries or severe reflections), precluding their inclusion in quantitative comparison.

Traditional SLAM methods, including ORB-SLAM2 and LSD-SLAM, failed completely due to the pure rotational motion in the scenes—a degenerate case for parallax-dependent depth estimation. The RGB-D method RTAB-Map established a reasonable performance baseline. Among learning-based approaches, RoomNet suffered from a significant domain gap when transitioning from cuboid rooms to large open spaces, while PlaneRCNN and DCL achieved the strongest baseline performance but degraded in environments with long corridors and large glass facades.

Our framework overcomes these limitations through architectural priors rather than data-driven assumptions, achieving a 44.9% RMSE reduction relative to the best baseline (PlaneRCNN), superior accuracy to the RGB-D baseline without depth sensors, and a new state-of-the-art in airport terminal registration.

5.2. Robustness Results

Our framework demonstrates exceptional resilience in challenging conditions where conventional methods fail. It outperforms deep learning approaches by >44.9% accuracy due to minimal sensitivity to dynamic objects. The model-driven design maintains stability in severe specular reflections—systematically overcoming depth estimation failures—and excels under dynamic lighting/textureless conditions, confirming superiority over sensor-dependent techniques.

The visualization pipeline (layout calculation, texture mapping, rendering) achieves unprecedented scalability: processing latency remains at 3 ms (330 FPS) for single streams, 17 ms (58.6 FPS) for 8 concurrent streams, and 40 ms (25.4 FPS) for 16-stream deployments (

Table 6. Performance of the Layout-Based Analysis and Rendering Stage.

#Streams	Avg. Processing Time/Frame (ms)	Throughput (FPS)
1	∼3	∼330
4	∼8	∼131.6
8	∼17	∼58.6
16	∼40	∼25.4

). This establishes new benchmarks for real-time indoor registration, enabling high-density surveillance with sub-40ms latency.

Our geometry-driven approach exhibits exceptional cross-domain generalization, exhibiting lower performance degradation when deployed in unseen environments such as shopping malls, offices, and metro stations after airport-specific training. This contrasts sharply with deep learning counterparts that suffer $>44.9\%$ accuracy drops due to domain shift. The key advantage lies in intrinsic immunity to appearance overfitting—enabling immediate deployment in new managed spaces with architectural models without retraining.

5.3. Visualized Results

To provide a comprehensive visual comparison, we selectively juxtapose our method against both a state-of-the-art deep learning approach and traditional methods across diverse scenarios. As shown in Figure 20, this comparative analysis spans normal enclosed spaces (scene 1, 3), high-dynamic-range environments (scene 2), and scenes with prominent reflective surfaces (scene 4). The results reveal fundamental limitations of existing approaches: traditional LSD-based methods with vanishing point estimation fail in open-plan corridors due to architectural constraints, while the modern deep learning pipeline (combining MSEG segmentation [50], DCL [49], and GeoLayout aggregation [51]) exhibits significant degradation under highlight and reflective conditions. In contrast, our method consistently maintains superior visual accuracy and geometric coherence across all challenging scenarios, demonstrating its unparalleled adaptability to complex indoor environments.

The accuracy of layout estimation critically determines 3D visualization quality. As evidenced in Figure 20, Figure 21 and Figure 22, traditional and deep learning methods produce severe artifacts due to geometric estimation errors, resulting in distorted registrations. In contrast, our framework maintains spatial coherence and sub-pixel accuracy across all tested conditions.

6. Ablation Experiment

To validate the contribution of each component in our framework, we conducted a series of ablation studies and evaluated them both qualitatively and quantitatively. The results clearly demonstrate the incremental improvements brought by our core innovations.

The baseline configuration employs only the Geometric Projection (GP) component, which directly projects the 3D model onto the image plane using calibrated camera parameters. While computationally efficient, this approach suffers from noticeable misalignment due to calibration inaccuracies and scene dynamics.

We then introduce the Coordinate Framework (CF), which establishes explicit geometric linkages across the four coordinate systems (world, model, camera, and image). This provides a more robust spatial reasoning foundation, reducing error propagation.

The complete system integrates Multi-stage Refinement (MF), which iteratively optimizes the layout through hypothesis generation and gradient descent-based selection. This component specifically addresses the residual errors that remain after coordinate transformation.

The quantitative impact of each component is summarized in

Table 7. Ablation study of framework components. GP: geometric projection, CF: coordinate framework, MF: multi-stage refinement. Score denotes the geometric consistency loss, with lower values being better.

Configuration	Score ↓	Time (ms) ↓	Improvement
GP	0.451	1.4	Baseline
+ CF	0.362	4.2	+19.7%
+ CF + MF (Ours)	0.184	21.5	+59.2%

. The introduction of CF brings a significant 19.7% accuracy improvement over the GP baseline. The subsequent addition of MF yields a further 59.2% gain, effectively resolving most spatial inconsistencies. This progressive integration results in an overall accuracy improvement of 78.9% compared to the baseline, while maintaining real-time performance.

7. Discussion and Limitations

Despite its strong performance, our framework has several limitations that define its current scope of applicability. The primary constraint is its reliance on known camera installation parameters, which restricts its use to managed environments where such information is available. While the method demonstrates robustness, its performance degrades under extreme changed conditions, such as significant rotational displacement of the camera between successive frames, leading to registration failures. Furthermore, the framework is dependent on the availability of 3D architectural models, although these are becoming increasingly standard in modern building information management (BIM) systems [52,53]. Consequently, our method is best suited for applications in managed indoor environments such as airports, shopping centers, and office buildings, particularly for surveillance, monitoring, and infrastructure-aware augmented reality or navigation systems.

8. Conclusions

This paper introduces a novel coordinate-system-based framework for real-time 3D registration in dynamic, complex indoor environments. By establishing a rigorous geometric linkage across four coordinate systems and leveraging the often-overlooked prior of known camera poses, our method achieves a significant 44.9% accuracy improvement over state-of-the-art methods while completely eliminating the need for depth sensors. The proposed solution integrates multi-stage geometric refinement to ensure robustness against calibration errors and environmental dynamics, delivers real-time performance at 25 FPS across 16 concurrent video streams, and has been validated through a six-month deployment in a live airport terminal. While the current implementation requires pre-surveyed camera parameters and a 3D model, it presents a highly practical and robust solution for large-scale surveillance and augmented reality in managed indoor spaces. Future work will focus on relaxing these prerequisites by exploring autonomous camera calibration and improving resilience to more extreme motion dynamics.