Occluder-Mask-Constrained 3D Reconstruction from Tower-Crane Construction Site Imagery

Abstract

3D reconstruction of construction scenes is an important enabling technology for digital and intelligent construction project management. Recurring foreground occluders and dynamic disturbances in tower-crane imagery can destabilize image registration and introduce spurious depth responses. This paper proposes an occluder-mask-constrained 3D reconstruction framework driven by multi-view geometric anomalies. Adjacent-view geometric outliers are spatially aggregated to generate foreground prompt points, which are converted into occluder masks using Segment Anything Model 2 (SAM2). The masks are propagated as unified pixel-validity constraints through sparse feature filtering, Adaptive Patch Deformation Multi-View Stereo (APD-MVS) matching-cost evaluation, support-region selection, and depth-map fusion. Experiments on three real construction-site datasets show increased sparse-registration completeness in the tested sequences and fewer visually identifiable occluder-induced artifacts in dense point clouds. A representative 308-image sequence was further evaluated against no-mask reconstruction, You Only Look Once version 8 (YOLOv8) bounding-box removal, manually prompted Segment Anything Model 2.1 (SAM2.1), a Segment Anything Model 3 (SAM3) text-prompt baseline, and Visibility-Aware Multi-View Stereo Network (Vis-MVSNet). The evaluation combines sparse-reconstruction metrics, pixel-level mask-quality metrics from a manually annotated validation subset, module-wise runtime accounting, controlled ablations, and aligned dense-point-cloud visualization. These results show improved sparse-stage registration completeness and visible artifact suppression. Because high-precision 3D reference point clouds are unavailable, the dense results are interpreted as visual evidence of artifact suppression rather than as proof of improved absolute dense-reconstruction accuracy.

1. Introduction

Accurate 3D reconstruction of construction sites provides geometric support for progress monitoring, quantity estimation, and safety-hazard inspection. Tower-crane cameras and surveillance devices have made continuous construction-site imagery increasingly available, enabling low-cost repeated image acquisition for reconstructing active construction areas [1,2,3,4]. Figure 1 shows the tower-crane-mounted image acquisition configuration considered in this study. Unlike controlled benchmark imagery, these image sequences are affected by changing site layouts, uneven texture distribution, local motion, and foreground occlusion.

Crane hooks and cables are a recurring source of reconstruction errors in such data. They are not part of the static construction scene, but their image features may still enter the reconstruction pipeline. Once retained as valid observations, these occluders can introduce false correspondences, degrade camera registration, and generate erroneous depth responses. During depth-map fusion, these local errors may appear as floating outliers or scattered spurious points in the final dense point cloud. The problem is therefore not limited to foreground segmentation; it is a visibility-related reconstruction failure that affects both sparse and dense stages.

Occlusion and visibility reasoning have been studied in multi-view reconstruction. Early dense Multi-View Stereo (MVS) methods excluded or down-weighted occluded pixels to reduce invalid photometric support [5]. Other methods introduced occluding contours or visibility relations to improve geometric consistency across views [6]. Occlusion-aware masking has also been used in mesh refinement to limit photometric error computation in occluded regions [7]. More recent learning-based MVS methods incorporate visibility modeling into cost aggregation and depth inference [8]. These studies confirm the role of visibility reasoning in robust reconstruction. However, most of them either target general scenes or operate at a limited stage of the reconstruction pipeline. They do not directly provide an automatic way to extract crane-hook occluders from tower-crane imagery and propagate the resulting visibility constraints through both Structure from Motion (SfM) and dense MVS. This setting motivates a reconstruction-aware prompt source and a visibility constraint shared by SfM and dense MVS.

Promptable segmentation offers a possible route for extracting irregular occluders. The Segment Anything Model (SAM) introduced point-, box-, and mask-guided segmentation with strong generalization ability [9]. Segment Anything Model 2 (SAM2) extends this promptable segmentation paradigm to images and videos, which is relevant to temporally collected construction-site imagery [10]. Meanwhile, segmentation-guided MVS methods, such as Segmentation-Driven Deformation Multi-View Stereo (SD-MVS) and Multi-Granularity Segmentation Prior Guided Multi-View Stereo (MSP-MVS), use segmentation priors to guide patch deformation, boundary handling, and multi-granularity matching [11,12]. These studies show that segmentation can regularize dense matching. However, segmentation masks alone do not determine which image regions are harmful to reconstruction. For tower-crane imagery, the missing component is a prompt source that is coupled with geometric failure, rather than only with visual appearance or semantic category.

This paper introduces an occluder-mask-constrained 3D reconstruction framework for tower-crane construction-site imagery. The framework detects adjacent-view geometric anomalies, converts anomaly clusters into prompt points, extracts occluder masks with SAM2, and uses the masks as visibility constraints in reconstruction. The masks are applied before sparse reconstruction to reject occluder-region features. They are also incorporated into Adaptive Patch Deformation Multi-View Stereo (APD-MVS) [13] through matching-cost computation, support-region selection, and depth-map fusion. In this formulation, the mask serves as a reconstruction-validity map: it selects feature observations for SfM and source-view projections for APD-MVS matching and fusion.

The main contributions are summarized as follows:

1.

A geometry-driven prompt generation strategy is proposed for tower-crane imagery. It generates occluder prompt points from adjacent-view geometric anomalies, without requiring additional object detection training data.

2.

An occluder-mask-constrained reconstruction framework is developed. The extracted masks are introduced into sparse feature filtering and APD-MVS-based dense reconstruction, including matching-cost evaluation, valid-anchor selection, and depth-map fusion.

3.

The method is evaluated on three real construction-site datasets. The experiments show improved sparse registration completeness, reduced SfM running time in most cases, and fewer visible spurious points caused by crane hooks in dense point clouds.

2. Related Work

2.1. Image-Based 3D Reconstruction for Construction Sites

Image-based 3D reconstruction provides a practical route for obtaining 3D representations of construction scenes from ordinary image sequences. Compared with laser scanning or other dedicated surveying systems, images acquired by tower-crane cameras, fixed site cameras, mobile devices, or unmanned aerial vehicles can be collected repeatedly with relatively low deployment cost. For this reason, image-based reconstruction is suitable for generating visual and geometric records of active construction areas, especially when continuous or repeated data acquisition is already available on site. Zheng et al. reviewed the key components of vision-based 3D reconstruction, including image acquisition, feature matching, camera pose estimation, and dense reconstruction [1]. For tower-crane image sequences, Lu et al. investigated image selection and 3D reconstruction by combining image hash features and motion trajectory constraints, showing that tower-crane imagery can be used as reconstruction input rather than only as visual monitoring records [2]. Xue et al. reviewed image-based building reconstruction in construction applications, and Sun et al. investigated near-real-time construction-site reconstruction from surveillance-camera imagery [3,4]. Recent learning-based SfM–MVS studies have introduced deep feature matching, depth estimation, attention mechanisms, and neural volume rendering to improve reconstruction robustness under challenging matching conditions [14,15]. However, these works mainly enhance the reconstruction pipeline itself and do not explicitly address foreground occluders such as crane hooks and cables in tower-crane imagery.

From a technical perspective, most image-based reconstruction pipelines follow a sparse-to-dense process. SfM first estimates camera poses and sparse 3D points from cross-view feature correspondences, geometric verification, triangulation, and bundle adjustment (BA) [16]. MVS then estimates dense depth maps or dense point clouds from registered images by evaluating cross-view photometric consistency [17]. Classical MVS methods, including Patch-based Multi-View Stereo (PMVS), Gipuma, and COLMAP-style MVS, rely on local photometric consistency, view selection, propagation, filtering, and geometric consistency to recover dense geometry [17,18,19]. More recent methods further improve dense matching in weakly textured or ambiguous regions. APD-MVS, for example, adaptively deforms local patches and expands the support region around unreliable pixels to improve textureless-resilient MVS [13]. Neural rendering and Gaussian-splatting-based reconstruction have also introduced new representations for scene reconstruction. For example, Stereo Gaussian Splatting (Stereo-GS) performs online 3D Gaussian Splatting reconstruction from stereo image streams, and planar-guided Gaussian Splatting incorporates geometric priors and SAM-family masks to improve reconstruction in texture-poor regions [20,21]. Although these methods improve reconstruction quality from the perspectives of matching, representation, or geometric regularization, most of them assume that the input observations mainly belong to the target scene. In tower-crane construction imagery, this assumption is often violated because crane hooks and cables may appear between the camera and the construction area. If such foreground occluder observations are retained, they may enter feature matching and camera registration in the sparse stage and may further produce spurious points during dense matching and fusion. Therefore, construction-site reconstruction from tower-crane imagery requires not only robust SfM–MVS algorithms but also a mechanism for identifying and excluding foreground occluder observations before and during reconstruction.

2.2. Visual Object Extraction

Visual object extraction aims to obtain the spatial extent of target objects from images, and its outputs may include bounding boxes, instance masks, semantic masks, or object-level pixel regions. Supervised object detection and segmentation methods have provided effective solutions for predefined visual categories. For example, Mask2Former unifies semantic segmentation, instance segmentation, and panoptic segmentation within a masked-attention mask-transformer architecture [22]. Detection Transformer (DETR) with Improved DeNoising Anchor Boxes (DINO) improves DETR-style end-to-end object detection through denoising training and improved anchor-box design [23]. You Only Look Once version 10 (YOLOv10) further develops real-time end-to-end object detection by optimizing the accuracy–efficiency trade-off of the YOLO series [24]. These methods can achieve strong performance when sufficient annotated data are available for the target categories. However, this condition is difficult to satisfy for occluder extraction from construction-site images. The foreground occluders in tower-crane imagery may include crane hooks, cables, lifting components, temporary structures, and other irregular objects, and no dedicated annotated dataset is available for these diverse occluder categories. Therefore, training a closed-set supervised detector or segmenter would require additional annotation work and may still generalize poorly across different sites, camera viewpoints, illumination conditions, occluder appearances, and image qualities.

When task-specific labeled data are unavailable, open-set, unsupervised, and promptable methods provide a more flexible route for visual object extraction. Cut and Learn (CutLER) uses self-supervised features to generate coarse object masks and trains a localization model, enabling object detection and instance segmentation without human annotations [25]. Grounding DINO introduces language-guided open-set object detection and can localize objects specified by category names or referring expressions [26]. X-Decoder unifies pixel-level segmentation and language-token prediction in a generalized decoding framework, supporting open-vocabulary segmentation and referring segmentation [27]. Nevertheless, for construction-site occluder extraction, category-level semantics alone may be unreliable because occluders vary greatly in type and appearance, and construction-site images are often affected by low resolution, motion blur, poor illumination, and background clutter. Prompt-guided mask extraction is therefore a more suitable route. Segment Everything Everywhere All at Once (SEEM) develops a promptable and interactive segmentation interface that supports diverse prompts, such as points, boxes, scribbles, masks, and text [28]. In the promptable segmentation line, the SAM formulates segmentation as a prompt-driven task and generates masks from points, boxes, or masks with strong zero-shot transfer ability [9]. High-Quality Segment Anything Model (HQ-SAM) improves the mask quality of the SAM for objects with complex structures while preserving its promptable design and zero-shot generalization [29]. SAM2 further extends promptable visual segmentation from images to videos by introducing a streaming-memory design for real-time video processing [10]. However, manually specifying visual prompts is labor-intensive and impractical for large image sequences. Consequently, reliable automatic prompt generation is essential for obtaining stable occluder masks in complex construction-site imagery.

2.3. Occlusion Handling and Reconstruction Constraints

Existing occlusion-handling approaches address visibility reasoning from complementary perspectives. Visibility-aware MVS methods exclude, down-weight, or explicitly model source-view observations that are unlikely to support the same surface point [5,6,8]. Occlusion-aware masking restricts photometric computation during mesh refinement [7]. Segmentation-guided MVS methods introduce semantic or instance priors into dense matching, patch deformation, or boundary handling [11,12]. Dynamic-scene mapping provides a related line of research: dynamic simultaneous localization and mapping (DynaSLAM) combines multi-view geometry and learned detection for dynamic-object suppression and background inpainting, while MaskFusion uses instance-level semantic masks to recognize, track, and reconstruct independently moving red–green–blue-depth (RGB-D) objects [30,31]. Neural-rendering methods address transient foreground content through alternative scene representations; for example, Neural Radiance Fields in the Wild (NeRF-W) introduces a transient component for unconstrained photo collections, and Gaussian-splatting methods incorporate geometric or segmentation priors into explicit radiance-based representations [20,21,32]. At the pipeline level, visibility cues can therefore originate from photometric consistency, geometric occlusion, motion cues, semantic priors, or learned source-view reliability, and they may be introduced at different stages of scene reconstruction.

Recurring crane-hook occlusion affects both sparse and dense reconstruction. The proposed framework converts spatially concentrated adjacent-view geometric anomalies into segmentation prompts and propagates the resulting occluder-validity map through the sparse-to-dense pipeline.

3. Methodology

3.1. Overview of the Proposed Framework

Figure 2 shows the overall pipeline of the proposed occluder-mask-constrained 3D reconstruction framework. The framework consists of four main modules: geometric-anomaly-based prompt generation, SAM2-based occluder mask extraction, occluder-mask-assisted sparse reconstruction, and occluder-mask-constrained dense reconstruction.

Adjacent-view feature matching and robust epipolar verification identify geometric anomalies relative to the dominant construction-scene geometry. Their spatially aggregated clusters provide point prompts for SAM2-based occluder segmentation.

The resulting pixel-validity maps are shared by the sparse and dense stages: they filter SfM observations and constrain APD-MVS matching-cost computation, support-region selection, and depth-map fusion. The workflow outputs a dense point cloud for experimental evaluation.

3.2. Geometric Anomaly Information Acquisition

The prompt generation module identifies image regions that violate the dominant adjacent-view geometry. In neighboring tower-crane frames, most static construction-scene points follow a stable epipolar relation. By contrast, crane hooks and cables may produce inconsistent matches because of local occlusion, relative motion, or strong depth discontinuity. The proposed method therefore uses the spatial aggregation of these inconsistent matches to generate point prompts for occluder segmentation.

3.2.1. Adjacent-View Geometry and Geometric Outlier Detection

Let two adjacent images be denoted as $I_i$ and $I_j$. After feature extraction and matching, the initial correspondence set is defined as

$${\mathcal{Q}}_{ij}={\left\{\left({\mathbf{x}}_{i,k},{\mathbf{x}}_{j,k},{\alpha }_k\right)\right\}}_{k=1}^N,$$

(1)

where ${\mathbf{x}}_{i,k}$ and ${\mathbf{x}}_{j,k}$ are the homogeneous coordinates of the k-th matched feature in $I_i$ and $I_j$, respectively. The term ${\alpha }_k$ denotes the LightGlue matching confidence after match filtering [33]. The score is clipped to $[0,1]$ before anomaly weighting, so the confidence contribution remains bounded across adjacent-image pairs.

For a correspondence that follows the dominant adjacent-view geometry, the epipolar constraint is

$${\mathbf{x}}_{j,k}^{\top }{\mathbf{F}}_{ij}{\mathbf{x}}_{i,k}=0,$$

(2)

where ${\mathbf{F}}_{ij}$ is the fundamental matrix between $I_i$ and $I_j$.

The initial correspondences may contain ordinary mismatches and occluder-related matches. A robust estimator is therefore used to recover the epipolar model mainly supported by the construction-scene region. This study adopts Universal Sample Consensus (USAC) with Marginalizing Sample Consensus (MAGSAC), implemented as USAC_MAGSAC, for fundamental matrix estimation [34,35]. The estimated model is expressed as

$${\mathbf{F}}_{ij}^{*}=arg\underset{{\mathbf{F}}_{ij}}{max}{Q}_M\left({\mathbf{F}}_{ij};{\mathcal{Q}}_{ij}\right),$$

(3)

where $Q_M(·)$ denotes the robust model score used by the MAGSAC framework.

Based on ${\mathbf{F}}_{ij}^{*}$, the initial correspondences are divided into an inlier set ${\mathcal{I}}_{ij}$ and an outlier set ${\mathcal{O}}_{ij}$:

$${\mathcal{I}}_{ij}\cup {\mathcal{O}}_{ij}={\mathcal{Q}}_{ij},{\mathcal{I}}_{ij}\cap {\mathcal{O}}_{ij}=⌀.$$

(4)

The inliers mainly support the dominant construction-scene geometry. The outlier set may include ordinary mismatches, weak-texture matches, and local image noise in addition to occluder-related observations. Spatial concentration is therefore used to identify candidate crane-hook or cable regions, while isolated outliers receive limited influence.

3.2.2. Geometric Anomaly Weighting and Prompt Point Generation

For each outlier correspondence, the geometric deviation from the estimated dominant geometry is measured using the symmetric epipolar distance:

$$e_k=\sqrt{{\displaystyle \frac{{\left({\mathbf{x}}_{j,k}^{\top }{\mathbf{F}}_{ij}^{*}{\mathbf{x}}_{i,k}\right)}^2}{{\left({\mathbf{F}}_{ij}^{*}{\mathbf{x}}_{i,k}\right)}_1^2+{\left({\mathbf{F}}_{ij}^{*}{\mathbf{x}}_{i,k}\right)}_2^2}}+{\displaystyle \frac{{\left({\mathbf{x}}_{j,k}^{\top }{\mathbf{F}}_{ij}^{*}{\mathbf{x}}_{i,k}\right)}^2}{{\left({\left({\mathbf{F}}_{ij}^{*}\right)}^{\top }{\mathbf{x}}_{j,k}\right)}_1^2+{\left({\left({\mathbf{F}}_{ij}^{*}\right)}^{\top }{\mathbf{x}}_{j,k}\right)}_2^2}}}.$$

(5)

The outlier weight is defined by combining the matching confidence and the truncated geometric deviation:

$$w_k=(1-{\alpha }_k)+min\left({\displaystyle \frac{e_k}{\eta }},1\right),$$

(6)

where $\eta$ is the normalization and truncation threshold for the geometric deviation. For the reported implementation, $\eta$ is set to the 8-pixel geometric threshold used during robust epipolar verification. Since ${\alpha }_k$ is bounded in $[0,1]$, a lower-confidence correspondence with a larger deviation obtains a higher weight. The truncation term avoids excessive influence from a small number of extremely large deviations.

The outlier locations are projected onto the image plane of the reference image and aggregated using regular grid cells. For the g-th grid cell, let ${\mathcal{O}}_g$ be the set of outlier correspondences whose reference-view points fall inside this cell. The anomaly score of the cell is

$$S_g=\sum _{k\in {\mathcal{O}}_g}{w}_k.$$

(7)

Cells with larger $S_g$ contain stronger and more concentrated geometric anomalies. Candidate cells are ranked according to their anomaly scores and outlier distributions. For each selected non-empty cell, the prompt point is computed as the weighted centroid of the outlier locations:

$${\mathbf{p}}_g={\displaystyle \frac{{\sum }_{k\in {\mathcal{O}}_g}{w}_k{\mathbf{u}}_k}{{\sum }_{k\in {\mathcal{O}}_g}{w}_k}},$$

(8)

where ${\mathbf{u}}_k$ is the two-dimensional coordinate of the k-th outlier in the reference image.

Grid-based centroid estimation suppresses isolated mismatches and moves the prompt toward the dominant anomaly cluster. The generated prompt points are then used for SAM2-based occluder mask extraction.

3.2.3. Prompt-Guided Mask Extraction and Validity Map Definition

The geometric prompt points provide sparse location cues for potential occluder regions. SAM2 is used as a prompt-guided segmentation module to convert these points into pixel-level occluder masks [10]. The Segment Anything Model 2.1 (SAM2.1) predictor configuration used in each experiment is reported in Section 4.2. For each image, the prompt points generated from adjacent-view geometric anomalies are fed into SAM2 as positive foreground prompts, and SAM2 predicts a binary mask aligned with the original image. Before validity-map construction, the predicted masks are combined with a sequence-level common-border mask and screened using simple abnormal-response rules. The common-border mask is obtained by intersecting boundary-connected border candidates across the image sequence. All-black masks and masks dominated by top-left or edge-highlighted regions are rejected. This post-processing removes camera-frame borders and abnormal segmentation responses from the reconstruction-valid image regions. The benchmark-specific border-candidate threshold and sequence length are reported in Section 4.2 and Appendix A (

Table A1. Key implementation parameters for the representative 308-image benchmark.

Stage	Module	Parameter	Value
Input	Images	Number/resolution	308 images; $1882\times 1056$ pixels
Geometric anomaly	Feature matching	Model/device	ETH-CVG LightGlue-SuperPoint [33,36]; CUDA
Geometric anomaly	Match filtering	Match-confidence threshold/minimum accepted score	Match-confidence threshold = 0.85; minimum accepted score = 0.30
Geometric anomaly	Confidence weighting	Confidence-score range	Clip LightGlue confidence to $[0,1]$ before weighting
Geometric anomaly	Temporal pairing	Window parameter/traversal stride/evaluated pairs	2 frames/1 frame/307 pairs
Geometric anomaly	Robust estimation	Estimator/RANSAC threshold	MAGSAC/8 px
Geometric anomaly	Deviation truncation	$\eta$	8 px; normalized geometric term capped at 1
Geometric anomaly	Geometric filtering	Minimum inlier ratio	0.20
Prompt generation	Grid aggregation	Grid size/minimum votes/minimum vote ratio	Grid size = 32 px; minimum votes = 3; minimum vote ratio = 0.08
Prompt generation	Retention strategy	Maximum retained cells/maximum prompts per image	Maximum retained cells = 5; maximum prompts per image = 5
SAM2	Automatically generated masks	Checkpoint/configuration	SAM2.1-Hiera-Large; sam2.1_hiera_l.yaml; image predictor
SAM2	Manually prompted reference	Checkpoint/configuration	SAM2.1-Hiera-Large; sam2.1_hiera_l.yaml; video-predictor propagation with interactive prompts
SAM2	Prompt label	Point-label semantics	Positive foreground prompts for occluders
Mask post-processing	Common border mask	Intensity threshold/temporal aggregation/connectivity	Intensity threshold: $max(R,G,B)\le 10$; temporal aggregation: intersection of 308 frames; retain boundary-connected regions only
Mask post-processing	Degenerate-mask screening	Rejected response types/merge	Reject all-black outputs and edge-dominated responses; merge accepted masks with the common border mask
COLMAP SfM	Camera	Model/intrinsics	PINHOLE; $f_x=1574.0475$, $f_y=1573.8264$, $c_x=941$, $c_y=528$; single-camera mode enabled
COLMAP SfM	SIFT	Maximum image size/feature cap/execution mode	max_image_size = 3200; max_num_features = 8192; GPU acceleration enabled
COLMAP SfM	Sequential matching	Overlap/quadratic overlap/loop detection	Overlap = 10; quadratic overlap enabled; loop detection disabled; GPU acceleration enabled
COLMAP SfM	Mapper	Bundle-adjustment refinement/minimum triangulation angle	Refine focal length: enabled; refine principal point: disabled; refine extra parameters: enabled; minimum triangulation angle = ${16}^{\circ }$
COLMAP SfM	Database protocol	Initialization/mask input	Separate empty database for each masking condition; rerun feature extraction and matching; masked branch adds ImageReader.camera_mask_path
APD-MVS	Source views	Total views/selected sources/candidate-view cap	5 views per reference image (1 reference + 4 sources); select the top 4 sources from at most 32 candidates
APD-MVS	PatchMatch	Maximum iterations/bilateral NCC parameters	max_iterations = 3; sigma_spatial = 5; sigma_color = 3
APD-MVS	Multi-scale	Number of rounds/scale schedule	Two rounds; scales = 2, 1; one initialization and three refinements per round
APD-MVS	Adaptive patch deformation	Activation/initialization radius/refinement radii	Activation: second round only; initialization radius = 6; refinement radii = 4, 2, 2
APD-MVS	Texture-adaptive support	Support radius/radius increment	Strong-texture pixels: radius = 5, increment = 2; weak-texture pixels: radius = 5, increment = 5
APD-MVS	RANSAC	Threshold/rotation trials	Threshold = 0.00875 in the second round; rotation trials = 2
APD-MVS	Depth range	Lower/upper scale factor	$0.6\times$/$1.2\times$ the camera-derived range
Depth fusion	Geometric checks	Reprojection error/relative depth difference/normal-angle difference	$<2$ px/$<0.01$/$<{10}^{\circ }$
Depth fusion	Dynamic consistency	Minimum consistent source views/confidence thresholds	At least one consistent source view; strong threshold = 0.30; weak threshold = 0.45

).

The prompts are derived from adjacent-view geometric inconsistency, so prompt selection is directly linked to regions that disturb multi-view reconstruction.

The segmentation result is converted into a binary validity map for subsequent reconstruction constraints. For image $I_i$, the validity map is defined as

$$M_i\left(\mathbf{x}\right)=\left\{\begin{array}{cc}1,\hfill & \mathbf{x}\in {\Omega }_i^{\mathrm{scene}},\hfill \\ 0,\hfill & \mathbf{x}\in {\Omega }_i^{\mathrm{occ}},\hfill \end{array}\right.$$

(9)

where $\mathbf{x}$ is a pixel location, ${\Omega }_i^{\mathrm{scene}}$ denotes the valid construction-scene region, and ${\Omega }_i^{\mathrm{occ}}$ denotes the occluder region. Pixels with $M_i\left(\mathbf{x}\right)=1$ are termed mask-valid, whereas pixels with $M_i\left(\mathbf{x}\right)=0$ are termed mask-invalid. The same terminology is used below for observations, projections, anchors, and candidate 3D points according to their associated pixel locations.

For a reference-view pixel $\mathbf{x}$ in image $I_i$ and its corresponding source-view location ${\mathbf{x}}^{\prime }$ in image $I_j$, the mask-based visibility indicator is defined as

$$v_{ij}(\mathbf{x},{\mathbf{x}}^{\prime })=M_i\left(\mathbf{x}\right)M_j\left({\mathbf{x}}^{\prime }\right).$$

(10)

The indicator $v_{ij}(\mathbf{x},{\mathbf{x}}^{\prime })$ is used in sparse correspondence filtering, dense matching, and depth-map fusion to enforce cross-view mask consistency.

3.3. Occluder-Mask-Constrained Sparse Reconstruction

The occluder masks are first applied before sparse reconstruction to filter feature observations before correspondence construction, geometric verification, and BA. The SfM stage retains the standard reprojection-error minimization framework [16]. The validity map determines the feature observations supplied to correspondence construction, geometric verification, triangulation, and BA.

3.3.1. Mask-Based Feature and Correspondence Filtering

For a detected feature point at pixel $\mathbf{x}$ in image $I_i$, the feature is retained only if

$$M_i\left(\mathbf{x}\right)=1.$$

(11)

For a candidate correspondence between image $I_i$ and image $I_j$, let $\mathbf{x}$ and ${\mathbf{x}}^{\prime }$ be the matched pixel locations. The correspondence is accepted only when

$$v_{ij}(\mathbf{x},{\mathbf{x}}^{\prime })=M_i\left(\mathbf{x}\right)M_j\left({\mathbf{x}}^{\prime }\right)=1.$$

(12)

The accepted correspondences are then used for geometric verification, image registration, and sparse point triangulation.

3.3.2. Mask-Constrained SfM Objective

After mask-based filtering, the observations entering BA are represented by the following mask indicator. For the observation of 3D point ${\mathbf{X}}_{\ell }$ in image $I_i$,

$$m_{i\ell }=M_i\left({\mathbf{x}}_{i\ell }\right),m_{i\ell }\in \{0,1\},$$

(13)

where ${\mathbf{x}}_{i\ell }$ is the corresponding image observation.

The resulting SfM objective is written as

$$\underset{\left\{{\Theta }_i\right\},\left\{{\mathbf{X}}_{\ell }\right\}}{min}\sum _{(i,\ell )\in \mathcal{V}}{m}_{i\ell }{∥{\mathbf{x}}_{i\ell }-\pi ({\Theta }_i,{\mathbf{X}}_{\ell })∥}_2^2,$$

(14)

where ${\Theta }_i$ denotes the camera parameters of image $I_i$, ${\mathbf{X}}_{\ell }$ denotes the ℓ-th 3D point, $\pi (·)$ is the projection function, and $\mathcal{V}$ is the observation set.

Equation (14) expresses BA over the mask-valid observation set and makes the observation-selection mechanism explicit.

3.4. Occluder-Mask-Constrained Dense Reconstruction

After sparse reconstruction, the same occluder masks are introduced into APD-MVS-based dense reconstruction. The constraints act on matching-cost computation, deformable PatchMatch support selection, and depth-map fusion. They restrict which pixels, projections, anchors, and candidate 3D points can support dense reconstruction.

3.4.1. APD-MVS Plane Hypothesis and Mask-Constrained Matching Cost

APD-MVS is a PatchMatch-based MVS method that uses adaptive patch deformation to improve depth estimation in difficult regions [13]. For a reference-view pixel $\mathbf{x}$, the local plane hypothesis is represented as

$${\mathbf{f}}_{\mathbf{x}}={\left[\begin{array}{cc}{\mathbf{n}}_{\mathbf{x}}^{\top }& d_{\mathbf{x}}\end{array}\right]}^{\top },$$

(15)

where ${\mathbf{n}}_{\mathbf{x}}$ and $d_{\mathbf{x}}$ denote the local plane normal and depth parameter, respectively.

Given a plane hypothesis, the reference patch centered at $\mathbf{x}$ is projected to source views. Let ${\mathbf{B}}_{\mathbf{x}}$ be the reference patch centered at $\mathbf{x}$, and let ${\mathbf{B}}_{\mathbf{x}}^j$ be the corresponding patch projected to source view j under the plane hypothesis ${\mathbf{f}}_{\mathbf{x}}$. The original single-view matching cost is defined by Normalized Cross-Correlation (NCC):

$$c_j\left(\mathbf{x},{\mathbf{f}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{x}}\right)=1-{\displaystyle \frac{cov\left({\mathbf{B}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{x}}^j\right)}{\sqrt{cov\left({\mathbf{B}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{x}}\right)cov\left({\mathbf{B}}_{\mathbf{x}}^j,{\mathbf{B}}_{\mathbf{x}}^j\right)}}}.$$

(16)

Let ${\mathbf{x}}_j^{\prime }\left({\mathbf{f}}_{\mathbf{x}}\right)$ be the projection of the reference pixel $\mathbf{x}$ into source view j under the current plane hypothesis. The mask-constrained single-view cost is defined as

$${\tilde{c}}_j\left(\mathbf{x},{\mathbf{f}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{x}}\right)=\left\{\begin{array}{cc}{c}_j\left(\mathbf{x},{\mathbf{f}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{x}}\right),\hfill & v_{ij}\left(\mathbf{x},{\mathbf{x}}_j^{\prime }\right)=1,\hfill \\ +\infty ,\hfill & v_{ij}\left(\mathbf{x},{\mathbf{x}}_j^{\prime }\right)=0.\hfill \end{array}\right.$$

(17)

In implementation, source-view observations with $v_{ij}=0$ are excluded from cost aggregation. If no valid source-view observation remains for a plane hypothesis, the hypothesis is rejected.

Let ${\tilde{c}}^{*}(\mathbf{x},{\mathbf{f}}_{\mathbf{x}})$ denote the aggregated mask-constrained matching cost over valid source views. The plane hypothesis search is written as

$${\mathbf{f}}_{\mathbf{x}}^{*}=arg\underset{{\mathbf{f}}_{\mathbf{x}}}{min}{\tilde{c}}^{*}\left(\mathbf{x},{\mathbf{f}}_{\mathbf{x}}\right).$$

(18)

3.4.2. Mask-Constrained Support Region in Deformable PatchMatch

For unreliable pixels, APD-MVS uses deformable PatchMatch to evaluate a plane hypothesis with additional anchor pixels. To avoid support-region propagation from occluder pixels, the valid anchor set is defined as

$${\mathcal{A}}_{\mathbf{x}}^{\mathrm{valid}}=\left\{\mathbf{a}\in {\mathcal{A}}_{\mathbf{x}}\mid M_i\left(\mathbf{a}\right)=1\right\}.$$

(19)

The corresponding deformable cost is computed over this valid anchor set:

$${\tilde{c}}_D\left(\mathbf{x},{\mathbf{f}}_{\mathbf{x}},{\mathcal{A}}_{\mathbf{x}}^{\mathrm{valid}}\right)=\lambda \tilde{c}\left(\mathbf{x},{\mathbf{f}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{x}}\right)+(1-\lambda ){\displaystyle \frac{1}{\left|{\mathcal{A}}_{\mathbf{x}}^{\mathrm{valid}}\right|}}\sum _{\mathbf{a}\in {\mathcal{A}}_{\mathbf{x}}^{\mathrm{valid}}}\tilde{c}\left(\mathbf{a},{\mathbf{f}}_{\mathbf{x}},{\mathbf{B}}_{\mathbf{a}}\right),$$

(20)

where $\lambda$ controls the balance between the center-pixel term and the anchor-based support term. The term $\tilde{c}(·)$ denotes the mask-constrained patch matching cost used in the APD-MVS support-region formulation. In implementation, if no valid anchor remains, the support term is skipped and only the mask-constrained center-pixel term is used.

The mask constraint therefore acts on both center-pixel evaluation and support-region construction.

3.4.3. Mask-Consistent Depth-Map Fusion

Finally, mask consistency is imposed during depth-map fusion. After depth maps are estimated, let pixel $\mathbf{x}$ in reference view $I_i$ and its depth $d_i\left(\mathbf{x}\right)$ be back-projected to a 3D point $\mathbf{X}$. Let ${\mathbf{x}}^{\prime }$ be the projection of $\mathbf{X}$ in source view $I_j$. For this source view, the candidate point is allowed to enter the final point cloud only when the mask consistency condition is satisfied:

$$v_{ij}\left(\mathbf{x},{\mathbf{x}}^{\prime }\right)=1.$$

(21)

The candidate point must also satisfy the geometric consistency checks used during fusion:

$$∥\mathbf{x}-\widehat{\mathbf{x}}∥<{\tau }_r,{\displaystyle \frac{\left|{\widehat{d}}_i\left(\mathbf{x}\right)-d_i\left(\mathbf{x}\right)\right|}{d_i\left(\mathbf{x}\right)}}<{\tau }_d,\angle \left({\mathbf{n}}_i\left(\mathbf{x}\right),{\mathbf{n}}_j\left({\mathbf{x}}^{\prime }\right)\right)<{\tau }_n.$$

(22)

Here, $\widehat{\mathbf{x}}$ and ${\widehat{d}}_i\left(\mathbf{x}\right)$ denote the reprojection location and returned depth after projecting from the source view back to the reference view. The thresholds ${\tau }_r$, ${\tau }_d$, and ${\tau }_n$ correspond to reprojection error, relative depth difference, and normal-angle difference, respectively.

Only candidates satisfying both mask and geometric consistency are fused into the final dense point cloud.

4. Experiments and Results

4.1. Datasets, Hardware, and Evaluation Protocol

Three real construction-site image datasets were used to evaluate the proposed method. They are denoted as Scene A, Scene B, and Scene C. All images were captured by monitoring cameras mounted on tower-crane trolleys, with the optical axes approximately pointing downward toward the construction areas. The datasets cover different site locations, tower-crane heights, sequence lengths, scene textures, and recurring foreground-occluder appearances.

Table 1. Construction-site image datasets used in the experiments.

Scene	Location	Number of Images	Tower-Crane Height (m)
A	Sanming, Fujian Province	391	100
B	Zhongshan, Guangdong Province	308	20
C	Xiamen, Fujian Province	276	50

summarizes the main dataset characteristics. Scene A contains 391 images collected from a construction site in Sanming, Fujian Province, at a tower-crane height of 100 m. Scene B contains 308 images collected from a construction site in Zhongshan, Guangdong Province, at a height of 20 m. Scene C contains 276 images collected from a construction site in Xiamen, Fujian Province, at a height of 50 m. All images were calibrated and distortion-corrected before reconstruction. Since the images were collected in temporal order, adjacent-view matching refers to feature matching between temporally neighboring frames.

The experimental evaluation is organized at two levels. First, all three scenes are used to assess the applicability of the proposed method under different construction-site conditions. This cross-scene evaluation focuses on sparse-reconstruction behavior, including registration completeness, sparse-point generation, track stability, reprojection error, and SfM runtime. Second, the 308-image Scene B sequence is selected as the representative controlled benchmark. It is used for pixel-level mask-quality evaluation, baseline comparison, stage-wise ablation studies, grid-aggregation analysis, runtime accounting, resource-consumption analysis, and aligned dense-point-cloud visualization. Using a fixed representative sequence ensures that the compared methods are evaluated under consistent image, reconstruction, and visualization settings.

Sparse reconstruction is evaluated using the number of registered images, the number of sparse 3D points, average track length, average reprojection error, and SfM runtime. Occluder-mask quality is evaluated on a manually annotated validation subset using intersection over union (IoU)-based metrics, including mean IoU (mIoU), mean Dice score (mDice), mean precision (mPrecision), mean recall (mRecall), micro IoU, and pixel accuracy. Dense reconstruction is assessed through aligned local point-cloud comparisons, with particular attention to floating outliers and scattered spurious points associated with recurring foreground occluders. High-precision reference point clouds are unavailable for the tested active construction sites. Therefore, the dense-stage comparisons are interpreted as visual evidence of occluder-induced artifact suppression rather than as quantitative proof of improved absolute dense-reconstruction accuracy.

All experiments were conducted on a workstation running Ubuntu 22.04.5 Long-Term Support (LTS). The workstation is equipped with an Intel Core i9-14900HX central processing unit (CPU) with 24 cores and 32 threads and an NVIDIA GeForce RTX 3090 graphics processing unit (GPU) with 24 GB of GPU memory. The software environment and benchmark-specific implementation settings are reported in Section 4.2 and Appendix A.

4.2. Implementation Details

The three-scene demonstrations and the controlled Scene B benchmark use the SAM2.1-Hiera-Large predictor with the sam2.1_hiera_l.yaml configuration. The controlled benchmark was conducted on the 308-image Scene B sequence at a resolution of $1882\times 1056$ pixels. The proposed method and the grid-aggregation ablation used image-predictor inference, whereas the manually prompted SAM2.1 reference used video propagation with interactive prompts. The main parameters of the representative benchmark are reported in Appendix A (Table A1).

The front end used SuperPoint features with LightGlue matching [33,36] and MAGSAC-based epipolar verification. Matcher confidence was bounded to $[0,1]$ before anomaly weighting, and the 8-pixel geometric threshold was used for robust partitioning and deviation truncation. Spatially aggregated geometric anomalies were converted into positive foreground prompts for SAM2.1. The resulting masks were post-processed and introduced into the SfM stage implemented with COLMAP 3.9.1 and into APD-MVS (official Computer Vision and Pattern Recognition (CVPR) 2023 GitHub implementation; https://github.com/whoiszzj/apd-mvs; accessed on 28 June 2026).

For the representative Scene B benchmark, the common-border mask was extracted from the 308-frame intersection of pixels satisfying $max(R,G,B)\le 10$ and was restricted to boundary-connected regions. The same benchmark-specific setting is listed in Appendix A (Table A1).

Each masking condition used a separately initialized COLMAP database. Feature extraction and matching were rerun for each condition.

4.3. Geometry-Driven Prompts and Mask Quality

Figure 3 presents representative prompt and mask extraction results for the three construction scenes. Grid-based aggregation concentrates prompt selection on anomaly regions supported by multiple neighboring matches. This spatial aggregation suppresses isolated outliers caused by weak texture, ordinary mismatches, local image noise, or repeated structures. SAM2.1 then converts the representative prompts into pixel-level masks. The automatic geometry-driven branches in the controlled mask-quality evaluation use the SAM2.1-Hiera-Large predictor reported in Table A1.

Table 2. Occluder-mask quality on a manually annotated validation subset.

Method	mIoU	mDice	mPrec.	mRec.	Micro IoU	Pixel Acc.
Proposed method with grid aggregation	0.640	0.724	0.816	0.725	0.570	0.931
No-grid direct geometric outliers	0.283	0.429	1.000	0.283	0.284	0.911
YOLOv8 bounding-box removal	0.314	0.458	0.649	0.428	0.283	0.865
Manually prompted SAM2.1 reference	0.973	0.986	0.997	0.977	0.973	0.997
SAM3 text-prompt baseline	0.255	0.406	0.981	0.256	0.255	0.906

reports pixel-level mask quality on a manually annotated validation subset from the representative sequence. The manually prompted SAM2.1 result provides an interactive reference for mask quality.

The proposed automatic method achieves an mIoU of 0.640 and an mDice of 0.724. Its mPrecision of 0.816 and mRecall of 0.725 indicate that the geometry-driven prompts provide usable occluder coverage without excessive foreground expansion. The remaining gap to the manually prompted SAM2.1 reference quantifies the effect of replacing direct human guidance with automatic geometry-driven prompts.

4.4. Sparse Reconstruction Evaluation and Scene C Diagnosis

Table 3. Influence of occluder masks on sparse reconstruction results.

Scene	Method	Registered Images	Sparse 3DPoints	AverageTrack Length	Average Reproj.Error	SfM Time(min)
A	Without mask	384	96,412	7.723	1.013	59.412
A	With mask	391	96,541	7.784	0.993	43.542
B	Without mask	308	73,746	6.657	0.868	54.326
B	With mask	308	73,122	6.761	0.819	36.841
C	Without mask	35	7520	3.821	0.802	59.721
C	With mask	276	45,871	4.561	0.859	48.542

reports the sparse reconstruction results with and without occluder-mask assistance for all three scenes.

For Scenes A and B, mask-based filtering reduces reprojection error and SfM runtime while maintaining or improving registration completeness. Scene C shows the strongest effect. The standard SfM pipeline registers 35 of 276 images, whereas foreground filtering recovers the complete 276-image sequence and increases the sparse-point count from 7520 to 45,871. The average track length also increases from 3.821 to 4.561.

The prompt and mask visualizations in Figure 3 show that the retained anomaly clusters are concentrated around recurring foreground occluders. For Scene C,

Table 4. Compact registration-graph diagnosis for Scene C under the sequence-matching setting.

Metric	Without Mask	With Mask
Images in the two-view graph	276	276
Connected components	1	1
Largest two-view graph component	276	276
Verified edges	2805	2770
Weighted inlier ratio	89.70%	94.74%
Cross-boundary verified edges at frames 192–193	46	33
Boundary weighted inlier ratio	61.38%	82.29%
Largest registered component after incremental mapping	35	276

and Figure 4 jointly diagnose sequence-level connectivity and the weakest temporal boundary. The first three rows of Table 4 show that both two-view graphs contain all 276 images in one connected component; thus, mask filtering does not reduce image-level graph coverage or split the sequence. The number of verified edges decreases slightly from 2805 to 2770, while the weighted inlier ratio increases from 89.70% to 94.74%. This combination indicates selective removal of lower-consistency correspondences rather than a general loss of connectivity. At the boundary between frames 192 and 193, cross-boundary verified edges decrease from 46 to 33, whereas the boundary weighted inlier ratio increases from 61.38% to 82.29%. Figure 4 visualizes the same trade-off across the sequence, with the largest improvement occurring in the shaded boundary interval. Most importantly, the final row of Table 4 shows that the largest registered component after incremental mapping increases from 35 to 276 images. Therefore, Table 4 links preserved graph coverage and improved correspondence consistency to the observed recovery of the complete Scene C sequence, supporting the conclusion that mask filtering enables more complete sparse registration.

4.5. Representative Baseline Comparison

The representative 308-image sequence is evaluated with explicit and implicit baselines to assess the relative contribution of geometry-driven prompts and alternative foreground-handling strategies. You Only Look Once version 8 (YOLOv8) bounding-box removal is an explicit detector-based baseline [37]. The manually prompted SAM2.1 result provides an interactive reference. The Segment Anything Model 3 (SAM3) text-prompt baseline uses the open-vocabulary prompt hook [38]. Visibility-Aware Multi-View Stereo Network (Vis-MVSNet) is included as a visibility-aware learning-based dense-reconstruction baseline using known camera poses.

Detailed baseline configurations and parameter-selection protocols are provided in Appendix B (

Table A2. Configuration details for the representative 308-image Scene B baselines.

Configuration Item	Details
YOLOv8s bounding-box removal
Model version and weights	Ultralytics 8.3.170; YOLOv8s (11.14 M parameters) [37]; initialized from the COCO-pretrained yolov8s.pt checkpoint and fine-tuned for the hook class
Class and pseudo-labeled training data	Single class hook; 471 SAM2.1 pseudo-labeled images; 400/71 train/validation split; seed = 42
Input resolution	Training: imgsz=640; Scene B inference: original $1882\times 1056$, letterboxed to imgsz=1280
Training hyperparameters	50 epochs; batch size = 8; patience = 20
Inference thresholds and mask conversion	Confidence = 0.10; NMS IoU = 0.70; max_det=300; fill each retained box as foreground, merge with the common border mask, and invert for COLMAP
Parameter selection	The confidence threshold was reduced from 0.25 to 0.10 after low recall was observed on the 71-image held-out validation split
SAM3 text-prompt baseline
Model and execution mode	Meta SAM3 0.1.0 image model [38]; BF16 inference on CUDA
Text prompt	hook
Instance-mask composition	Union of retained instance masks
Input resolution	Original $1882\times 1056$; internal direct resize to $1008\times 1008$; output restored to the original resolution
Inference thresholds	Concept confidence = 0.05; internal mask threshold = 0.5
Post-processing	Morphological closing kernel = 3; opening kernel = 3; remove components $<200$ px; fill holes $\le 1000$ px; merge the common border mask and invert
Parameter selection	The prompt hook and concept-confidence threshold of 0.05 were selected through a three-image usability pilot without pixel-level ground truth
Manually prompted SAM2.1
Model and configuration	SAM2.1-Hiera-Large; sam2.1_hiera_l.yaml configuration
Interactive prompting protocol	27 prompt frames and 295 points (259 positive and 36 negative); one object (obj_id=1); video-predictor propagation
Input resolution	Original $1882\times 1056$; internal image_size=1024; output restored to the original resolution
Mask binarization	Foreground: mask logits $>0$
Post-processing	Morphological closing kernel = 5; opening kernel = 3; remove components $<500$ px; fill holes $\le 2000$ px; merge the common border mask and invert
Vis-MVSNet
Implementation and checkpoint	Official Vis-MVSNet repository; step-20000 checkpoint
Inputs	308 known camera views
Input resolution	Resize to $1280\times 720$; crop to $1280\times 704$; output depth $640\times 352$
Depth-estimation settings	mode=soft; num_src=4; max_d=256; interval_scale=1; cas_depth_num=64,32,16; cas_interv_scale=4,2,1
Fusion settings	view=4; vthresh=3; pthresh=0.8,0.7,0.8; reprojection error = 1 px; relative depth difference = 0.01

). For the explicit-mask methods, the Scene B images, common-border constraint, COLMAP settings, and APD-MVS settings were held fixed; only the mask-generation module changed between methods. The 25 manually annotated Scene B frames were used only for final mask-quality evaluation and not for threshold sweeping. The mask-quality results in Table 2 show that geometry-driven prompts achieve higher overlap quality than YOLOv8 bounding-box removal and the SAM3 text-prompt baseline on the annotated validation subset. Figure 5 provides aligned local point-cloud views for the proposed method, the no-mask reconstruction, YOLOv8 bounding-box removal, manually prompted SAM2.1, and the SAM3 text-prompt baseline. Compared with the no-mask baseline, the proposed method visibly suppresses hook-induced floating artifacts while retaining more visible main-scene structure in the aligned point-cloud comparison. Figure 6 further compares the proposed full workflow with Vis-MVSNet from a top-view perspective. In this aligned top-view visualization, the proposed workflow retains more visible scene structure than the Vis-MVSNet result.

4.6. Ablation Studies

4.6.1. Sparse- and Dense-Stage Masking

Table 5. Ablation study of sparse- and dense-stage masking on the representative sequence. Runtime columns report SfM, depth estimation, fusion, and their sum; the reported total excludes frontend preprocessing.

Method	SfMMask	MVSMask	Reg.Images	SparsePoints	SfM	Depth	Fusion	Total
No mask	No	No	308	73,746	00:54:19.56	00:29:43.09	00:12:07.94	01:36:10.59
Sparse-stage mask only	Yes	No	308	73,122	00:36:50.46	00:29:41.40	00:10:28.35	01:17:00.21
Dense-stage mask only	No	Yes	308	73,746	00:54:19.56	00:23:01.20	00:06:25.04	01:23:45.80
Full method	Yes	Yes	308	73,122	00:36:50.46	00:24:04.08	00:08:18.19	01:09:12.73

reports stage-wise masking results. Dense-stage masking toggles matching-cost evaluation, valid-anchor selection, and depth-map fusion. Each condition started from a separate empty COLMAP database; feature extraction and matching were rerun. Figure 7 presents the aligned local point-cloud comparison.

Sparse-stage masking reduces visible occluder-induced artifacts. Dense-stage masking removes more floating artifacts but retains less visible scene structure. Among the tested ablation settings, the full workflow produces the fewest visually identifiable artifacts while retaining more visible scene structure than the dense-stage-mask-only condition.

4.6.2. Effect of Grid-Based Aggregation

Table 6. Ablation study of grid-based anomaly aggregation.

Metric	With Grid Aggregation	Without Grid Aggregation
Number of prompts	1522	30,668
Average SAM2 raw foreground ratio	0.098	0.569
Retained non-border masks after post-processing	28	1
mIoU	0.640	0.283
mDice	0.724	0.429
mRecall	0.725	0.283
Frontend runtime	00:02:15.65	00:02:25.46

evaluates grid-based anomaly aggregation. Without aggregation, all 30,668 geometric outliers are directly treated as positive prompts. This increases the average raw foreground-mask ratio from 0.098 to 0.569 and causes excessive foreground capture. After post-processing, the number of retained non-border masks decreases from 28 to 1, and mIoU decreases from 0.640 to 0.283.

4.7. Runtime and Resource Consumption

For the representative 308-image sequence, the measured workflow time is 01:11:28.38 for the proposed method and 01:36:10.59 for the no-mask SfM/APD-MVS baseline. All reported runtimes are directly measured single-GPU wall-clock times on the workstation described above.

Table 7. Measured runtime and resource consumption for the representative proposed workflow and no-mask baseline. The workflow total includes frontend preprocessing.

Method	Frontend	SfM	Depth	Fusion	Total	s/Image	GPUMiB	APD OutputGiB
No-mask SfM + APD-MVS	00:00:00.00	00:54:19.56	00:29:43.09	00:12:07.94	01:36:10.59	18.74	2660	5.32
Proposed method	00:02:15.65	00:36:50.46	00:24:04.08	00:08:18.19	01:11:28.38	13.92	2637	4.28

reports frontend preprocessing, SfM, depth estimation, and fusion times; its workflow total includes frontend preprocessing. Frontend preprocessing accounts for approximately 3.2% of the proposed workflow total. Mask constraints exclude mask-invalid pixels and projections from dense matching and fusion.

The dense-stage system-memory peaks are 14,970.2 mebibytes (MiB) for the proposed method and 14,649.0 MiB for the no-mask baseline. The corresponding CPU high-water marks are 13,110.9 MiB and 13,012.6 MiB.

The current implementation performs offline reconstruction. For the representative 308-image sequence, geometry-anomaly detection, prompt generation, SAM2 image-predictor inference, and mask post-processing require 00:02:15.65. SfM mapping, APD-MVS depth estimation, and depth-map fusion account for most of the runtime. Potential deployment optimizations include keyframe selection, change-detection-triggered reconstruction, incremental SfM, task queues, and block-wise depth-map fusion.

4.8. Dense Reconstruction Visualization

Figure 8 compares dense point clouds reconstructed with and without occluder-mask constraints in the three scenes. Figure 9 provides enlarged comparisons using matched viewpoints and crop ranges within each scene.

Introducing occluder masks reduces visually identifiable floating outliers and scattered spurious points in all three scenes. The effect is most pronounced in Scene C, where vertically distributed artifacts around the crane-hook region are substantially reduced after mask-constrained reconstruction. These dense comparisons remain qualitative because reference point clouds are unavailable.

5. Discussion

5.1. Mechanism and Applicability

The binary validity-map formulation matches the implemented reconstruction operations and the available pixel-level mask annotations. A future probabilistic extension could replace the hard validity indicator with calibrated reliability weights when uncertainty supervision or repeated reference measurements are available.

5.2. Limitations and Future Work

The tested datasets cover three active construction sites with different tower-crane heights, image counts, scene textures, and recurring occluder appearances.

Directly using all raw geometric outliers as positive prompts leads to excessive foreground capture. The method remains sensitive to overlapping occluders, thin cables, small distant components, low-resolution imagery, nighttime illumination, motion blur, weak texture, repeated structures, and ambiguous SAM2 boundaries. Future work will address broader acquisition conditions, reference geometry for quantitative dense-stage assessment, temporal consistency, and confidence-aware mask refinement.

6. Conclusions

This paper proposed an occluder-mask-constrained 3D reconstruction framework for tower-crane construction-site imagery. Spatially aggregated adjacent-view geometric anomalies are converted into SAM2.1 prompts, and the resulting occluder masks are propagated as shared pixel-validity constraints through sparse feature filtering, APD-MVS matching-cost evaluation, support-region selection, and depth-map fusion.

Experiments on three real construction-site datasets show improved sparse-registration completeness and fewer visually identifiable occluder-induced artifacts. In Scene C, the number of registered images increases from 35 to 276. On the representative 308-image Scene B benchmark, the proposed method achieves an mIoU of 0.640 and an mDice of 0.724, while reducing the measured workflow time from 01:36:10.59 to 01:11:28.38. Controlled ablations confirm the importance of grid-based anomaly aggregation and joint sparse- and dense-stage masking. Since high-precision reference point clouds are unavailable, the dense-stage results should be interpreted as visual evidence of artifact suppression rather than proof of improved absolute reconstruction accuracy. Future work will address broader acquisition conditions, reference geometry for quantitative dense-stage assessment, temporal consistency, and confidence-aware mask refinement.