2. Materials and Methods
Early approaches to 3D object removal combined 2D inpainting with Neural Radiance Fields (NeRFs). Mirzaei et al. introduced Spatial Perceptual Inpainting for NeRF (SPIn-NeRF) [
3], which uses LPIPS-based perceptual supervision and geometric priors to promote 3D consistency through NeRF volume rendering. Wang et al. subsequently proposed Innerf360 [
4], extending this paradigm to 360-degree scenes through depth-space distortion and density regularization for depth-constrained segmentation alignment. Haque et al. developed Instruct-NeRF2NeRF [
5], which incorporates denoising diffusion implicit-model sampling into an iterative editing loop and uses the gradual convergence of NeRF optimization to regularize independently edited views. NeRFiller [
6] addresses large missing regions using a joint multi-view inpainting prior that averages diffusion-model noise predictions across views. Guo et al. combined a geometric diffusion prior with balanced score distillation to improve geometric consistency and reduce rendering artifacts in completed 3D scenes [
7]. Although these implicit approaches can remove target objects, large geometric gaps may still produce unresolved multi-view conflicts. The resulting regions can appear blurred or over-smoothed and may lack high-frequency detail. Incomplete removal can also leave residual geometry, holes, or artifacts at the original object location [
8].
To reduce residual geometry and ghosting artifacts, recent studies have explored explicit scene representations. Such representations offer efficient rendering and can preserve sharp structural boundaries. InFusion combines 3D Gaussian Splatting (3DGS) with diffusion-prior-guided depth completion [
9], whereas VEIGAR uses explicit inpainting and geometric alignment to improve multi-view consistency [
10]. Related work has also investigated generative priors and latent-space alignment. ReplaceAnything3D supports text-guided object replacement in compositional scenes [
11], and In-N-Out aligns explicit and implicit latent features at corresponding geometric locations to preserve globally coherent texture and illumination [
12]. Although these methods can produce clean edits at training viewpoints, their quality may degrade at novel viewpoints. In particular, monocular-depth projections may lack global metric consistency, leading to geometric drift or floating structures as the viewpoint changes. We address these limitations with a depth-aware mask-generation pipeline based on metric depth priors [
13], spherical parameterization adapted from mip-NeRF 360 [
2], and a joint objective that combines smooth inter-layer and original-scene consistency constraints. Together, these components reduce floating artifacts and geometric drift while preserving fine background texture. To represent a continuous 3D scene, NeRF-based methods [
1,
14] cast a ray
from camera center
in viewing direction
. The anti-aliased formulation represents each pixel as a conical frustum. At distance
t, the frustum has radius
, where
is proportional to the cone opening angle. We align the
z-axis with the ray direction, define the radial distance from the ray axis as
r, and denote the azimuthal angle by
. The near and far boundaries of the frustum are
and
, respectively:
The upper bound
reflects the linear increase in cone radius with depth, with
denoting the cone-opening coefficient. The conical samples are mapped to Cartesian coordinates
as follows:
The corresponding differential volume element, obtained from the Jacobian determinant, is:
For each sampling interval
, its contribution weight
follows the discrete form of the physical rendering equation:
Here,
denotes cumulative transmittance, and
is the distance between adjacent samples. Equations (1)–(4) follow the standard NeRF volume-rendering formulation [
1] and mip-NeRF conical sampling [
14].
Multi-view inpainting requires a coherent 3D scene to be recovered from independently inpainted and potentially inconsistent observations. Our pipeline first completes the masked regions in 2D and then optimizes a NeRF-based 3D representation to synthesize consistent inpainted novel views, as illustrated in
Figure 1. Given input RGB images
, binary masks restrict the 2D inpainting operation to the target regions and prevent unnecessary changes elsewhere.
For multi-object scenes, we introduce a geometry-aware mask-generation pipeline based on Prior Depth Anything (PriorDA). As shown in
Figure 2, a compact detector is trained using a limited set of bounding-box annotations. Metric depth cues from PriorDA improve the localization of target objects
across views and guide the generation of segmentation masks
.
The original images
and masks
are then passed to a Fourier-based inpainting network. Fast Fourier Convolutions (FFCs) [
15] provide a global receptive field, enabling the network to synthesize structurally coherent content within large, irregularly shaped masks.
To support large, unbounded scenes, we adopt the scene-contraction function from mip-NeRF 360 [
2], which maps unbounded coordinates to the interior of a ball of radius 2:
We then apply differential sampling and express depth
t using the normalized coordinate
s:
Here,
and
denote the camera near and far planes, respectively. We use the identity mapping
, which is trivially invertible. Consequently, uniform intervals in
s correspond to uniform intervals in depth
t, and the weight-resampling intervals follow the same normalized parameterization. The total objective for 3D inpainting is:
We set
and
as the default values. The sensitivity analysis changes one parameter at a time while holding the other at its default value and evaluates
and
in addition to the default configuration. Following the interlevel regularization of mip-NeRF 360 [
2], the smooth inter-layer loss
is defined as:
The vectors s and contain interval endpoints, and w and are the corresponding weight vectors, with and . The term denotes the weights obtained by resampling onto the histogram intervals defined by .
To regularize the density distribution, we use the distortion loss
adapted from mip-NeRF 360 [
2]:
Here,
s defines the sampling intervals and
w denotes their weights. The distortion loss penalizes both the weighted pairwise distances between samples and the weighted width of each interval, thereby discouraging diffuse or excessively concentrated density distributions. The original-scene loss
constrains unmasked background regions and reduces content drift:
Even after extensive training, a 2D inpainting model may fail to recover the true content behind a removed object, while overly large weights in individual histogram intervals can produce floating artifacts (
Figure 3). We therefore incorporate the original scene into the reconstruction objective to improve color and density estimation. For each training view, the original image set
supplies the ground-truth background color
at every unmasked pixel
, whereas
denotes the corresponding rendered color. The sampling intervals and weights,
and
, remain separate quantities regularized by
and
. By constraining the unmasked background,
improves color and density estimates and limits the content drift commonly observed in 2D-to-3D inpainting pipelines. Although
can slightly reduce pixel-level metrics such as PSNR, it improves global semantic consistency, as reflected by the FID results reported in the Results section.
Although SPIn-NeRF [
3] and InFusion [
9] also use NeRF or 3DGS to promote view-consistent inpainting, our method is specifically designed for unbounded 360° environments. It combines spherical parameterization adapted from mip-NeRF 360 [
2] with the joint objective
to anchor the background structure. LaMa [
15] supplies the FFC-based 2D inpainting prior, which is lifted into a consistent 3D representation through NeRF optimization (
Figure 4).