Next Article in Journal
Design Theory and Application of Reinforcement for Existing Slab Culverts Through the Additional Deck Slab
Previous Article in Journal
A Hybrid NLLB and Large Language Model Pipeline for Diachronic Intralingual Translation of 16th-Century Slovene Literary Heritage
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

View-Consistent 3D Inpainting in Unbounded Scenes via Anti-Aliased Neural Radiance Fields

1
School of Optoelectronic Engineering, Xidian University, Xi’an 710071, China
2
Xi’an Aviation Computing Technology Research Institute, Aviation Industry Corporation of China (AVIC), Xi’an 710000, China
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2026, 16(14), 7316; https://doi.org/10.3390/app16147316
Submission received: 14 April 2026 / Revised: 11 July 2026 / Accepted: 16 July 2026 / Published: 21 July 2026
(This article belongs to the Section Computing and Artificial Intelligence)

Abstract

Recent advances in NeRF-based inpainting have enabled the completion of masked regions across multi-view images. However, two major challenges remain: generating accurate masks efficiently in the presence of complex multi-object interference and maintaining view consistency without floating artifacts in large-scale, unbounded 360° environments. To address these challenges, we propose a 3D inpainting framework with two principal components. First, a depth-aware mask-generation pipeline produces high-quality, view-consistent masks from sparse annotations. Second, spherical parameterization is combined with a joint objective comprising smooth inter-layer and original-scene constraints to suppress floating artifacts and content drift in unbounded scenes. We also introduce IM2360, a multi-object 360° dataset for evaluating 3D inpainting methods. Experiments show that our approach outperforms existing NeRF-based inpainting methods in PSNR, LPIPS, and FID. On a single NVIDIA GeForce RTX 4090, our method requires an average of 25.21 min of training per scene, compared with 98.14 min for SPIn-NeRF, 38.71 min for OR-NeRF, and 11.29 min for InFusion. These results indicate that the proposed method provides a favorable balance between reconstruction quality and computational cost for high-fidelity restoration of complex immersive scenes.

1. Introduction

In immersive applications such as augmented reality (AR) and virtual reality (VR), constructing clean 3D environments from real-world captures is essential. The objective of 3D scene inpainting is to remove specified objects or occlusions and reconstruct the geometry and appearance of the missing regions from contextual information. This task builds on 2D image inpainting, in which generative priors, including diffusion models, synthesize missing content from a single viewpoint. Directly applying independent 2D inpainting models to multi-view image sets, however, often produces cross-view inconsistencies because these models lack explicit 3D spatial awareness and are susceptible to interference from multiple objects. To bridge the gap between 2D edits and 3D consistency, recent frameworks use 3D representations, including the Neural Radiance Field (NeRF) formulation introduced by Mildenhall et al. [1], as a regularizing structure. By representing a scene as continuous volume-density and radiance fields, NeRF can integrate independently inpainted 2D observations into a unified 3D representation. Nevertheless, local editing within a continuous radiance field remains challenging. Existing 3D inpainting pipelines have three principal limitations: they are sensitive to multi-object interference, which can produce inaccurate masks and geometric errors; they often fail to maintain structural and scale consistency in large, unbounded 360-degree environments; and insufficient scene-level constraints during optimization can lead to the loss of fine background texture and global content drift.
To address these limitations, we propose a view-consistent 3D inpainting framework designed to suppress artifacts while preserving fine background texture in complex environments. The main contributions are as follows:
  • We develop a depth-aware mask-generation pipeline that incorporates metric depth priors to produce view-consistent masks from sparse annotations and reduce interference among multiple objects.
  • We develop a 3D inpainting framework for unbounded 360° environments that uses spherical parameterization, adapted from mip-NeRF 360 [2], to contract large-scale scenes into a bounded coordinate domain.
  • We introduce a joint objective comprising smooth inter-layer and original-scene constraints to suppress floating artifacts and preserve texture fidelity in the inpainted regions.
Experimental results demonstrate that the proposed method improves both visual quality and quantitative performance relative to existing approaches while producing more consistent inpainted novel views.

2. Materials and Methods

Early approaches to 3D object removal combined 2D inpainting with Neural Radiance Fields (NeRFs). Mirzaei et al. introduced Spatial Perceptual Inpainting for NeRF (SPIn-NeRF) [3], which uses LPIPS-based perceptual supervision and geometric priors to promote 3D consistency through NeRF volume rendering. Wang et al. subsequently proposed Innerf360 [4], extending this paradigm to 360-degree scenes through depth-space distortion and density regularization for depth-constrained segmentation alignment. Haque et al. developed Instruct-NeRF2NeRF [5], which incorporates denoising diffusion implicit-model sampling into an iterative editing loop and uses the gradual convergence of NeRF optimization to regularize independently edited views. NeRFiller [6] addresses large missing regions using a joint multi-view inpainting prior that averages diffusion-model noise predictions across views. Guo et al. combined a geometric diffusion prior with balanced score distillation to improve geometric consistency and reduce rendering artifacts in completed 3D scenes [7]. Although these implicit approaches can remove target objects, large geometric gaps may still produce unresolved multi-view conflicts. The resulting regions can appear blurred or over-smoothed and may lack high-frequency detail. Incomplete removal can also leave residual geometry, holes, or artifacts at the original object location [8].
To reduce residual geometry and ghosting artifacts, recent studies have explored explicit scene representations. Such representations offer efficient rendering and can preserve sharp structural boundaries. InFusion combines 3D Gaussian Splatting (3DGS) with diffusion-prior-guided depth completion [9], whereas VEIGAR uses explicit inpainting and geometric alignment to improve multi-view consistency [10]. Related work has also investigated generative priors and latent-space alignment. ReplaceAnything3D supports text-guided object replacement in compositional scenes [11], and In-N-Out aligns explicit and implicit latent features at corresponding geometric locations to preserve globally coherent texture and illumination [12]. Although these methods can produce clean edits at training viewpoints, their quality may degrade at novel viewpoints. In particular, monocular-depth projections may lack global metric consistency, leading to geometric drift or floating structures as the viewpoint changes. We address these limitations with a depth-aware mask-generation pipeline based on metric depth priors [13], spherical parameterization adapted from mip-NeRF 360 [2], and a joint objective that combines smooth inter-layer and original-scene consistency constraints. Together, these components reduce floating artifacts and geometric drift while preserving fine background texture. To represent a continuous 3D scene, NeRF-based methods [1,14] cast a ray r ( t ) = o + t d from camera center o in viewing direction d . The anti-aliased formulation represents each pixel as a conical frustum. At distance t, the frustum has radius r ˙ t , where r ˙ is proportional to the cone opening angle. We align the z-axis with the ray direction, define the radial distance from the ray axis as r, and denote the azimuthal angle by θ . The near and far boundaries of the frustum are t 0 and t 1 , respectively:
t t 0 , t 1 θ [ 0 , 2 π ) r [ 0 , r ˙ t ]
The upper bound r ˙ t reflects the linear increase in cone radius with depth, with r ˙ denoting the cone-opening coefficient. The conical samples are mapped to Cartesian coordinates x = ( x , y , z ) as follows:
x = r cos θ y = r sin θ z = t
The corresponding differential volume element, obtained from the Jacobian determinant, is:
d x d y d z = r d r d θ d t
For each sampling interval t i , t i + 1 , its contribution weight w i follows the discrete form of the physical rendering equation:
w i = T i 1 exp σ i δ i
Here, T i = exp j = 1 i 1 σ j δ j denotes cumulative transmittance, and δ i = t i + 1 t i is the distance between adjacent samples. Equations (1)–(4) follow the standard NeRF volume-rendering formulation [1] and mip-NeRF conical sampling [14].
Multi-view inpainting requires a coherent 3D scene to be recovered from independently inpainted and potentially inconsistent observations. Our pipeline first completes the masked regions in 2D and then optimizes a NeRF-based 3D representation to synthesize consistent inpainted novel views, as illustrated in Figure 1. Given input RGB images I i i = 1 n , binary masks restrict the 2D inpainting operation to the target regions and prevent unnecessary changes elsewhere.
For multi-object scenes, we introduce a geometry-aware mask-generation pipeline based on Prior Depth Anything (PriorDA). As shown in Figure 2, a compact detector is trained using a limited set of bounding-box annotations. Metric depth cues from PriorDA improve the localization of target objects A i i = 1 n across views and guide the generation of segmentation masks M i i = 1 n .
The original images I i i = 1 n and masks M i i = 1 n are then passed to a Fourier-based inpainting network. Fast Fourier Convolutions (FFCs) [15] provide a global receptive field, enabling the network to synthesize structurally coherent content within large, irregularly shaped masks.
To support large, unbounded scenes, we adopt the scene-contraction function from mip-NeRF 360 [2], which maps unbounded coordinates to the interior of a ball of radius 2:
S ( x ) = x , x 2 1 2 1 x 2 x x 2 , x 2 > 1
We then apply differential sampling and express depth t using the normalized coordinate s:
s = k ( t ) k t n k t f k t n t = k 1 s · k t f + ( 1 s ) · k t n
Here, t n and t f denote the camera near and far planes, respectively. We use the identity mapping k ( t ) = t , which is trivially invertible. Consequently, uniform intervals in s correspond to uniform intervals in depth t, and the weight-resampling intervals follow the same normalized parameterization. The total objective for 3D inpainting is:
L = α L e l + β L dist + L G T
We set α = 0.01 and β = 0.005 as the default values. The sensitivity analysis changes one parameter at a time while holding the other at its default value and evaluates α { 0.056 , 0.1 } and β { 0.01 , 0.056 } in addition to the default configuration. Following the interlevel regularization of mip-NeRF 360 [2], the smooth inter-layer loss L e l is defined as:
L e l ( s , w , s ^ , w ^ ) = i 1 w ^ i max 0 , w i s ^ w ^ i 3 w i s ^ + w ^ i
The vectors s and s ^ contain interval endpoints, and w and w ^ are the corresponding weight vectors, with i w i 1 and i w ^ i 1 . The term w s ^ denotes the weights obtained by resampling ( s , w ) onto the histogram intervals defined by s ^ .
To regularize the density distribution, we use the distortion loss L dist adapted from mip-NeRF 360 [2]:
L dist = i , j w i w j s i + s i + 1 2 s j + s j + 1 2 + 1 3 i w i 2 s i + 1 s i
Here, s defines the sampling intervals and w denotes their weights. The distortion loss penalizes both the weighted pairwise distances between samples and the weighted width of each interval, thereby discouraging diffuse or excessively concentrated density distributions. The original-scene loss L G T constrains unmasked background regions and reduces content drift:
L G T = p M C ^ ( p ) C G T ( p ) 2 2
Even after extensive training, a 2D inpainting model may fail to recover the true content behind a removed object, while overly large weights in individual histogram intervals can produce floating artifacts (Figure 3). We therefore incorporate the original scene into the reconstruction objective to improve color and density estimation. For each training view, the original image set { I o , i } i = 1 n supplies the ground-truth background color C G T ( p ) at every unmasked pixel p M , whereas C ^ ( p ) denotes the corresponding rendered color. The sampling intervals and weights, ( s , w ) and ( s ^ , w ^ ) , remain separate quantities regularized by L e l and L dist . By constraining the unmasked background, L G T improves color and density estimates and limits the content drift commonly observed in 2D-to-3D inpainting pipelines. Although L e l can slightly reduce pixel-level metrics such as PSNR, it improves global semantic consistency, as reflected by the FID results reported in the Results section.
Although SPIn-NeRF [3] and InFusion [9] also use NeRF or 3DGS to promote view-consistent inpainting, our method is specifically designed for unbounded 360° environments. It combines spherical parameterization adapted from mip-NeRF 360 [2] with the joint objective L G T + L e l + L dist to anchor the background structure. LaMa [15] supplies the FFC-based 2D inpainting prior, which is lifted into a consistent 3D representation through NeRF optimization (Figure 4).

3. Results

3.1. Experiments

All experiments were conducted on a single NVIDIA GeForce RTX 4090 GPU with 24 GB of memory (NVIDIA Corporation, Santa Clara, CA, USA). ROI-based mask generation was used to isolate specified objects in multi-object images. A fixed random seed controlled the stochastic sampling patterns used during training and rendering. The NeRF backbone follows the Zip-NeRF/AA-NeRF 360 architecture with sinusoidal positional encoding. We optimized the network with Adam, using an initial learning rate of 5 × 10 4 that decayed exponentially over 100,000 iterations and a batch size of 4096 rays. Training the proposed 3D reconstruction stage required 23.10–31.56 min per scene (mean: 25.21 min). A compact detector trained from bounding-box annotations generated the initial object locations, and PriorDA [13] supplied metric-depth guidance for mask refinement. LaMa [15], an FFC-based network with a global receptive field, generated the 2D inpainting prior. All baselines used the same masks and 2D inpainting inputs. Unless otherwise specified, α = 0.01 and β = 0.005 .
Images were captured at a resolution of 1920 × 1080 , and camera poses were estimated using COLMAP (version 3.8) structure from motion. For each scene, 80–100 images were used for training and 20 held-out images were used for testing. Each image contained approximately 2–5 annotated ROIs. We report a single deterministic run per scene using a fixed random seed; results were not averaged across seeds.
We introduce IM2360 to evaluate object removal in large, unbounded 360-degree scenes. The dataset contains seven real-world outdoor scenes, each comprising 80–120 images with annotated ROIs. Individual images contain multiple objects, together with the locations and class labels required to generate target-specific masks and inpainted inputs.
We evaluated both local inpainting quality and global reconstruction quality using peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet Inception Distance (FID). Metrics were computed globally and, where applicable, within the masked regions of synthesized novel views.
SPIn-NeRF differs from our method primarily in its NeRF-based 3D optimization procedure. It iteratively feeds rendered novel views back into the large-mask Fourier-convolution inpainting network and uses the updated images to refine the radiance field. For a controlled comparison, all NeRF-based methods were fitted to the same 2D inpainted images, and their rendered novel views were evaluated against the corresponding reference images.

3.2. Qualitative Results

As shown in Figure 3, spherical parameterization enables the network to represent large, unbounded scenes and synthesize coherent inpainted novel views across a full 360-degree capture.
Figure 5 compares ground-truth images with results from the proposed method, SPIn-NeRF, and InFusion on IM2360. Figure 6 further shows that the proposed framework removes the target object while preserving cross-view consistency, without conspicuous geometric drift or floating artifacts.

3.3. Quantitative Results

We first evaluate global reconstruction quality using PSNR, SSIM, and LPIPS. As shown in Table 1, the proposed method yields a lower mean SSIM than SPIn-NeRF. Prior work on perceptual image quality has shown that SSIM may favor over-smoothed outputs in which high-frequency texture is suppressed [16,17]. Accordingly, the near-unity SSIM reported for SPIn-NeRF (0.9995) should be interpreted together with its substantially worse LPIPS (0.4839) and PSNR (24.16 dB). Our method achieves a mean SSIM of 0.8879, LPIPS of 0.05079, and PSNR of 30.48 dB. A bootstrap analysis across the seven scenes yields 95% confidence intervals of 30.48 ± 2.1 dB for PSNR and 0.051 ± 0.028 for LPIPS, indicating that the aggregate improvements are not attributable to a single scene.
OR-NeRF [8] obtains the lowest overall performance on IM2360, with mean values of 14.08 dB for PSNR, 0.2793 for SSIM, and 0.7950 for LPIPS. These results suggest that mask-guided NeRF optimization without scene contraction and the proposed joint constraints is poorly suited to large, unbounded 360° environments. Compared with the explicit 3DGS-based InFusion method, our approach improves mean PSNR from 28.34 to 30.48 dB and reduces mean LPIPS from 0.2053 to 0.05079. The observed geometric drift in the InFusion results is consistent with the difficulty of applying monocular depth-completion priors to unbounded scenes with large scale variation. In contrast, AA-NeRF 360 combines anti-aliased conical sampling with spherical parameterization to maintain a denser and more stable background representation.
We use Fréchet Inception Distance (FID) to evaluate perceptual realism at both the full-image and masked-region levels. As shown in Table 2, the proposed method achieves the lowest mean global FID (48.77), compared with 245.5 for SPIn-NeRF, 62.272 for InFusion, and 462.03 for OR-NeRF. It also achieves the lowest mean local FID (21.13), compared with 23.92 for SPIn-NeRF and 93.476 for InFusion; OR-NeRF does not report local FID. These results indicate that the proposed method better preserves global semantic consistency while generating perceptually realistic content within the inpainted regions.
To assess cross-dataset generalization, Table 3 reports masked-region LPIPS on the public SPIn-NeRF dataset [3]. The proposed method achieves 0.4096, compared with 0.4650 for SPIn-NeRF, 0.6273 for OR-NeRF, and 0.4210 for InFusion.
To quantify computational cost, we measured the per-scene training time of the proposed method, SPIn-NeRF, InFusion, and OR-NeRF on IM2360. All methods were executed on a single NVIDIA GeForce RTX 4090 using the same scene inputs and the optimization schedules prescribed by their respective implementations (Table 4).
The proposed method requires an average of 25.21 min per scene, representing reductions of 74.3% relative to SPIn-NeRF (98.14 min) and 34.9% relative to OR-NeRF (38.71 min). InFusion is the fastest method, with an average training time of 11.29 min. However, Table 1 and Table 2 show that InFusion also yields lower mean PSNR (28.34 versus 30.48 dB) and higher mean LPIPS (0.2053 versus 0.05079), global FID (62.272 versus 48.77), and local FID (93.476 versus 21.13). Thus, the proposed method does not minimize training time in absolute terms but provides a favorable trade-off between computational cost, reconstruction accuracy, and perceptual quality.
We next evaluate the contributions of the original-scene loss L G T and the smooth inter-layer loss L e l . Figure 7 presents the qualitative results. Adding L G T alone slightly decreases PSNR from 28.13 to 28.07 because of its strict background-preservation constraint. Adding L e l alone suppresses floating artifacts and increases PSNR to 28.60. Using both terms yields the best overall result (PSNR: 28.79; SSIM: 0.8483; LPIPS: 0.05766), indicating that global background anchoring and inter-layer regularization are complementary.
To isolate the contribution of each design choice, we ablate depth-aware supervision, spherical parameterization, and anti-aliased conical sampling, and we also evaluate a configuration that retains only L dist . Table 5, Table 6 and Table 7 report per-scene SSIM, PSNR, and LPIPS for all seven IM2360 scenes. Table 8, Table 9 and Table 10 report the corresponding sensitivity analyses for α and β .
Removing depth-aware supervision, spherical parameterization, or anti-aliased sampling produces similar mean performance across the seven scenes: 28.44–28.49 dB PSNR, 0.7523–0.7531 SSIM, and 0.3290–0.3297 LPIPS. By contrast, retaining only L dist reduces mean PSNR to 8.51 dB and mean SSIM to 0.3598 while increasing mean LPIPS to 0.9272. This degradation is more than an order of magnitude larger than the differences among the other three ablations, indicating that L G T and L e l are necessary for coherent convergence. Depth-aware masking, spherical parameterization, and anti-aliased sampling provide smaller individual gains in the aggregate metrics. Stone3 yields the lowest PSNR and SSIM in nearly every setting (22.24–22.25 dB, compared with 27.84–31.97 dB for the other scenes), consistent with its relative difficulty in Table 1.
The sensitivity analysis evaluates four one-parameter-at-a-time settings beyond the default configuration ( α = 0.01 , β = 0.005 ). Setting α to 0.1 or β to 0.01 yields results close to those of the architectural ablations (PSNR: 28.47/28.33 dB; SSIM: 0.7518/0.7505; LPIPS: 0.3307/0.3295). Setting α or β to 0.056 produces greater deviations (PSNR: 27.94/27.10 dB; SSIM: 0.7323/0.7153; LPIPS: 0.3602/0.3644). This effect is most pronounced for Stone1 and Table, where β = 0.056 decreases PSNR by 2.34 and 2.11 dB, respectively, relative to β = 0.01 . These results indicate that performance is less robust to large values of β than to large values of α , although the largest observed decrease remains substantially smaller than the collapse obtained with L dist alone.

4. Discussion

Like other neural radiance field methods, AA-NeRF 360 can overfit low-texture scenes, for which additional geometric priors may be beneficial. Nevertheless, the method preserves detailed spatial structure in inpainted novel views and may therefore support non-contact digital restoration of cultural heritage objects. As illustrated in Figure 8, the framework reconstructs a damaged pottery artifact and synthesizes a completed 3D representation. This application demonstrates the potential of view-consistent inpainting for digital conservation, while further domain-specific validation remains necessary.
Limitations. Most quantitative experiments were conducted on the newly collected IM2360 dataset, which was designed to represent large-scale, unbounded 360° object-removal scenarios. We additionally evaluated masked-region LPIPS on the public SPIn-NeRF benchmark [3]; as shown in Table 3, the proposed method achieves 0.4096, compared with 0.4650 for SPIn-NeRF, 0.6273 for OR-NeRF, and 0.4210 for InFusion. Although this result provides evidence of cross-dataset generalization, evaluation on additional public benchmarks is needed. A direct quantitative comparison with InNeRF360 [4] was not possible because official training code was unavailable at the time of submission. Moreover, InNeRF360 primarily targets indoor bounded 360° scenes, whereas IM2360 focuses on outdoor unbounded environments, which limits direct comparability.

Author Contributions

Conceptualization, Z.G., X.W. and X.L. (Xiang Li); methodology, Z.G.; software, X.L. (Xiaoyan Lin); validation, Z.G., Q.C., X.L. (Xiaoyan Lin) and X.S.; formal analysis, Z.G. and M.Z.; investigation, Z.G. and Z.W.; resources, X.W. and J.Z.; data curation, Z.G. and D.L.; writing—original draft preparation, Z.G.; writing—review and editing, X.W. and X.L. (Xiang Li); visualization, Z.G.; supervision, X.W. and X.L. (Xiang Li); project administration, X.W.; funding acquisition, X.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Science Foundation for Distinguished Young Scholars of China, grant number 61904138; an internal project, number 2019JQ-235; the Fundamental Research Funds for the Central Universities, grant number 20101186281; and the 111 Project, grant number B17035. The APC was funded by Xidian University.

Data Availability Statement

The IM2360 dataset and the code developed during this study are available from the corresponding author upon reasonable request.

Acknowledgments

The authors would like to thank the editors and the anonymous reviewers for their insightful comments and constructive suggestions. We also acknowledge the administrative and technical support provided by the laboratories at Xidian University and the Xi’an Aviation Computing Technology Research Institute. The authors have reviewed and edited the output and take full responsibility for the content of this publication.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
NeRFNeural Radiance Fields
3DGS3D Gaussian Splatting
ARAugmented Reality
VRVirtual Reality
FFCFast Fourier Convolutions
ROIRegion of Interest
PSNRPeak Signal-to-Noise Ratio
SSIMStructural Similarity Index
LPIPSLearned Perceptual Image Patch Similarity
FIDFréchet Inception Distance
PriorDAPrior Depth Anything

References

  1. Mildenhall, B.; Srinivasan, P.P.; Tancik, M.; Barron, J.T.; Ramamoorthi, R.; Ng, R. NeRF: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 2021, 65, 99–106. [Google Scholar] [CrossRef]
  2. Barron, J.T.; Mildenhall, B.; Verbin, D.; Srinivasan, P.P.; Hedman, P. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 19–24 June 2022. [Google Scholar]
  3. Mirzaei, K.; Kwon, G.; Lin, C.H.; Gwak, J.; Kwak, S.; Nguyen, P.H.; Yeung, S. SPIn-NeRF: Multiview segmentation and perceptual inpainting with neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 18–22 June 2023; pp. 20669–20679. [Google Scholar]
  4. Wang, D.; Lin, X.; Sun, X.; Zhu, M.; Wen, Z. Innerf360: Text-guided 3D-consistent object inpainting on 360-degree neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 17–21 June 2024; pp. 12677–12686. [Google Scholar]
  5. Haque, Z.; Tancik, M.; Yi, A.A.; Kanazawa, A. Instruct-NeRF2NeRF: Editing 3D Scenes with 2D Visual Instruction. arXiv 2023, arXiv:2303.12789. [Google Scholar]
  6. Weber, J.; Laina, I.; Rupprecht, C. NeRFiller: Completing Scenes via 3D Inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 17–21 June 2024. [Google Scholar]
  7. Guo, J.; Zhang, M.; Luo, X.; Lan, Y.; Liu, C.; Li, R.; Zhang, K.; Yang, G.; Liu, D. NeRF Inpainting with Geometric Diffusion Prior and Balanced Score Distillation. arXiv 2024, arXiv:2411.15551. [Google Scholar]
  8. Yin, Y.; Fu, Z.; Yang, F.; Lin, G. OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fields. arXiv 2023, arXiv:2305.10503. [Google Scholar]
  9. Liu, Z.; Ouyang, H.; Wang, Q.; Cheng, K.L.; Xiao, J.; Zhu, K.; Xue, N.; Liu, Y.; Shen, Y.; Cao, Y. InFusion: Inpainting 3D Gaussian Splatting via Seamless 2D Inpainting. arXiv 2024, arXiv:2404.11613. [Google Scholar]
  10. Do, P.K.N.; Tran, B.N.; Nguyen, N.; Nguyen, D.D. VEIGAR: View-consistent Explicit Inpainting and Geometry Alignment for 3D Object Removal. arXiv 2025, arXiv:2506.15821. [Google Scholar]
  11. Bartrum, E.; Nguyen-Phuoc, T.; Xie, C.; Li, Z.; Khan, N.; Avetisyan, A.; Lanman, D.; Xiao, L. ReplaceAnything3D: Text-Guided 3D Scene Editing with Compositional Neural Radiance Fields. arXiv 2024, arXiv:2401.17895. [Google Scholar]
  12. Hu, D.; Fu, H.; Guo, J.; Peng, L.; Chu, T.; Liu, F.; Liu, T.; Gong, M. In-N-Out: Lifting 2D Diffusion Prior for 3D Object Removal via Tuning-Free Latents Alignment. In Advances in Neural Information Processing Systems (NeurIPS), Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada, 10–15 December 2024; Curran Associates, Inc.: Red Hook, NY, USA, 2024. [Google Scholar]
  13. Wang, Z.; Chen, S.; Yang, L.; Wang, J.; Zhang, Z.; Zhao, H.; Zhao, Z. Depth Anything with any prior. arXiv 2025, arXiv:2505.10565. [Google Scholar]
  14. Barron, J.T.; Mildenhall, B.; Tancik, M.; Hedman, P.; Martin-Brualla, R.; Srinivasan, P.P. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 11–17 October 2021; pp. 5855–5864. [Google Scholar]
  15. Suvorov, R.; Logacheva, E.; Mashikhin, A.; Rem, E.; Ashukha, A.; Silvestrov, A.; Kong, N.; Goka, H.; Park, K.; Lempitsky, V. Resolution-robust Large Mask Inpainting with Fourier Convolutions. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2022. [Google Scholar]
  16. Zhang, R.; Isola, P.; Efros, A.A.; Shechtman, E.; Wang, O. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–22 June 2018. [Google Scholar]
  17. Johnson, J.; Alahi, A.; Fei-Fei, L. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, 8–16 October 2016. [Google Scholar]
Figure 1. Overview of the proposed framework. The three colored boxes denote the input, 2D inpainting, and NeRF-based 3D inpainting stages, and the solid arrows indicate data flow between operations. The brackets labeled L G T and L e l indicate the original-scene and smooth inter-layer constraints, respectively. The red dashed box and connecting lines identify the enlarged region in the rendered result. A small set of manually annotated images is used to train an object detector, which predicts regions of interest (ROIs) across the image set. Salient-object detection then generates a mask for each image. The original images and masks are processed by a 2D inpainting network, and the resulting multi-scale inpainted images and camera poses are used to optimize the NeRF-based 3D inpainter for novel-view synthesis.
Figure 1. Overview of the proposed framework. The three colored boxes denote the input, 2D inpainting, and NeRF-based 3D inpainting stages, and the solid arrows indicate data flow between operations. The brackets labeled L G T and L e l indicate the original-scene and smooth inter-layer constraints, respectively. The red dashed box and connecting lines identify the enlarged region in the rendered result. A small set of manually annotated images is used to train an object detector, which predicts regions of interest (ROIs) across the image set. Salient-object detection then generates a mask for each image. The original images and masks are processed by a 2D inpainting network, and the resulting multi-scale inpainted images and camera poses are used to optimize the NeRF-based 3D inpainter for novel-view synthesis.
Applsci 16 07316 g001
Figure 2. Multi-view mask generation. (a) Original input image containing multiple objects; the Chinese text on the traffic cone reads “Traffic Control, Security Office” and is not used by the method. (b) Object-detection output with class labels and bounding boxes. (c) Final binary mask for the selected target object (the box labeled “box_0” in (b)). A compact object detector localizes instances of the specified class, after which salient-object detection generates the corresponding mask. The depth-aware pipeline improves target isolation in multi-object scenes across viewpoints.
Figure 2. Multi-view mask generation. (a) Original input image containing multiple objects; the Chinese text on the traffic cone reads “Traffic Control, Security Office” and is not used by the method. (b) Object-detection output with class labels and bounding boxes. (c) Final binary mask for the selected target object (the box labeled “box_0” in (b)). A compact object detector localizes instances of the specified class, after which salient-object detection generates the corresponding mask. The depth-aware pipeline improves target isolation in multi-object scenes across viewpoints.
Applsci 16 07316 g002
Figure 3. Novel-view synthesis on the 360-degree dataset. (a) Failure case produced by SPIn-NeRF. (b) Result produced by the proposed method. (c) Ground-truth image.
Figure 3. Novel-view synthesis on the 360-degree dataset. (a) Failure case produced by SPIn-NeRF. (b) Result produced by the proposed method. (c) Ground-truth image.
Applsci 16 07316 g003
Figure 4. Comparison of the scene representations before and after object removal. (a) Original scene model. (b) Inpainted 3D model produced by the proposed method. (c) Scene model after removal of the target object.
Figure 4. Comparison of the scene representations before and after object removal. (a) Original scene model. (b) Inpainted 3D model produced by the proposed method. (c) Scene model after removal of the target object.
Applsci 16 07316 g004
Figure 5. Qualitative comparison of ground-truth and rendered images for the IM2360 scenes using the proposed method, SPIn-NeRF, and InFusion. AA-NeRF 360 reconstructs unbounded 360-degree scenes and synthesizes view-consistent inpainted novel views.
Figure 5. Qualitative comparison of ground-truth and rendered images for the IM2360 scenes using the proposed method, SPIn-NeRF, and InFusion. AA-NeRF 360 reconstructs unbounded 360-degree scenes and synthesizes view-consistent inpainted novel views.
Applsci 16 07316 g005
Figure 6. Inpainted results from novel viewpoints after removal of the target object.
Figure 6. Inpainted results from novel viewpoints after removal of the target object.
Applsci 16 07316 g006
Figure 7. Qualitative loss ablation. (a) Results obtained using the 2D inpainting prior and Zip-NeRF. (b) Results obtained with L G T . (c) Results obtained with L e l . (d) Results obtained with both L G T and L e l . The original-scene constraint improves agreement with unmasked regions, whereas the inter-layer loss suppresses floating artifacts.
Figure 7. Qualitative loss ablation. (a) Results obtained using the 2D inpainting prior and Zip-NeRF. (b) Results obtained with L G T . (c) Results obtained with L e l . (d) Results obtained with both L G T and L e l . The original-scene constraint improves agreement with unmasked regions, whereas the inter-layer loss suppresses floating artifacts.
Applsci 16 07316 g007
Figure 8. Cultural-heritage restoration example. (a) Ground-truth image of the damaged pottery artifact, with an enlarged view of the damaged region. (b) Inpainted novel view generated by AA-NeRF 360 after reconstructing the artifact as a complete 3D object, with an enlarged view of the restored region.
Figure 8. Cultural-heritage restoration example. (a) Ground-truth image of the damaged pottery artifact, with an enlarged view of the damaged region. (b) Inpainted novel view generated by AA-NeRF 360 after reconstructing the artifact as a complete 3D object, with an enlarged view of the restored region.
Applsci 16 07316 g008
Table 1. Quantitative comparison on the newly collected IM2360 dataset using SSIM, PSNR, and LPIPS. Best results are shown in bold. ↑ indicates that higher values are better, whereas ↓ indicates that lower values are better.
Table 1. Quantitative comparison on the newly collected IM2360 dataset using SSIM, PSNR, and LPIPS. Best results are shown in bold. ↑ indicates that higher values are better, whereas ↓ indicates that lower values are better.
SceneSSIM ↑PSNR ↑LPIPS ↓
SPInOR-NeRFInFusionOursSPInOR-NeRFInFusionOursSPInOR-NeRFInFusionOurs
Chair0.99980.31740.92480.971826.1516.0430.4536.070.36190.79760.14640.00804
Lion0.99960.34990.87610.909922.8212.6227.9529.890.48140.78820.24720.03738
Stone10.99970.21250.87460.899226.2914.9426.6529.870.46120.78570.22400.04100
Stone20.99940.27420.88790.838823.1313.2626.6828.820.58210.75620.20560.07102
Stone30.99870.26640.86240.836718.8115.6925.9727.900.66590.88680.23310.09809
Table0.99980.29840.91640.910826.2913.0931.2632.080.39900.80890.19810.04236
Wood0.99980.23610.88570.848325.6312.9129.4328.790.43590.74180.18300.05766
Mean0.99950.27930.88970.887924.1614.0828.3430.480.48390.79500.20530.05079
Table 2. FID comparison on the newly collected IM2360 dataset. OR-NeRF reports only global FID; unavailable local FID values are denoted by –. Best results are shown in bold. ↓ indicates that lower values are better.
Table 2. FID comparison on the newly collected IM2360 dataset. OR-NeRF reports only global FID; unavailable local FID values are denoted by –. Best results are shown in bold. ↓ indicates that lower values are better.
SceneFID (Local) ↓FID (Global) ↓
SPInOR-NeRFInFusionOursSPInOR-NeRFInFusionOurs
Chair27.1757.4085.83132.16371.178.1214.417
Lion28.72143.00128.16336.5462.7943.549108.8
Stone148.90115.17332.68226.6447.79176.36025.16
Stone224.07197.23741.31438.1467.1590.31983.58
Stone330.41106.41626.70428.7456.6667.65438.08
Table1.99521.0563.02367.73485.8525.47431.75
Wood6.19814.03810.27189.3542.8224.42649.63
Mean23.9293.47621.13245.5462.0362.27248.77
Table 3. Masked-region LPIPS comparison on the SPIn-NeRF public dataset [3]. Baseline results are taken from the respective papers. Bold indicates the best result, and ↓ indicates that lower values are better.
Table 3. Masked-region LPIPS comparison on the SPIn-NeRF public dataset [3]. Baseline results are taken from the respective papers. Bold indicates the best result, and ↓ indicates that lower values are better.
SPIn-NeRF [3]OR-NeRF [8]InFusion [9]Ours
LPIPS ↓0.46500.62730.42100.4096
Table 4. Per-scene training-time comparison on the IM2360 dataset. Training time is reported in minutes, and lower values indicate lower computational cost. All methods were executed on a single NVIDIA GeForce RTX 4090 using the same scene inputs and their respective prescribed optimization schedules. The shortest training time in each column is shown in bold.
Table 4. Per-scene training-time comparison on the IM2360 dataset. Training time is reported in minutes, and lower values indicate lower computational cost. All methods were executed on a single NVIDIA GeForce RTX 4090 using the same scene inputs and their respective prescribed optimization schedules. The shortest training time in each column is shown in bold.
MethodChairLionStone1Stone2Stone3TableWoodMean
SPIn-NeRF651061131081211047098.14
InFusion111012101681211.29
OR-NeRF3632453852313738.71
Ours25.3024.0123.5925.4231.5623.1023.4625.21
Table 5. Ablation study on the IM2360 dataset—SSIM ↑, broken down per scene, for four ablation settings: without depth-aware supervision (w/o Depth), without spherical parameterization (w/o Spherical), without anti-aliased/conical sampling (w/o Anti-alias), and the distortion loss alone ( L dist only). All settings use the default α = 0.01 , β = 0.005 unless otherwise ablated. The α / β sensitivity analysis is reported separately in Table 8. ↑ indicates that higher values are better.
Table 5. Ablation study on the IM2360 dataset—SSIM ↑, broken down per scene, for four ablation settings: without depth-aware supervision (w/o Depth), without spherical parameterization (w/o Spherical), without anti-aliased/conical sampling (w/o Anti-alias), and the distortion loss alone ( L dist only). All settings use the default α = 0.01 , β = 0.005 unless otherwise ablated. The α / β sensitivity analysis is reported separately in Table 8. ↑ indicates that higher values are better.
Scenew/o Depthw/o Sphericalw/o Anti-Alias L dist Only
Chair0.88010.88050.87990.3627
Lion0.82160.82450.82160.4020
Stone10.76690.76220.76520.3104
Stone20.79640.79700.79640.4200
Stone30.45840.45940.45850.2478
Table0.78170.78280.77940.4765
Wood0.76510.76520.76520.2993
Mean0.75290.75310.75230.3598
Table 6. Ablation study on the IM2360 dataset—PSNR ↑ (dB), with the same four ablation settings as Table 5. ↑ indicates that higher values are better.
Table 6. Ablation study on the IM2360 dataset—PSNR ↑ (dB), with the same four ablation settings as Table 5. ↑ indicates that higher values are better.
Scenew/o Depthw/o Sphericalw/o Anti-Alias L dist Only
Chair31.9431.9731.954.040
Lion28.6828.8428.654.476
Stone128.9728.8628.8910.89
Stone229.7029.7229.6611.14
Stone322.2422.2522.247.812
Table29.9429.9529.8412.02
Wood27.8427.8627.859.195
Mean28.4728.4928.448.51
Table 7. Ablation study on the IM2360 dataset—LPIPS ↓, with the same four ablation settings as Table 5. ↓ indicates that lower values are better.
Table 7. Ablation study on the IM2360 dataset—LPIPS ↓, with the same four ablation settings as Table 5. ↓ indicates that lower values are better.
Scenew/o Depthw/o Sphericalw/o Anti-Alias L dist Only
Chair0.14690.14720.14740.9737
Lion0.24200.23760.24170.8990
Stone10.33180.33850.33220.9201
Stone20.27220.27380.27290.8976
Stone30.68910.68980.68880.9553
Table0.40730.40620.41090.9443
Wood0.21380.21370.21380.9004
Mean0.32900.32950.32970.9272
Table 8. Sensitivity of α and β on the IM2360 dataset—SSIM ↑, broken down per scene. Four one-parameter-at-a-time settings are evaluated beyond the default configuration ( α = 0.01 , β = 0.005 ). In each column, only the named parameter is changed, while the other parameter is held at its default value. ↑ indicates that higher values are better.
Table 8. Sensitivity of α and β on the IM2360 dataset—SSIM ↑, broken down per scene. Four one-parameter-at-a-time settings are evaluated beyond the default configuration ( α = 0.01 , β = 0.005 ). In each column, only the named parameter is changed, while the other parameter is held at its default value. ↑ indicates that higher values are better.
Scene α = 0.1 β = 0.01 α = 0.056 β = 0.056
Chair0.88060.87980.87050.8660
Lion0.82260.82090.80990.8043
Stone10.76820.76400.71390.6681
Stone20.79600.79580.77710.7767
Stone30.46350.45990.45090.4416
Table0.76720.76760.75830.7135
Wood0.76460.76520.74540.7369
Mean0.75180.75050.73230.7153
Table 9. Sensitivity of α and β on the IM2360 dataset—PSNR ↑ (dB), with the same four one-parameter-at-a-time settings as Table 8. ↑ indicates that higher values are better.
Table 9. Sensitivity of α and β on the IM2360 dataset—PSNR ↑ (dB), with the same four one-parameter-at-a-time settings as Table 8. ↑ indicates that higher values are better.
Scene α = 0.1 β = 0.01 α = 0.056 β = 0.056
Chair31.9931.9631.6131.32
Lion28.7328.5028.2327.70
Stone128.9828.8627.8426.52
Stone229.9729.5829.1228.91
Stone322.3822.1722.2121.40
Table29.404329.4029.1027.29
Wood27.8227.8527.4426.58
Mean28.4728.3327.9427.10
Table 10. Sensitivity of α and β on the IM2360 dataset—LPIPS ↓, with the same four one-parameter-at-a-time settings as Table 8. ↓ indicates that lower values are better.
Table 10. Sensitivity of α and β on the IM2360 dataset—LPIPS ↓, with the same four one-parameter-at-a-time settings as Table 8. ↓ indicates that lower values are better.
Scene α = 0.1 β = 0.01 α = 0.056 β = 0.056
Chair0.14540.14580.16630.1652
Lion0.24110.23940.26250.2570
Stone10.32900.33610.40780.4538
Stone20.27450.26910.30370.2907
Stone30.68760.67970.69310.6825
Table0.42330.42340.45090.4580
Wood0.21430.21290.23680.2433
Mean0.33070.32950.36020.3644
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Guo, Z.; Wu, X.; Cheng, Q.; Li, X.; Lin, X.; Sun, X.; Zhu, M.; Wen, Z.; Liu, D.; Zhang, J. View-Consistent 3D Inpainting in Unbounded Scenes via Anti-Aliased Neural Radiance Fields. Appl. Sci. 2026, 16, 7316. https://doi.org/10.3390/app16147316

AMA Style

Guo Z, Wu X, Cheng Q, Li X, Lin X, Sun X, Zhu M, Wen Z, Liu D, Zhang J. View-Consistent 3D Inpainting in Unbounded Scenes via Anti-Aliased Neural Radiance Fields. Applied Sciences. 2026; 16(14):7316. https://doi.org/10.3390/app16147316

Chicago/Turabian Style

Guo, Zhaoxiang, Xin Wu, Qiang Cheng, Xiang Li, Xiaoyan Lin, Xue Sun, Mingxia Zhu, Zelin Wen, Delian Liu, and Jianqi Zhang. 2026. "View-Consistent 3D Inpainting in Unbounded Scenes via Anti-Aliased Neural Radiance Fields" Applied Sciences 16, no. 14: 7316. https://doi.org/10.3390/app16147316

APA Style

Guo, Z., Wu, X., Cheng, Q., Li, X., Lin, X., Sun, X., Zhu, M., Wen, Z., Liu, D., & Zhang, J. (2026). View-Consistent 3D Inpainting in Unbounded Scenes via Anti-Aliased Neural Radiance Fields. Applied Sciences, 16(14), 7316. https://doi.org/10.3390/app16147316

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop