Skip to Content
You are currently on the new version of our website. Access the old version .
AerospaceAerospace
  • Article
  • Open Access

10 March 2026

21 Pages

Complex Illumination-Aware 3D Gaussian Reconstruction for Uncooperative Space Objects

,
,
,
and
1
Innovation Academy for Microsatellites of Chinese Academy of Sciences, Shanghai 201304, China
2
University of Chinese Academy of Sciences, Beijing 101408, China
*
Author to whom correspondence should be addressed.
This article belongs to the Section Astronautics & Space Science

Abstract

High-precision 3D reconstruction of non-cooperative space targets is a critical technology for on-orbit servicing (OOS) and situational awareness, driven by the growing number of OOS missions. However, traditional visual algorithms struggle to acquire accurate geometric information due to the unique high-dynamic-range lighting and strong specular reflections characteristic of the space environment. This paper proposes Space-Gaussian, a compact 3D Gaussian reconstruction method tailored for complex lighting environments. Built upon the 3D Gaussian Splatting (3DGS) framework, the method incorporates a physically based rendering pipeline and a microfacet bidirectional reflectance distribution function model. By decoupling geometric structure from material properties and utilizing deferred rendering, it effectively suppresses geometric artifacts and specular highlights arising from non-Lambertian surface reflections. Comparative experiments on a high-fidelity simulation dataset demonstrate that Space-Gaussian outperforms mainstream methods—including Neural Radiance Fields (NeRF), Instant-NGP, GaussianShader, and 3DGS—in geometric reconstruction accuracy, novel view synthesis quality, and real-time rendering. On our self-created dataset, our approach achieves a significant performance boost over existing 3DGS methods. The results highlight its potential for high-fidelity, real-time 3D perception on resource-constrained spacecraft platforms.

1. Introduction

With the boom in commercial aerospace and deep-space exploration, the increasing number of space objects has dramatically escalated the potential for spacecraft collisions, presenting significant challenges for space situational awareness systems [1]. Consequently, high-precision 3D perception of non-cooperative space targets has become critical to the success of on-orbit servicing (OOS) and active debris removal (ADR) missions [2]. However, the unique high-dynamic-range (HDR) lighting in outer space, combined with the strong non-Lambertian reflectance characteristics of spacecraft surfaces such as multi-layer insulation (MLI) and solar panels, results in a deep coupling between geometric structure and surface appearance [3]. When facing such extreme lighting conditions, traditional vision algorithms often fail to distinguish between physical structures and specular artifacts, and there is currently a lack of standard datasets and testing benchmarks specifically designed for illumination perception, making it difficult to quantify the robustness of different algorithms under extreme lighting conditions [4]. Consequently, the reconstructed models not only lack photorealism but also suffer from severe geometric distortions, failing to meet the stringent requirements of space operations.
In the current landscape of reconstruction technologies, neural radiance fields (NeRF) [5] have achieved impressive photorealistic rendering, yet their substantial training and inference overheads struggle to meet the strict real-time constraints of spaceborne platforms. As an alternative, 3D Gaussian Splatting (3DGS) [6] has emerged as a new paradigm that facilitates real-time rendering through explicit representations, although it remains dependent on spherical harmonics for color fitting. Because this approach inherently assumes low-frequency lighting, it struggles when encountering the sharp specular highlights typical of space targets. In these instances, 3DGS tends to generate numerous fragmented pseudo-geometries to forcibly fit high-frequency signals, leading to geometric overfitting that destroys surface smoothness and renders reconstruction results misleading for physical interactions. Therefore, integrating physically grounded lighting processing into Gaussian splatting while maintaining its real-time performance represents a critical bottleneck that must be addressed.
In light of this, we propose Space-Gaussian, a physics-aware reconstruction framework tailored for complex lighting environments. Our core motivation is to explicitly decouple geometry, material, and lighting within the lightweight Gaussian representation by introducing physical constraints, thereby eliminating geometric ambiguities caused by specularities. Specifically, we deeply integrate a physically based rendering (PBR) [7] pipeline into the 3DGS framework. We design an anisotropic bidirectional reflectance distribution function (BRDF) model based on the microfacet theory to replace spherical harmonics [8], endowing Gaussian primitives with physical material properties. Additionally, we introduce an efficient deferred rendering strategy [9], achieving precise computation of interactions between environmental lighting and materials while preserving real-time performance.
The main contributions of this paper are summarized as follows:
  • We propose a physically consistent reconstruction approach specifically tailored for non-cooperative space targets. By decoupling geometry from material properties within the 3DGS framework, our method effectively mitigates geometric artifacts caused by intense specular reflections.
  • We establish a robust PBR neural rendering scheme for 3DGS. By incorporating a microfacet BRDF model, we enable explicit Gaussian representations to accurately simulate complex light–material interactions on highly reflective surfaces.
  • We introduce a solar orientation initialization strategy as a directional physical prior. This approach leverages the known sun direction in space to provide a deterministic constraint for light-source modeling, significantly improving optimization stability and convergence.
  • We design an efficient deferred rendering pipeline. This architecture allows for the high-fidelity integration of complex shading and PBR attributes while preserving the real-time rendering advantages of the 3DGS paradigm.

3. Method

3.1. System Overview

The Space-Gaussian method proposed in this paper aims to address the geometry–appearance coupling problem of space targets under strong illumination through explicit physical constraints. The overall workflow of the system consists of the following main stages: First, the physically-aware Gaussian representation upgrade, where we extend standard 3D Gaussian primitives into entities with physical properties, endowing them with microfacet material attributes such as albedo, metallicity, roughness, and normals to replace spherical harmonic color coefficients; second, G-Buffer-based deferred rendering, where we design a lightweight deferred rendering pipeline that rasterizes Gaussian attributes into screen-space geometry buffers, followed by pixel-level shading combined with a microfacet BRDF model and environmental lighting; thirdly, initialization of sunlight direction since sunlight is the primary light source in the scene; and finally, a two-stage robust optimization, employing a geometry warm-up followed by joint optimization strategy, utilizing the geometric fitting capability of 3DGS to provide initial values, and then correcting specular artifacts through the physical rendering pipeline. The flow mentioned above is presented in Figure 3.
Figure 3. An overview of the proposed framework.
This method is built upon the 3DGS [6] framework. For clarity, we first briefly review the mathematical representation of standard 3DGS. In standard 3DGS, each Gaussian is modeled as an ellipsoid characterized by a 3D position p k and a covariance matrix Σ , defined as:
G ( p ) = exp − 1 2 ( p − p k ) T Σ − 1 ( p − p k ) .
During the rendering process, 3D Gaussians are projected into 2D Gaussians in the camera space. The transformed 2D covariance matrix is approximated as: Σ ′ = J W Σ W T J T , where W is the view transformation matrix and J is the Jacobian of the perspective projection. Each Gaussian is also assigned an opacity o and a view-dependent color c i represented by Spherical Harmonics (SH). The final pixel color C is computed by blending the Gaussians in front-to-back depth order:
C = ∑ i = 1 N c i T i α i , where T i = ∏ j = 1 i − 1 ( 1 − α j ) ,
where α i is computed by multiplying the opacity o i with the Gaussian weight, which is determined by the 2D covariance matrix Σ i ′ .

3.2. Physically Based Deferred Rendering

In space environments, lighting conditions are complex and variable, and target surfaces often exhibit high-frequency reflection characteristics. To more accurately simulate such complex lighting phenomena, we introduce physically based deferred rendering technology. Adopting the BRDF model proposed by Disney in 2012 [8], each Gaussian is associated with a set of material-related properties, including albedo λ ∈ [ 0 ,   1 ] 3 , metallicity m ∈ [ 0 ,   1 ] , and roughness r ∈ [ 0 ,   1 ] . The normal vector n ∈ [ 0 ,   1 ] 3 can be computed from the tangent vectors of each 2D Gaussian via n = t u × t v .

3.2.1. Microfacet BRDF Modeling

We adopt the common microfacet form of the Disney principled BRDF, decomposing reflection into diffuse and specular components. For any pixel with an outgoing direction v that is pointing to the camera and incident direction l that is pointing to the light source, let the half-vector be h = v + l ∥ v + l ∥ . Then, the BRDF is written as:
f r ( l , v ) = k d f d ( l , v ) + k s f s ( l , v ) ,
where k d , k s are energy distribution coefficients. To represent the transition between dielectric and metallic states, we employ the metallicity parameter m to interpolate the material properties. Consequently, the specular base reflectivity is defined as follows:
F 0 = ( 1 − m ) · 0.04 · 1 + m · λ ,
where λ is the albedo. The specular term employs the GGX normal distribution and Smith geometric shadowing term, and uses the Schlick approximation to calculate the Fresnel term:
f s ( l , v ) = D ( n , h ) G ( n , l , v ) F ( v , h ) 4 ( n · l ) ( n · v ) + ϵ ,
F ( v , h ) = F 0 + ( 1 − F 0 ) ( 1 − v · h ) 5 ,
where ϵ is a small constant to prevent numerical instability. Roughness r controls the profile of the microfacet distribution; we set α = r 2 and calculate D ( · ) and G ( · ) accordingly.
The diffuse term uses the energy-conserving Lambertian form:
f d ( l , v ) = λ π .
Simultaneously, to ensure energy conservation, we set k s = F ( v , h ) and k d = ( 1 − k s ) ( 1 − m ) .

3.2.2. Lighting Discretization and Outgoing Radiance

The rendering equation is given by:
L o ( v ) = ∫ Ω f r ( l , v ) L i ( l ) ( n · l ) d ω .
Considering that lighting in space scenes is typically dominated by directional sunlight, we discretize the integral into a summation over a finite number of light sources:
L o ( v ) ≈ ∑ q = 1 Q f r ( l q , v ) I q max ( 0 , n · l q ) + L amb ,
where I q represents the RGB intensity of the q-th light source, and L amb represents the ambient and indirect light term.

3.2.3. Deferred Rendering and Gaussian Blending

To decouple physically based shading from the forward splatting process of 3DGS, we construct a material G-buffer in the screen space. Specifically, for each Gaussian i, in addition to position and covariance, we maintain material and geometric attributes:
A i = { λ i , m i , r i , n i , o i } .
During splatting, we follow the forward alpha blending of Equation (2), accumulating the contributions of different Gaussians to the same pixel in front-to-back depth order. Unlike direct color blending, the deferred rendering stage first blends to obtain pixel-level material/normal attributes (denoted as λ ¯ , m ¯ , r ¯ , n ¯ ), and then performs unified BRDF shading calculations at the pixel level:
x ¯ = ∑ i = 1 N x i w i ∑ i = 1 N w i + ϵ , w i = T i α i ,
where x represents any attribute requiring blending (e.g., λ , m , r ), and w i is the effective weight of that Gaussian for the pixel. For normals, we normalize them after blending:
n ¯ ← n ¯ ∥ n ¯ ∥ + ϵ .
Finally, the blended material attributes are substituted back into the discretized rendering equation to compute the final pixel color C final :
C final = ∑ q = 1 Q f r ( l q , v ; λ ¯ , m ¯ , r ¯ , n ¯ ) I q max ( 0 , n ¯ · l q ) + L amb .

3.2.4. Parameter Constraints and Differentiable Optimization

Since material parameters must satisfy value range constraints ( λ ∈ [ 0 ,   1 ] 3 , m ∈ [ 0 ,   1 ] , r ∈ [ 0 ,   1 ] ), we represent them as Sigmoid mappings of unconstrained variables:
λ = σ ( λ ^ ) , m = σ ( m ^ ) , r = σ ( r ^ ) ,
This mapping mechanism effectively guarantees numerical stability during the optimization process. For the normal n , we ensure it strictly satisfies the unit geometric constraint through cross-product of tangent vectors followed by normalization.

3.3. Sunlight Direction Initialization

In the space environment, the scene is illuminated by multiple sources, including the sun, Earth-reflected light, and starlight. Among these, the sun acts as the dominant directional light source with extremely high intensity. Therefore, in our discretized lighting model, we designate the primary light-source component ( q = 1 ) to explicitly represent the sunlight direction l sun in the world coordinate system. To maintain the unit vector constraint during optimization, l sun is parameterized using spherical coordinates ( θ , ϕ ) :
l sun = [ sin θ cos ϕ , sin θ sin ϕ , cos θ ] T .
Before the joint optimization, we perform an initialization of the sunlight direction to provide a reliable starting point, which is crucial for decoupling geometry and material properties. The initialization leverages the intense specular highlights observed in the captured images and the known camera poses.
First, we identify the pixel region Ω spec exhibiting high intensities across the multiview image set I . For an image I k ∈ I captured at camera pose k, we extract the 2D pixel coordinates u p = [ u , v ] T of the specular highlights by applying an intensity threshold.
Given the camera intrinsic matrix K and the extrinsic matrix T k = [ R k | t k ] , where R k ∈ S O ( 3 ) is the rotation matrix and t k ∈ R 3 is the translation vector from the world coordinate system to the camera coordinate system, we can determine the view direction v p in the world space. The 3D ray direction d c in the camera coordinate system corresponding to pixel u p is computed as:
d c = K − 1 [ u , v , 1 ] T .
The view direction v p pointing from the target surface to the camera optical center in the world coordinate system is then given by:
v p = − R k − 1 d c ∥ R k − 1 d c ∥ .
Simultaneously, we approximate the surface normal n p at the corresponding 3D point from the coarse geometry obtained during the warm-up stage. Assuming the specular highlight is generated by an ideal mirror reflection on the microfacet, the incident sunlight direction l p for a specific pixel p can be derived using the reflection law:
l p = 2 ( n p · v p ) n p − v p .
To obtain a robust global estimation of the sunlight direction, we aggregate the calculated incident directions from all valid specular pixels across multiple views. The initial sunlight direction l init is computed as the normalized average of these vectors:
l init = ∑ k ∑ p ∈ Ω spec k l p ∑ k ∑ p ∈ Ω spec k l p ,
where Ω spec k denotes the set of specular pixels in the k-th image. This physics-based geometric initialization effectively mitigates the ambiguity between geometry and lighting during the early stages of training, ensuring a physically plausible convergence.

3.4. Loss Function

To learn the geometric structure and surface material properties of the scene, we design a total loss function incorporating geometric constraints:
L = L c + λ n L n + λ smooth L smooth + λ tone L tone .
L c is the RGB reconstruction loss, defined as a weighted combination of the L 1 photometric loss and the D-SSIM structural similarity loss:
L c = ( 1 − λ ) L 1 + λ L D - SSIM ,
where λ is a balancing coefficient. Following the original 3DGS implementation [6], we set λ = 0.2 .
L n = 1 − n ˜ T n is the normal consistency loss. This term aims to promote alignment between Gaussian distributions and the geometric surface by minimizing the cosine difference between the rendered normal n obtained via alpha blending and the pseudo-ground truth surface normal n ˜ . To obtain n ˜ , we first render the depth map D using the same alpha-blending scheme as color rendering:
D = ∑ i = 1 N d i T i α i ,
where d i is the depth of the i-th Gaussian center relative to the camera. The pseudo-ground truth normal n ˜ is then derived from the spatial gradients of the depth map in screen space ( u , v ) :
n ˜ = − ∂ D ∂ u , − ∂ D ∂ v , 1 T − ∂ D ∂ u , − ∂ D ∂ v , 1 T .
L smooth = ∥ ∇ n ∥ exp ( − ∥ ∇ C gt ∥ ) is the edge-aware normal smoothness loss, intended to regularize normal variations in weakly textured regions.
Additionally, to address HDR issues in strong space lighting environments, we introduce a tone-mapping loss L tone . The standard L 1 loss is overly sensitive to numerical errors in highlight regions, which can lead to the optimization being dominated by specular highlights and overexposure artifacts. We apply a tone-mapping operator before calculation to balance gradients between highlight and shadow regions:
L tone = ∥ T ( C ) − T ( C gt ) ∥ 1 , where T ( x ) = x 1 + x .
The optimization process utilizes the Adam optimizer to jointly refine the complete set of Gaussian parameters, encompassing geometric attributes—position, orientation, scale, and opacity—as well as physical material properties including diffuse albedo, roughness, metallicity, and surface normals. This holistic approach ensures that all spatial and material components converge within a unified framework.

4. Experiments

4.1. Dataset and Experimental Setup

4.1.1. Dataset

To validate the proposed method, we constructed a high-fidelity, physics-based simulation dataset as shown in Figure 4. We selected high-precision 3D satellite models—specifically CubeSat, Sentinel-6, and Starlink—publicly released by agencies such as the ESA and the NASA and utilized the Blender rendering engine to generate synthetic image data with physical lighting attributes. To ensure the physical accuracy of the rendering parameters, the material properties used in our Blender simulation are directly sourced from the official 3D spacecraft models provided by the NASA and ESA. These official models are meticulously configured to reflect the authentic optical characteristics and physical materials of real-world satellites, thereby providing a reliable physical foundation for our synthetic dataset and minimizing the domain gap with actual space observations. In terms of scene configuration, we simulated realistic on-orbit observation tasks, constructing a LEO environment containing an Earth background and setting the observer satellite to execute fly-around observation tasks of the target satellite along specific orbits. During data acquisition, the system precisely recorded the absolute trajectory of the observer in the world coordinate system and the camera intrinsic matrix. However, given that this study aims to solve the 3D reconstruction problem of non-cooperative space targets, the relative pose between the two satellites is difficult to obtain in space, so the relative pose is not treated as known data.
Figure 4. Some image collages from our dataset.
Addressing the background interference in on-orbit images of non-cooperative targets, this study implemented a background suppression and target extraction strategy. Considering the diversity and complexity of space backgrounds like Earth, deep space and stars, we employed the classic GrabCut image segmentation algorithm [25] to separate the foreground as shown in Figure 5. GrabCut is an iterative foreground extraction technique based on Graph Cut. Its core idea is to model the color distributions of the foreground and background separately by constructing Gaussian mixture models. The algorithm first uses the initial bounding box—which, in our implementation, is automatically generated using a threshold-based saliency detection method to ensure reproducibility—as a weak supervisory signal to partition the image region into definite background, probable foreground, and undefined regions. Subsequently, by constructing a Markov random field energy function, the pixel region assignment problem is transformed into a minimum cut optimization problem in the graph theory. During the iteration process, the algorithm utilizes alternating updates of GMM parameters and pixel labels to progressively optimize the segmentation boundary until the energy function converges. Compared with dee-learning methods, GrabCut achieves precise fitting of the satellite body edges using statistical features of color and texture without requiring large-scale annotated data training. Through this preprocessing step, background-removed satellite images were generated, retaining only the texture and geometric information of the target body, which significantly improved the reconstruction effects of subsequent algorithms.
Figure 5. Data preprocessing pipeline.

4.1.2. Baselines

To comprehensively evaluate the performance of the proposed algorithm, we selected the following four representative 3D reconstruction methods as comparison baselines:
  • NeRF [5]: As a pioneering work based on implicit neural representation, NeRF utilizes MLP to map spatial coordinates and viewing directions to volume density and color, synthesizing novel view images through differentiable volume rendering techniques. Although it established high-quality standards for continuous scene representation, its computational cost is extremely high due to expensive viewing ray sampling during training and inference, resulting in slow speeds.
  • Instant-NGP [22]: This method introduces a hybrid architecture combining multi-resolution hash encoding with tiny MLPs. By leveraging the advantages of GPU parallel computing, Instant-NGP greatly alleviates the memory access bottleneck of neural networks, increasing training and inference speeds by several orders of magnitude while maintaining high reconstruction quality. It is a representative of fast neural reconstruction frameworks.
  • 3DGS [6]: This is a real-time rendering method based on explicit representation. Unlike implicit fields, 3DGS uses a set of anisotropic 3D Gaussian ellipsoids to represent the scene, combined with an efficient differentiable rasterization pipeline, achieving real-time rendering frame rates and photorealistic synthesis quality. It is currently the mainstream solution balancing reconstruction quality and rendering speed and also the primary baseline for improvement in this paper.
  • GaussianShader [24]: This method integrates a simplified physically based rendering pipeline into the 3DGS framework to handle reflective surfaces. It serves as a representative baseline for PBR-enhanced 3DGS methods, allowing us to evaluate the effectiveness of our space-specific material and lighting modeling compared with general-purpose PBR extensions.
All baseline methods are implemented based on official open-source code. To ensure a fair and objective comparison, all baseline methods were configured using the optimal hyperparameter settings recommended by their respective authors for novel view synthesis tasks. This approach aims to evaluate each method at its intended peak performance, thereby providing a reliable baseline for assessing the improvements introduced by our proposed framework.

4.1.3. Evaluation Metrics

To comprehensively measure the performance of 3D reconstruction algorithms in non-cooperative target scenarios, this paper employs metrics from two dimensions: image synthesis quality and resource computational overhead.
In terms of image synthesis quality, we selected three mainstream metrics. First is the peak signal-to-noise ratio (PSNR), which measures the mean squared error between the reconstructed image and the ground truth reference image at the pixel level; a higher value (unit: dB) indicates less distortion in the reconstruction and higher pixel-level restoration accuracy. Second is the structural similarity index measure (SSIM), which evaluates image quality comprehensively from three dimensions: luminance, contrast, and structure. Its value ranges from [ 0 , 1 ] , with values closer to 1 indicating that the reconstructed image is more consistent with the ground truth in terms of structural texture, aligning better with the human perception of structural information. Finally, learned perceptual image patch similarity (LPIPS) utilizes a pre-trained VGG deep neural network to extract features and calculate the distance between the reconstructed image and the ground truth in the feature space. A lower LPIPS value indicates that the two are closer in visual perception; compared with traditional metrics, it better reflects the recovery of high-frequency textures and details.
Targeting the resource-constrained nature of satellite computing platforms, we also focused on resource overhead and efficiency. This primarily involves two key metrics relevant to deployment feasibility: First is peak VRAM, which tracks the maximum video memory usage during both training and inference, directly reflecting the algorithm’s hardware memory requirements. The other one is execution time; since missions involving non-cooperative targets are typically time-critical, the algorithm’s computational latency is of vital importance.

4.1.4. Implementation Details

The proposed method is implemented based on the PyTorch2.6 framework, with the core rendering pipeline accelerated using CUDA. The experimental platform is configured with 12 × Intel(R) Xeon(R) Platinum 8260C CPU processors and an NVIDIA GeForce RTX 3090 GPU. Specific training parameters are set as follows:
Optimization Strategy: The total number of iterations is set to 20,000, using the Adam optimizer throughout. The initial learning rates for each learnable parameter are set as follows: position parameters at 1.6 × 10 − 4 ; scaling component of 5 × 10 − 3 and rotation component of 1 × 10 − 3 for covariance parameters; and an opacity parameter of 0.05 . For physical material attributes, we set the learning rates for roughness, metallic, reflectance, and normal to 5 × 10 − 3 , 0.01 , 5 × 10 − 3 , and 6 × 10 − 3 respectively.
Initialization and SH Settings: To fully fit the space lighting environment, the order of Spherical Harmonics is set to 3. The Gaussian point cloud initialization utilizes the sparse point cloud generated by the COLMAP SfM algorithm for a warm start, providing a good geometric prior for the optimization process.
Sparsification and Density Control: To ensure the compactness of the scene representation, we set the opacity threshold to 0.05 to prune redundant Gaussians with low contributions and execute gradient-based density control every 100 iterations. No additional model quantization compression strategy is applied in this experimental stage.

4.2. Results and Analysis

4.2.1. Quantitative Evaluation

As illustrated in Figure 6, our method achieves an optimal balance between rendering quality and memory consumption, significantly outperforming state-of-the-art NeRF and 3DGS techniques in both geometric accuracy and view synthesis quality under complex lighting conditions.
Figure 6. Performance comparison of different methods in terms of rendering quality (PSNR) versus memory consumption.
Table 1 presents the quantitative evaluation results of each method on the test set. The data indicates that in test scenes containing strong specular reflections and HDR lighting, traditional implicit methods including NeRF and Instant-NGP often struggle to accurately model drastic view-dependent lighting changes, leading to geometric divergence or foggy artifacts. Although standard 3DGS has an advantage in inference speed, it still suffers from specular artifacts and detail loss when dealing with non-Lambertian surfaces. While GaussianShader improves upon standard 3DGS by incorporating PBR, it still falls short in extreme space lighting conditions due to its reliance on general environment maps rather than explicit directional sunlight modeling. In contrast, the proposed method, benefiting from explicit material parameter modeling and physical rendering constraints, achieved significant improvements in both PSNR and SSIM metrics, proving its superior image synthesis capability under complex lighting. Meanwhile, the advantage in geometric accuracy metrics also verifies the positive role of physical constraints in geometric structure recovery.
Table 1. Quantitative results and mean values for various datasets. The intensity of the red color corresponds to performance, with deeper shades indicating superior results. An upward arrow indicates that a higher value is better, while a downward arrow indicates that a lower value is better.

4.2.2. Qualitative Comparison

Figure 7 shows the comparison of the qualitative rendering results of different methods under typical test views. We can deeply analyze the pros and cons of each method from two dimensions—geometric structure recovery and appearance detail performance:
Figure 7. Comparison of different algorithms under complex illumination. NeRF-based methods exhibit foggy blur at boundaries, while standard 3DGS produces geometric noise in specular regions. In contrast, our method recovers the cleanest geometry and texture.
Implicit representation methods such as NeRF and Instant-NGP exhibit pronounced volumetric haze when processing the high-frequency reflections typical of space targets. As illustrated in the first and second columns of the figure, this effect is particularly noticeable at the edges of solar panels. This occurs because volume-rendering techniques attempt to reconcile sharp view-dependent color transitions by populating the spatial radiance field with semi transparent density. Consequently, this process blurs the structural boundaries of the object and introduces significant floating noise within the deep space background.
The standard 3DGS method achieves sharp boundaries using explicit Gaussian ellipsoids but introduces significant geometric artifacts in non-Lambertian regions such as golden MLI films. This failure can be mathematically attributed to the frequency mismatch between the SH used for color representation and the high-frequency nature of specular reflections. In standard 3DGS, the view-dependent color c ( v ) is parameterized by SH basis functions Y l m ( v ) up to degree l m a x :
c ( v ) = ∑ l = 0 l m a x ∑ m = − l l c l m Y l m ( v )
where c l m are the learnable coefficients. SH functions are inherently low-pass filters. Even with l m a x = 3 , they can only capture smooth, low-frequency lighting variations. However, the specular reflection of space materials follows a highly concentrated microfacet distribution, such as the GGX distribution D ( n , h ) used in our method, which exhibits a sharp, high-frequency peak when the half-vector h aligns with the normal n .
Because the low-order SH cannot mathematically fit this high-frequency Dirac-like specular peak, the photometric loss L c remains high in highlight regions. To minimize this loss, the optimization process is forced to compensate for the color error by altering the geometric properties of the Gaussians. Specifically, it distorts the covariance matrix Σ to create numerous elongated, acicular primitives that are only visible from specific narrow viewpoints, thereby artificially mimicking specular highlights. This geometric overfitting manifests as an unnaturally jagged surface topography that lacks physical coherence, causing severe highlight flickering and a loss of visual realism during viewpoint transitions. GaussianShader mitigates some of these artifacts by introducing material properties, but its simplified lighting assumptions still lead to suboptimal highlight shapes and residual noise on highly reflective satellite components. It is worth noting that the geometric artifacts observed in standard 3DGS primarily stem from the inherent mathematical limitations of using low-order spherical harmonics to represent high-frequency signals. Consequently, even if the baseline models were subjected to the same level of optimization scrutiny as our method, the fundamental frequency mismatch would likely persist, making it challenging to completely eliminate erroneous geometries in specular regions.
In contrast, Space-Gaussian proposed in this paper successfully decouples color changes into functions of material attributes and lighting directions, benefiting from the introduction of the BRDF material model. As can be seen from the figure, our method not only reconstructs the smooth and continuous satellite body surface but also effectively suppresses artifacts on solar panels. The results indicate that the reconstruction accuracy of standard 3DGS decreases in direct proportion to the size of a satellite’s solar arrays. This pattern is especially visible in configurations like Starlink as shown in Figure 8, where expansive solar panels dominate the structure. When evaluating satellites with smaller panel footprints, such as the Cubesat, the improvement offered by our approach relative to the baseline becomes more modest. Despite these variations in baseline performance, Space-Gaussian exhibits unwavering robustness across all tested satellite configurations.
Figure 8. A Starlink whose structure is mostly composed of solar panels.

4.3. Ablation Study and Analysis

To verify the contribution of each core module proposed in this paper to the final performance, we design a comprehensive set of ablation studies. These studies are both conducted quantitatively, with particular emphasis on analyzing the physical-level effects caused by the removal or alteration of individual components, thereby demonstrating the scientific validity of our architectural choices. In the BRDF setting, the physically based BRDF rendering and material modeling modules are disabled, and the method degenerates into the standard 3DGS framework that directly fits view-dependent colors using SH. This variant is mainly intended to evaluate the importance of PBR for recovering specular highlights and fine-grained material details.
In the material optimization variant, the metallic and roughness parameters of all Gaussian primitives in the scene are fixed to global constants, while only the diffuse albedo is optimized during training. This experiment aims to assess the contribution of spatially varying material properties to accurately modeling heterogeneous materials commonly observed on complex satellite surfaces.
For the solar orientation initialization configuration, the prior sun direction provided by ephemeris data is not used. Instead, the scene illumination direction is randomly initialized and jointly optimized with other parameters. This ablation is designed to examine the convergence behavior and robustness of the proposed method under unknown or inaccurate lighting conditions.
Finally, in the deferred shading setting, the G-buffer generation and deferred shading pipeline are removed, and color computation is performed directly during the rasterization stage. This comparison validates the advantage of the deferred rendering pipeline in decoupling geometry from illumination and enhancing the reconstruction of high-frequency details.
To quantitatively evaluate these variants, Table 2 reports the results of all ablation configurations on the test set. As can be observed, the complete Space-Gaussian model consistently achieves the best performance across all metrics, demonstrating the necessity and effectiveness of each proposed module. Specifically, the removal of the BRDF module leads to a significant drop in PSNR (from 36.10 dB to 32.12 dB), quantitatively confirming its critical role in handling complex specular reflections.
Table 2. Ablation studies on the components of Space-Gaussian. An upward arrow indicates that a higher value is better, while a downward arrow indicates that a lower value is better.

5. Discussion and Limitations

5.1. The Scarcity of Non-Cooperative Space Target Datasets

While 3D reconstruction datasets for terrestrial scenes such as urban buildings and indoor environments have become increasingly comprehensive through benchmarks like ScanNet and DTU, high-fidelity datasets for non-cooperative space targets remain remarkably scarce. This shortage significantly restricts the advancement of deep-learning algorithms designed for on-orbit perception. The primary obstacle to acquiring real-world data is the high technical and financial threshold associated with space missions.
Beyond the difficulties of data acquisition, existing simulation datasets like SPEED+ [26], SHIRT [27], SPADES [28], SPARK, and URSO [29] often fail to provide sufficient physical realism. Most current synthetic data focus on pose estimation and employ simplified lighting models or low-dynamic-range environment maps that cannot accurately reproduce the extreme contrast of the space environment. More importantly, the surface materials of space targets—including multi-layer insulation, solar arrays, and composite antennas—exhibit complex reflectance properties that are frequently overlooked. Traditional simulations generally ignore strong specular reflections and anisotropic effects, resulting in a substantial domain gap when models trained on synthetic data are applied to real scenarios.
This study addresses these limitations by introducing a simulation dataset specifically engineered to enhance physical accuracy. We have implemented the precise modeling of direct sunlight and Earth albedo while establishing a material library based on actual physical measurements to capture the intricate scattering effects of satellite components. Although this dataset represents a significant step forward in replicating the physical attributes of light and matter, the diversity of target types and the complexity of their motion states remain areas for future expansion. By refining these simulation techniques, we aim to provide a more robust foundation for the development of autonomous on-orbit perception systems.

5.2. Dilemma of SfM-Based Pose Estimation and Feature Matching

In the reconstruction workflow of implicit neural fields and 3D Gaussian splatting, precise camera pose estimation and sparse point cloud initialization are fundamental to achieving high quality results. While traditional structure from motion algorithms like COLMAP are standard, they encounter significant obstacles when applied to non-cooperative space targets. This failure stems primarily from the extreme optical properties of spacecraft surfaces, particularly the strong specular reflections produced by multi-layer insulation films. These metallic surfaces act as mirrors, causing detected feature points to shift across the object as the viewpoint changes rather than remaining anchored to physical locations. Such violations of multiview consistency prevent bundle adjustment from converging, which introduces substantial pose errors and produces stratified point clouds.
Furthermore, the lack of distinct textures and the prevalence of periodic patterns on the satellite structures exacerbate matching ambiguities. Many spacecraft bodies consist of smooth metal skins that offer minimal corner or edge features, resulting in extremely sparse feature sets. Conversely, the lattice patterns of large solar panels often trigger perceptual aliasing, where the matching algorithm incorrectly identifies correspondences between identical looking patches. In practice, these repetitive structures frequently lead to camera drift or total scene breakage, especially in satellite models characterized by expansive solar arrays. These limitations highlight the inherent fragility of feature-based matching in orbital environments.
Due to the inherent difficulty of matching feature points on certain aircraft surfaces, conventional SfM pipelines often fail [30]. Consequently, exploring COLMAP-free methodologies represents a critical direction for future research.

5.3. Model Limitations and Efficacy Analysis

Space-Gaussian significantly enhances reconstruction quality by effectively decoupling geometry from appearance through a microfacet-based PBR pipeline. Traditional frameworks often produce semi transparent artifacts or spatial noise when attempting to fit specular highlights. In contrast, this approach interprets highlights as a function of material properties and lighting directions, ensuring that the reconstructed surfaces remain dense and geometrically accurate. Consequently, the model successfully captures sharp highlight transitions and realistic shadow dynamics on complex reflective surfaces such as solar panels and thermal insulation.
Despite these advancements, the framework faces limitations regarding lighting complexity and viewpoint generalization. The current reliance on simplified environment maps and spherical harmonics fails to account for intricate global illumination effects, such as inter reflections between satellite components. This deficiency can introduce inaccuracies in albedo estimation and shadow rendering within occluded regions. Moreover, the model remains sensitive to extreme viewpoint deviations. In the absence of comprehensive geometric priors, observing unrecorded surfaces during constrained orbital maneuvers can lead to significant artifacts or structural gaps.
Finally, integrating physically based rendering necessitates a compromise between visual fidelity and computational efficiency. The inclusion of BRDF components and normal estimations increases the processing burden, resulting in a performance reduction of approximately thirty percent compared with standard 3DGS. Such an overhead poses challenges for deployment on hardware with limited onboard processing power. Future development will therefore focus on streamlining the calculation flow to achieve a more favorable balance between rendering precision and real-time operational requirements in space environments.

6. Conclusion and Future Work

Addressing the challenges of 3D reconstruction for on-orbit non-cooperative targets caused by HDR lighting and strong specular reflections, this paper proposes a physics-aware Gaussian reconstruction framework named Space-Gaussian. This method innovatively integrates a PBR pipeline into the 3DGS architecture. By introducing a microfacet BRDF model and deferred rendering technology, it achieves effective decoupling of geometric structure and surface material attributes. Extensive experiments on a high-fidelity simulation dataset demonstrate that Space-Gaussian significantly outperforms mainstream methods such as NeRF, Instant-NGP, GaussianShader, and standard 3DGS. It not only effectively eliminates geometric artifacts and highlights leakage in strong specular reflection areas but also achieves the best balance among novel view synthesis quality, geometric reconstruction accuracy, and real-time rendering, exhibiting excellent robustness and engineering application potential.
Looking to the future, this study plans to deepen and expand the following dimensions: first, enhancing complex lighting interactions, by introducing ambient occlusion and higher-order lighting estimation models to further improve reconstruction stability under extreme shadow variations and self-occlusion conditions; second, conducting experiments using real-world satellite imagery in the future, although the acquisition of such datasets remains a significant challenge; and third, improving the algorithm to be COLMAP-free, enabling it to adapt to a wider range of scenarios.

Author Contributions

Conceptualization, Z.Q., Z.Z. and L.C.; methodology, Z.Q.; software, Z.Q.; validation, Z.Q., Z.Z., R.X., J.Z. and L.C.; formal analysis, Z.Q.; investigation, R.X.; resources, R.X.; data curation, J.Z.; writing—original draft preparation, Z.Q.; writing—review and editing, Z.Z.; visualization, J.Z.; supervision, L.C.; project administration, L.C.; funding acquisition, L.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
OOSOn-Orbit Servicing
ADRActive Debris Removal
HDRHigh Dynamic Range
MLIMulti-layer Insulation
NeRFNeural Radiance Fields
3DGS3D Gaussian Splatting
BRDFBidirectional Reflectance Distribution Function
PBRPhysically Based Rendering
MVSMultiview Stereo
LEOLow Earth Orbit
SfMStructure from motion
RPORendezvous and Proximity Operations
MLPMulti-layer Perceptron

References

  1. Gu, Y.; Li, Z.; Liu, H.; Luo, Q.; Liu, H.; Wu, G. Multi-objective early warning mission planning by multiple satellites using a critical task aggregation-based NSGA-II algorithm. Adv. Space Res. 2025, in press. [Google Scholar] [CrossRef] [Scilit]
  2. Flores-Abad, A.; Ma, O.; Pham, K.; Ulrich, S. A review of space robotics technologies for on-orbit servicing. Prog. Aerosp. Sci. 2014, 68, 1–26. [Google Scholar] [CrossRef] [Scilit]
  3. Park, T.H.; D’Amico, S. Robust multi-task learning and online refinement for spacecraft pose estimation across domain gap. Adv. Space Res. 2024, 73, 5726–5740. [Google Scholar] [CrossRef] [Scilit]
  4. Rangelov, D.; Waanders, S.; Waanders, K.; van Keulen, M.; Miltchev, R. Three-Dimensional Reconstruction Techniques and the Impact of Lighting Conditions on Reconstruction Quality: A Comprehensive Review. Lights 2025, 1, 1. [Google Scholar] [CrossRef] [Scilit]
  5. Mildenhall, B.; Srinivasan, P.P.; Tancik, M.; Barron, J.T.; Ramamoorthi, R.; Ng, R. NeRF: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 2021, 65, 99–106. [Google Scholar] [CrossRef] [Scilit]
  6. Kerbl, B.; Kopanas, G.; Leimkuehler, T.; Drettakis, G. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph. 2023, 42, 1–14. [Google Scholar] [CrossRef] [Scilit]
  7. Cook, R.L.; Torrance, K.E. A reflectance model for computer graphics. SIGGRAPH Comput. Graph. 1981, 15, 307–316. [Google Scholar] [CrossRef] [Scilit]
  8. Burley, B.; Studios, W.D.A. Physically-based shading at disney. In Proceedings of the ACM Siggraph; Association for Computing Machinery: New York, NY, USA, 2012; Volume 2012, pp. 1–7. [Google Scholar]
  9. Saito, T.; Takahashi, T. Comprehensible rendering of 3-D shapes. In Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques; SIGGRAPH ’90; Association for Computing Machinery: New York, NY, USA, 1990; pp. 197–206. [Google Scholar] [CrossRef] [Scilit]
  10. Roberts, L.G. Machine Perception of Three-Dimensional Solids. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 1963. [Google Scholar]
  11. Longuet-Higgins, H.C. A computer algorithm for reconstructing a scene from two projections. Nature 1981, 293, 133–135. [Google Scholar] [CrossRef] [Scilit]
  12. Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
  13. Levoy, M.; Pulli, K.; Curless, B.; Rusinkiewicz, S.; Koller, D.; Pereira, L.; Ginzton, M.; Anderson, S.; Davis, J.; Ginsberg, J.; et al. The digital Michelangelo project: 3D scanning of large statues. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques; SIGGRAPH ’00; Association for Computing Machinery: New York, NY, USA, 2000; pp. 131–144. [Google Scholar] [CrossRef] [Scilit]
  14. Friend, R.B. Orbital Express program summary and mission overview. In Proceedings of the Sensors and Systems for Space Applications II; SPIE: Bellingham, WA, USA, 2008; Volume 6958, pp. 11–21. [Google Scholar] [CrossRef] [Scilit]
  15. Forshaw, J.L.; Aglietti, G.S.; Navarathinam, N.; Kadhem, H.; Salmon, T.; Pisseloup, A.; Joffre, E.; Chabot, T.; Retat, I.; Axthelm, R.; et al. RemoveDEBRIS: An in-orbit active debris removal demonstration mission. Acta Astronaut. 2016, 127, 448–463. [Google Scholar] [CrossRef] [Scilit]
  16. Biesbroek, R.; Aziz, S.; Wolahan, A.; Cipolla, S.; Richard-noca, M.; Piguet, L. The ClearSpace-1 mission: ESA and ClearSpace team up to remove debris. In Proceedings of the 8th European Conference on Space Debris; ESA Space Debris Office: Darmstadt, Germany, 2021. [Google Scholar]
  17. Choy, C.B.; Xu, D.; Gwak, J.; Chen, K.; Savarese, S. 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction. In Proceedings of the Computer Vision—ECCV 2016, Honolulu, HI, USA, 11–14 October 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer: Cham, Switzerland, 2016; pp. 628–644. [Google Scholar] [CrossRef] [Scilit]
  18. Charles, R.Q.; Su, H.; Kaichun, M.; Guibas, L.J. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2017; pp. 77–85. [Google Scholar] [CrossRef] [Scilit]
  19. Artzy, E.; Frieder, G.; Herman, G.T. The theory, design, implementation and evaluation of a three-dimensional surface detection algorithm. Comput. Graph. Image Process. 1981, 15, 1–24. [Google Scholar] [CrossRef] [Scilit]
  20. Barron, J.T.; Mildenhall, B.; Verbin, D.; Srinivasan, P.P.; Hedman, P. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 21–26 July 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 5460–5469. [Google Scholar] [CrossRef] [Scilit]
  21. Verbin, D.; Hedman, P.; Mildenhall, B.; Zickler, T.; Barron, J.T.; Srinivasan, P.P. Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. IEEE Trans. Pattern Anal. Mach. Intell. 2025, 47, 9426–9437. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  22. Müller, T.; Evans, A.; Schied, C.; Keller, A. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. (TOG) 2022, 41, 102. [Google Scholar] [CrossRef] [Scilit]
  23. Forray, C.; Delporte, P.; Delaygue, N.; Genin, F.; Derksen, D. Joint attitude estimation and 3D neural reconstruction of non-cooperative space objects. arXiv 2025, arXiv:2506.20638. [Google Scholar] [CrossRef] [Scilit]
  24. Jiang, Y.; Tu, J.; Liu, Y.; Gao, X.; Long, X.; Wang, W.; Ma, Y. GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces. In Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2024; pp. 5322–5332. [Google Scholar] [CrossRef] [Scilit]
  25. Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 2004, 23, 309–314. [Google Scholar] [CrossRef] [Scilit]
  26. Park, T.H.; Märtens, M.; Lecuyer, G.; Izzo, D.; D’Amico, S. SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap. In Proceedings of the 2022 IEEE Aerospace Conference (AERO); IEEE: Piscataway, NJ, USA, 2022; pp. 1–15. [Google Scholar] [CrossRef] [Scilit]
  27. Park, T.H.; D’Amico, S. SHIRT: Satellite Hardware-In-the-Loop Rendezvous Trajectories Dataset. Stanford Digital Repository. 2022. Available online: https://purl.stanford.edu/zq716br5462 (accessed on 31 January 2026).
  28. Rathinam, A.; Qadadri, H.; Aouada, D. SPADES: A Realistic Spacecraft Pose Estimation Dataset using Event Sensing. In Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA); IEEE: Piscataway, NJ, USA, 2024; pp. 11760–11766. [Google Scholar] [CrossRef] [Scilit]
  29. Proença, P.F.; Gao, Y. Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA); IEEE: Piscataway, NJ, USA, 2020; pp. 6007–6013. [Google Scholar] [CrossRef] [Scilit]
  30. Issitt, A.; Mahendrakar, T.; Alvarez, A.; White, R.T.; Sizemore, A. On Optimal Observation Orbits for Learning Gaussian Splatting-Based 3D Models of Unknown Resident Space Objects. In Proceedings of the AIAA SCITECH 2025 Forum; American Institute for Aeronautics and Astronautics (AIAA): Reston, VA, USA, 2025; p. 1780. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.