Joint 3D Reconstruction and Classification of Aircraft Based on Single-Image Neural Implicit Optimization

Highlights

What are the main findings?

• A 3D reconstruction pipeline based on neural implicit optimization is proposed, enabling the three-dimensional reconstruction of aircraft utilizing a single optical remote sensing image as input.
• A unified engineering framework is established through feature fusion to jointly perform aircraft 3D reconstruction and classification.

What are the implications of the main findings?

• The proposed 3D reconstruction method effectively leverages prior knowledge to mitigate the ill-posed nature of single-view reconstruction, thereby substantially improving the reconstruction quality.
• The integrated framework utilizes the geometric cues derived from the reconstruction process to enhance classification accuracy and robustness, providing a practical and effective solution for optical remote sensing image interpretation.

Abstract

3D reconstruction and classification of aircraft are two active research areas in optical remote sensing image processing which are of great significance for applications such as airport monitoring and intelligence analysis. The traditional approaches usually focus only on one of these two tasks, and all these methods suffer from inherent limitations. In the field of 3D reconstruction, most current methods require multiple-view images as input, which is rarely feasible in remote sensing. However, single-view 3D reconstruction is an inherently ill-posed problem. Existing methods, including voxel generation and mesh template deformation, still suffer from limited accuracy and poor shape fidelity. In the field of image classification, the existing methods are mainly based on deep learning. These methods require a large amount of labeled data, and they may also be misled by the color and texture features of the target in the dataset. In this paper, we propose a unified framework for simultaneous 3D reconstruction and classification, specifically tailored for aircraft targets in optical remote sensing imagery. The key innovations are threefold: First, we introduce the Signed Distance Field (SDF) implicit representation to build a prior-guided 3D reconstruction framework pre-trained on 3D model datasets. Second, to achieve the reconstruction process with a single image as input, we design a new joint optimization pipeline. We propose a novel dual-kernel differentiable rendering method, which is fused behind the SDF generation network for iterative optimization of the implicit code and pose parameters. Third, a gated feature fusion module is developed to combine the optimal latent vector from reconstruction with the classification backbone. This integration enables the joint output of 3D meshes and category labels within a unified loop. The resulting optimal latent code plays a dual role as a generative seed for high-fidelity 3D reconstruction and as a low-dimensional feature representation for target classification. Quantitative evaluations validate the superiority of our joint framework. Compared with the strong mesh-based competitor AtlasNet, the proposed method yields a 12.2% boost in mean F-score. In object classification, leveraging the 3D implicit geometric features boosts the performance to a peak accuracy of 97.88%, outperforming advanced remote sensing backbones such as RSMamba and EAM by 2.03% and 2.54%. Additionally, ablation studies confirm the indispensability of our key designs, revealing that our dual-task feature fusion strategy brings an absolute gain of 1.18% in classification accuracy, while omitting the clustering prior stages and the dual-kernel rendering method leads to a 30.4% and 10.1% degradation in Chamfer distance.

1. Introduction

The rapid advancement of satellite imaging technology has led to an explosion of high-resolution remote sensing data, providing unprecedented opportunities for Earth observation. Among the various tasks in RS image analysis, the acquisition of three-dimensional structural information is of great importance. Unlike traditional 2D imagery, 3D representations offer a comprehensive understanding of target geometry, which is essential for applications ranging from disaster monitoring and urban planning to military intelligence [1]. Specifically, for high-value targets such as aircraft, obtaining their three-dimensional models from remote sensing images is of great significance for research in downstream fields such as target recognition, electromagnetic characteristic analysis, and echo simulation.

Despite its significance, recovering 3D geometry from a single viewpoint remains a classic ill-posed problem in computer vision and remote sensing. The primary challenge stems from the inherent depth ambiguity and severe self-occlusion; for any given 2D observation, an infinite number of 3D configurations can produce the same projection [2]. While multi-view stereo techniques have matured significantly, they rely heavily on dense multi-angle observations [3]. In the remote sensing domain, however, such data are often a luxury. Due to the fixed orbital trajectories and limited revisit cycles of satellite platforms, many high-interest targets are captured from only a single perspective, particularly in time-sensitive monitoring scenarios [4]. Consequently, there is an urgent practical need for robust single-view 3D reconstruction methods that can imagine the occluded parts of a target with high fidelity.

The classification of aircraft in remote sensing images is also an important task, which has considerable research significance for applications such as airport monitoring, aviation security, and national defense construction [5]. Traditional methods typically involve manual extraction of image features, combined features, and classification. However, the design of artificial image features mainly relies on relevant professional knowledge and experience, which although has good interpretability, lacks generalization ability in complex environments and is difficult to meet the requirements of actual classification tasks. In recent years, classification methods based on deep learning have been widely applied in this field and have achieved good accuracy rates. Nevertheless, neural networks are black-box methods and do not have intuitive interpretability. Current neural networks also have limitations in generalization and computational cost.

Given the limitations of the above methods, in this paper we propose a pioneering approach for remote sensing image interpretation, which is to integrate the two tasks of 3D target reconstruction and classification into the same framework. On the one hand, single-view 3D reconstruction is an ill-posed problem with incomplete information, and it requires the utilization of prior knowledge about this type of target. Therefore, in the process of single-view reconstruction, there is generally an implicit classification and recognition process of the target. On the other hand, the result of 3D reconstruction is the 3D model of the target, which itself is a target feature with sufficient information and can be used for the target classification task. Therefore, integrating these two tasks into the same engineering framework is logically consistent.

For the 3D reconstruction process, we construct a signed distance field (SDF) generation network to implicitly learn the prior information in the aircraft model library, and implement the inversion process from the 2D image to the SDF representation by a differentiable rendering method. Unlike explicit meshes or point clouds, SDF represents geometry as a continuous field. The SDF value at any point in space represents the distance from the point to the surface boundary, and the sign indicates whether the point is inside (−) or outside (+) of the shape, so SDF implicitly represents the surface of a 3D model as the zero-level set of a scalar field. In this way, SDF can preserve the smooth surface of a 3D model without taking up too much space, and different 3D models can be switched by controlling the input of the neural network.

The SDF constructed by the generative network can generate the corresponding simulation image through the differentiable rendering process. The image is compared with the real image at the pixel level to calculate the loss function, and then the input latent space vector and pose parameters are optimized through backpropagation. Finally, the 3D model which conforms to the real remote sensing image is obtained. More importantly, the optimal input vector obtained by optimization, or what we call latent space encoding, can also be used as a feature vector of the image, which is coupled into the classification network through the attention module to enhance the accuracy and robustness of the classification process. The engineering details will be elaborated in the methodology section. The main contributions of this work are summarized as follows:

• This study proposes a joint framework for aircraft target classification and reconstruction in optical remote sensing images, providing a new engineering approach for aircraft classification and single-view 3D reconstruction.
• We introduce the Signed Distance Field (SDF) representation to single-view 3D reconstruction for optical remote sensing imagery, designing a novel joint optimization pipeline that utilizes dual rendering kernels. Extensive qualitative experiments on open-source datasets and quantitative evaluations on simulated benchmarks demonstrate the reliability and geometric fidelity of the proposed reconstruction process.
• We develop an attention-based fusion mechanism that seamlessly integrates geometric features derived from the 3D reconstruction process with deep semantic features from a standard backbone network. This synergy effectively enhances classification accuracy and robustness against appearance variations. Experimental results confirm that our method achieves competitive or superior performance compared to existing state-of-the-art classification networks on public datasets.

2. Related Work

2.1. 3D Reconstruction

3D reconstruction refers to the process of recovering the three-dimensional representation of a target from single-view or multi-view remote sensing images, a field that has gained increasing attention in recent years. Traditional 3D reconstruction in the remote sensing domain has long relied on photogrammetric pipelines integrating Structure-from-Motion (SfM) and Multi-View Stereo (MVS) techniques [6]. In geographical and remote sensing applications, SfM frameworks have been widely deployed to achieve high-accuracy topographic mapping and target modeling by establishing dense feature correspondences across multiple overlapping images [7].

However, conventional SfM-based methods inherently require dense view sampling and sufficient image overlap to guarantee robust keypoint matching [8]. If the viewpoints are separated by a large baseline or exhibit severe illumination changes, the feature matching process becomes an equally challenging, ill-posed problem [9]. Furthermore, in spaceborne remote sensing, satellite platforms operate under strict orbital constraints and limited revisit frequencies, making it inherently difficult to acquire dense, multi-view imagery of specific sensitive ground targets. The depression angles of available optical satellite images are also subject to physical range limitations, which frequently precludes the capture of complete, omnidirectional visual information. Consequently, while multi-view reconstruction remains constrained by data availability, single-view 3D reconstruction undoubtedly possesses higher practical value and deployment flexibility in real-world remote sensing scenarios.

2.1.1. Explicit Representation Method

The choice of 3D shape representation fundamentally determines the upper bound of reconstruction fidelity and efficiency. Methods in this field can be broadly categorized into four groups according to their output representation: voxel grids, point clouds, meshes, and implicit neural representations. As a natural 3D extension of 2D pixels, voxel representations were among the first to be integrated into deep reconstruction frameworks. 3D-R2N2 [10] is a well-known end-to-end training method based on voxel. It learns a mapping from images of objects to underlying 3D shapes from a large collection of synthetic data. Pix2Vox [11] proposed a context-aware fusion module that adaptively selects high-quality reconstructions from multi-view inputs, achieving significant improvements at 24× the speed of 3D-R2N2. However, voxel memory consumption grows cubically with resolution ($O\left(N^3\right)$). Even with optimized octree structures [12], high-resolution voxel reconstruction remains limited to the ${32}^3$–${64}^3$ range.

Point clouds have also been widely adopted as an intermediate representation, with a flexible topological structure and low computational complexity. PSGN [13] pioneered direct 3D point cloud coordinate prediction from a single image, introducing Chamfer Distance and Earth Mover’s Distance as permutation-invariant loss functions. However, when point clouds are used as the final output of a reconstruction method, they lack explicit connectivity information regarding surface topology, thereby necessitating additional post-processing steps to extract watertight meshes, such as Poisson surface reconstruction or ball pivoting.

Triangle meshes are the most manipulable representation in computer graphics [14], supporting rendering, deformation, and editing. Pixel2Mesh [15] generates meshes by deforming an initial ellipsoid through a graph convolutional network. Pixel2Mesh++ [16] further exploits cross-view information from multiple images to significantly improve the geometric quality of occluded regions. Yanni Wang et al. [17] used a similar template deformation method to reconstruct a mesh from a single-view remote sensing image. However, these methods are constrained by a fixed template topology, limiting their ability to represent objects with varying genus [18].

2.1.2. Neural Implicit Representation Method

In recent years, neural implicit representations for 3D reconstruction have become a promising alternative to traditional methods [19]. Unlike discrete voxels, point clouds, or meshes, implicit representations parameterize the 3D surface as a continuous function defined by a neural network, fundamentally overcoming resolution limits and topological constraints.

Occupancy Networks [20] pioneered the implicit paradigm by defining the 3D surface as the continuous decision boundary of a deep network classifier. This approach supports diverse inputs (single images, noisy point clouds, and coarse voxel grids) and demonstrated for the first time that complex geometry can be encoded at infinite resolution without excessive memory cost. However, occupancy functions carry information only near the surface, leading to vanishing gradients in regions far from the boundary. Subsequently, some new implicit expression methods emerged, among which the most important ones are the NeRF [21] and 3DGS [22]. Although these two implicit representations can be well coupled with differentiable rendering methods [23], they are essentially designed for the task of new perspective synthesis and do not have the ability to extract high-resolution 3D models. Additionally, they also require dense multi-perspective input.

DeepSDF [24] advanced the field by learning continuous SDF with a latent-code-conditioned decoder network, encoding an entire shape category within a single model. Unlike occupancy, SDF provides meaningful gradients throughout the space, with the zero-level set precisely defining the surface. SDF-StyleGAN [25] extended the generative power of StyleGAN2 into the SDF space, enabling high-quality 3D shape generation and style editing.

VolSDF [26] and NeuS [27] integrated SDF with volumetric rendering by modeling volume density as a function of the signed distance. VolSDF introduced a Laplace Cumulative Distribution Function (CDF) mapping from SDF to density, achieving unsupervised disentanglement of shape and appearance. NeuS2 [28] further accelerated the training process while maintaining reconstruction quality. SDFusion [29] and AutoSDF [30] introduced generative priors into the SDF latent space. SDFusion leverages a diffusion model operating in the SDF latent space, supporting multi-modal conditioning from images, text, and partial shapes with weighted combination of different conditions. AutoSDF models the 3D shape distribution as a non-sequential auto-regressive distribution over a discretized latent representation, supporting arbitrary spatially-anchored conditional queries for shape completion, and language-guided generation within a unified framework.

For the specific application of aircraft targets in remote sensing, SDF framework offers three distinct advantages. Firstly, the SDF zero-level set corresponds to the model surface of the target, and the gradient field naturally aligns with the surface normal. We can obtain the mesh of the target from the SDF through a simple post-processing method. Secondly, SDF can construct a differentiable rendering method through the volume density function, and the backpropagation process will not be hindered. Finally, aircraft geometry is characterized by strict aerodynamic symmetry, and the continuous transitional nature of SDF in space is inherently well-suited to encoding such smooth, streamlined forms. Based on the above reasons, our work adopts the SDF representation method to construct a category-level reconstruction model specifically for aircraft targets.

Despite the success of pioneering works in general computer vision or multi-view settings, directly transferring them to single-view remote sensing reconstruction poses a severe challenge. Standard differentiable rendering frameworks directly optimize raw network weights from a simple initialization, which invariably fails under single-view constraints due to the extreme geometric ambiguity and ill-posed nature of the problem. To bridge this gap, our work introduces an architectural departure: we first learn a continuous low-dimensional latent space manifold via pre-training, and subsequently optimize the latent space vectors rather than raw network weights during the rendering process. This manifold constraint naturally injects a powerful geometric regularizer, ensuring plausible 3D asset synthesis even from a single overhead image. The methodology section will provide a more detailed explanation of our reconstruction process.

2.2. Object Recognition in Remote Sensing

Object recognition is a fundamental task in remote sensing image processing [31], serving as a prerequisite for downstream applications such as target detection, semantic segmentation, and 3D reconstruction. Over the past few decades, this field has transitioned from handcrafted feature-based methods to deep learning-based paradigms [32], driven by continuous architectural innovations.

Before the deep learning era, visual recognition primarily relied on handcrafted descriptors, such as SIFT [33] and HOG [34], paired with shallow classifiers like Support Vector Machines (SVMs) [35]. Although effective in constrained settings, these methods suffered from limited representational capacity and generalized poorly to the complex backgrounds and diverse target appearances characteristic of remote sensing imagery.

This bottleneck was mitigated by the convolutional neural network (CNN) revolution. Early landmark architectures including AlexNet [36], VGGNet [37], and GoogLeNet [38] systematically advanced feature extraction capabilities through increased network depth and multi-scale receptive fields. Notably, ResNet [39] introduced deep residual learning to resolve the optimization degradation problem, establishing itself as the de facto backbone for diverse remote sensing visual tasks.

Recently, the community has witnessed another paradigm shift toward capturing global context and achieving open-world generalization. Driven by the self-attention mechanism, Vision Transformers (e.g., Swin Transformer [40], ViTAE [41]) and large-scale Remote Sensing Foundation Models (e.g., SkySense [42]) have redefined visual recognition with robust zero-shot transferability. Concurrently, to alleviate the quadratic computational overhead of Transformers on high-resolution imagery, State Space Models (SSMs) like Mamba [43] have emerged, exemplified by RSMamba [44] and Pan-Mamba [45], achieving linear complexity while preserving long-range dependency modeling. Furthermore, multi-source data fusion [46] and instruction-driven multi-modal frameworks like InstructSAM [47] have pushed the boundaries toward open-vocabulary queries.

Despite these remarkable advances, current remote sensing object recognition paradigms still suffer from fundamental limitations. Most existing methods adopt a purely 2D, end-to-end black-box paradigm that maps input images directly to category labels. Consequently, they exhibit insufficient robustness against severe blurs, occlusions, and multi-view pose variations, while inherently lacking structural interpretability. To bridge this gap, our proposed unified framework couples 3D geometric reconstruction with classification. By leveraging implicit 3D geometric consistency, our method not only achieves satisfactory shape synthesis but also delivers a classification mechanism with enhanced multi-view robustness and explicit geometric interpretability.

3. Methodology

3.1. Overall Framework

The proposed joint framework is designed to reconstruct precise 3D aircraft geometry and determine semantic categories from a single optical remote sensing image. The methodology comprises two primary phases: an offline prior learning stage and an online joint optimization stage.

In the offline stage, we leverage the ShapeNet dataset to train a SDF generator. An auto-decoder architecture is employed to establish a continuous, differentiable latent manifold, where each latent code $z\in {\mathbb{R}}^k$ uniquely represents an aircraft’s topology. This latent space captures geometric structural priors essential for reconstructing complex shapes from limited 2D observations.

In the online stage, target interpretation is formulated as an “analysis-by-synthesis” optimization problem. Given an initial mask $M_{init}$ and pose ${\theta }_{init}$ derived from classical computer vision techniques, the framework iteratively refines the latent code z and pose parameters $\theta$ through a differentiable rendering pipeline. The process minimizes the discrepancy between the rendered projection and observed image features.

Ultimately, the optimized latent code $z^{\ast }$ serves a dual purpose: In the 3D reconstruction task, it serves as the generation seed for the SDF decoder, used to reconstruct high-fidelity 3D meshes. At the same time, it can also be regarded as a low-dimensional feature representation of the target, which is integrated into the classification network through a gated attention module to determine the specific category of the aircraft.

While conventional 2D remote sensing classification is easily perturbed by superficial texture variations, extreme viewpoints, or shadow interferences, our framework mitigates these vulnerabilities through a bidirectional regularization. On one hand, the classification branch is grounded in the target’s intrinsic 3D physical topology, which inherently enhances recognition robustness against environmental noise. On the other hand, the implicit geometric refinement is regularized by class-aware semantic boundaries, preventing the rendering optimization from overfitting to localized 2D image anomalies. This coupling ensures that the reconstructed geometry and semantic labels are physically and logically consistent. The framework of the entire process is shown in Figure 1.

3.2. Signed Distance Field

3D geometry representation is fundamental to reconstruction and recognition. While traditional methods like voxels, point clouds, and meshes are widely used, they face limitations in high-resolution remote sensing: voxel grids suffer from $O\left(N^3\right)$ memory complexity, point clouds lack explicit topological connectivity, and meshes often require fixed topologies. Conversely, Neural Implicit Fields represent 3D shapes as continuous level sets, offering memory efficiency and full differentiability. The core of our geometric representation is the Signed Distance Field. Formally, for a given 3D shape, the SDF generation network can be regarded as a function $f(·)$, which maps a spatial coordinate $p=(x,y,z)\in {\mathbb{R}}^3$ to a scalar value $s\in \mathbb{R}$, representing the signed distance from p to the closest surface $\mathcal{S}$ of the object. The mathematical definition is expressed as:

$$f\left(p\right)=\mathrm{sgn}\left(p\right)·\underset{q\in \mathcal{S}}{\min }{\parallel p-q\parallel }_2$$

(1)

where $f\left(p\right)<0$ denotes the interior of the volume, $f\left(p\right)>0$ the exterior, and $f\left(p\right)=0$ precisely defines the object’s boundary.

As illustrated in Figure 2, this representation yields several critical advantages for aircraft reconstruction. First, the surface model can be efficiently extracted as the zero-level set using the Marching Cubes algorithm. Second, the gradient of the SDF with respect to the input coordinates, ${\nabla }_{p}f\left(p\right)$, is inherently aligned with the surface normal direction. Third, the SDF is completely smooth and continuous, which makes it easy to optimize using the differentiable volume rendering method.

The SDF generation network is based on a Multi-Layer Perceptron (MLP) architecture. The network takes a K-dimensional latent vector z, which encodes the specific geometric identity of an aircraft, and a 3D coordinate p as joint inputs. The architecture consists of multiple fully connected layers followed by a Tanh activation function at the output layer to yield the final SDF value. To preserve high-frequency geometric details and prevent the vanishing of input signals in deeper layers, a skip connection is implemented. Specifically, the input vector $[z;p]$ is concatenated to the hidden features of the middle layer, allowing the network to re-anchor its feature transformation to the original coordinate space. This structure ensures that the learned manifold is continuous and smooth, enabling precise latent space optimization.

3.3. Cluster-Aware Prior Learning

To construct a robust and semantically-informed geometric prior, we utilize a subset of the ShapeNet dataset, specifically focusing on the Airplane category (ID: 02691156), from which approximately 1200 high-quality mesh models are selected. Given that raw CAD models often contain non-manifold geometries such as self-intersections or open boundaries that inherently destabilize SDF computation, we employ ManifoldPlus to perform watertight processing on all meshes. This preprocessing ensures that the learned implicit field is unambiguously defined across both interior and exterior spatial regions.

As illustrated in Figure 3, the processed models are categorized into four distinct categories based on their representative morphological characteristics in typical remote sensing scenarios: airliners featuring high-aspect-ratio slender fuselages and swept-back wings; transport aircraft characterized by high-straight-wing configurations and robust airframes; fighters distinguished by compact structures and aerodynamic delta wings; and private planes comprising smaller-scale aircraft with simplified configurations.

Unlike conventional implicit representation learning that focuses exclusively on geometric reconstruction fidelity, our approach establishes a structured latent manifold by embedding category-specific semantic anchors. For each morphological category, we pre-define a unique cluster center $C_k$ within the K-dimensional latent space to serve as a topological anchor. Crucially, this coarse clustering is not a substitute for the final classification task; instead, it acts as a structural-semantic regularizer designed to enhance the discriminative capacity of the manifold. The offline training process is governed by a joint loss function formulated as:

$${\mathcal{L}}_{total}={\mathcal{L}}_{rec}+\lambda {\mathcal{L}}_{clu}$$

(2)

$${\mathcal{L}}_{clu}={\parallel z_i-C_k\parallel }_2^2$$

(3)

where ${\mathcal{L}}_{rec}$ represents the reconstruction loss, defined as the mean squared error (MSE) between the network’s predicted SDF values and the ground-truth distances derived from the watertight meshes. The second term, ${\mathcal{L}}_{clu}$, denotes the clustering loss, which penalizes the Euclidean distance between the latent code $z_i$ of an individual instance and its corresponding category anchor $C_k$.

The core motivation for this semantic-guided prior structuring is to address the severe ambiguity inherent in single-view remote sensing interpretation. Standard neural implicit spaces optimize purely for geometric reconstruction fidelity, resulting in a highly continuous but unconstrained manifold where geometric boundaries between different aircraft categories are blurry. In single-view remote sensing scenarios, images often suffer from severe top-down perspective compression and loss of critical 3D cues. Without a structured prior, the online optimization trajectory can easily drift, trapped in local minima where the network fits a 2D image silhouette by generating highly distorted or structurally impossible 3D shapes.

By embedding morphology-clustered anchors during the offline pre-training phase, the neural network is forced to optimize the implicit manifold structure through a process of geometric-semantic alignment. It preserves a continuous latent space necessary for smooth shape interpolation and rendering, while simultaneously dispersing different aircraft categories toward distinct, well-separated directions in the high-dimensional space. This strategy guarantees that the optimized latent code $z^{\ast }$ is inherently dual-purpose: it remains generative for high-fidelity surface reconstruction and highly discriminative for downstream target classification. This learned manifold serves as a dynamic, class-aware search space that effectively bridges the generative and discriminative stages of our framework.

3.4. Image Preprocessing and Initial Pose Estimation

The optimization landscape of differentiable rendering is inherently non-convex. To ensure the convergence of the subsequent latent code refinement, we adopt a coarse-to-fine strategy by estimating a reliable initial 6-DoF pose ${\theta }_{init}=[R_{init},T_{init}]$ and scale $S_{init}$ via the method of complex moments.

3.4.1. Mask Extraction

For an input optical remote sensing image I, we first extract the binary target mask $M_{obs}$ through a widely used segmentation network SAM. The mask is defined over a normalized coordinate grid $(x,y)\in {[-1,1]}^2$. The zero-order moment $m_{00}$, representing the target area, is calculated as $m_{00}={\sum }_{x,y}{M}_{obs}(x,y)$. The centroid $(c_x,c_y)$, which serves as the initial horizontal translation $T_{init}$, is determined by:

$$c_x={\displaystyle \frac{1}{m_{00}}}\sum _{x,y}x·M_{obs}(x,y),c_y={\displaystyle \frac{1}{m_{00}}}\sum _{x,y}y·M_{obs}(x,y)$$

(4)

3.4.2. Scale and Pose Initialization via Complex Moments Analysis

Before detailing the optimization loop, it is essential to justify our choice of pose initialization. Generally, three strategies can be considered for pose estimation in this context:

• Learning-based approach: A lightweight pose estimation network could be trained to predict initial parameters. However, this necessitates a substantial amount of labeled data, increasing the annotation workload and potentially compromising the system’s robustness when encountering out-of-distribution aircraft types.
• Exhaustive search: We could employ a “global search” mentality by initiating the optimization from a large number of densely sampled poses. While this increases the probability of finding the global optimum, the resulting computational overhead for online optimization becomes prohibitive for real-time applications.
• Analytical-Optimization Hybrid (Our Approach): We adopt a strategic middle-ground. By leveraging the inherent geometric regularity of aircraft targets—such as axial symmetry and high-contrast structural features—we utilize the analytical method of complex moments for coarse initialization. This is followed by a fine-tuning process within the differentiable rendering loop.

This hybrid strategy ensures a high-quality starting point for the non-convex optimization while maintaining computational efficiency and minimizing the dependency on extensive labeled datasets.

To achieve a robust initialization for aircraft with varying aspect ratios, we utilize second-order central moments ${\mu }_{20},{\mu }_{02}$, and ${\mu }_{11}$. The initial scale $S_{init}$ is derived from the eigenvalues of the covariance matrix of the mask’s spatial distribution:

$$S_{init}=\sqrt{{\displaystyle \frac{({\mu }_{20}+{\mu }_{02})+\sqrt{{({\mu }_{20}-{\mu }_{02})}^2+4{\mu }_{11}^2}}{2}}}$$

(5)

The heading angle $\varphi$ is estimated using complex moments to handle the symmetry of the aircraft’s longitudinal axis. Let $z=(x-c_x)+i(y-c_y)$ be the complex representation of the relative coordinates. We compute the second-order and third-order complex moments as:

$$C_{20}=\sum z^2·M_{obs}\left(z\right),C_{30}=\sum z^3·M_{obs}\left(z\right)$$

(6)

The principal orientation angle ${\theta }_{base}$ is derived from the phase of $C_{20}$ as ${\theta }_{base}={\displaystyle \frac{1}{2}}\arg \left(C_{20}\right)$. However, ${\theta }_{base}$ possesses a $\pi$ ambiguity (i.e., inability to distinguish the nose from the tail). To resolve this, we leverage the asymmetry captured by the third-order moment $C_{30}$. We evaluate the projection of $C_{30}$ onto the candidate angles $\alpha \in \{{\theta }_{base},{\theta }_{base}+\pi \}$:

$$P\left(\alpha \right)=\mathrm{Re}(C_{30}·e^{-i3\alpha })$$

(7)

The final heading angle $\varphi$ is selected as the $\alpha$ that maximizes the magnitude of the asymmetric projection $P\left(\alpha \right)$, ensuring that the initial 3D model is correctly oriented along the aircraft’s forward vector. This analytical initialization significantly reduces the search space for the subsequent differentiable rendering-based refinement.

3.5. Differentiable Volume Rendering

The core challenge in reconstructing 3D aircraft geometry from a single image is bridging the gap between the implicit SDF and the discrete 2D pixel grid. Traditional rendering techniques are often non-differentiable at surface boundaries, making it difficult to backpropagate gradients from pixel-wise losses back to the latent code z and pose $\theta$. Therefore, we adopt a differentiable volume rendering [48] framework, as shown in Figure 4.

Given a camera ray $\mathbf{r}\left(t\right)=\mathbf{o}+t\mathbf{d}$, where $\mathbf{o}$ is the ray origin and $\mathbf{d}$ is the unit direction, we sample N points along the ray within a predefined range $[t_{near},t_{far}]$. For each sampled point ${\mathbf{p}}_i=\mathbf{r}\left(t_i\right)$, the decoder $f_{\theta }$ predicts the SDF value $s_i$. To render the pixel intensity (or mask value) $C\left(\mathbf{r}\right)$, we convert the SDF values into volume densities ${\sigma }_i$ [26] and apply the standard volume rendering equation:

$$C\left(\mathbf{r}\right)={\displaystyle \sum _{i=1}^N}{T}_i{\alpha }_i,T_i={\displaystyle \prod _{j=1}^{i-1}}(1-{\alpha }_j)$$

(8)

where ${\alpha }_i=1-\exp (-{\sigma }_i{\delta }_i)$ is the opacity of the i-th sample, ${\delta }_i=t_{i+1}-t_i$ is the step size, and $T_i$ represents the accumulated transmittance.

To address the coupled optimization challenge between camera pose estimation and implicit scene reconstruction in orthographic neural rendering, we develop a robust two-stage coarse-to-fine joint optimization framework, which effectively decouples pose alignment and latent code refinement to mitigate local optima and enhance both optimization stability and reconstruction fidelity. Correspondingly, we design two different functions to map the SDF values to density values, as shown in Figure 5:

Phase I: Logistic Occupancy Kernel. In the coarse optimization phase, we utilize a Logistic-based density kernel to ensure smooth gradient flow across the object boundaries. Given the inverse scale parameter $\alpha$, the density ${\sigma }_{log}$ is formulated as:

$${\sigma }_{log}\left(s\right)=\alpha ·{\displaystyle \frac{1}{1+e^{s·\alpha }}}$$

(9)

This formulation ensures that as a sampling point moves from the interior ($s<0$) to the exterior ($s>0$), the density smoothly transitions from $\alpha$ to 0.

Phase II: Laplace Occupancy Kernel. For high-precision reconstruction, we transition to a Laplace-based kernel. This kernel provides a sharper transition at the surface interface ($s=0$). Utilizing the scale parameter $\beta$, the density ${\sigma }_{lap}$ is defined as:

$${\sigma }_{lap}\left(s\right)={\displaystyle \frac{1}{\beta }}\left[0.5+0.5·\mathrm{sgn}\left(s\right)·\left(e^{-\frac{\left|s\right|}{\beta }}-1\right)\right]$$

(10)

The intrinsic differences in their mathematical properties lead to distinct optimization behaviors, as analyzed below:

The Logistic kernel is a smoothly descending curve. In the early stages of optimization, when the estimated camera poses and coarse geometry are far from the ground truth, it serves as a global attractor. Traditional hard-threshold rendering often suffers from gradient vanishing or discontinuities because pixels only receive information when a ray physically intersects the current zero-level set. By employing a Logistic occupancy with a moderate scale $\alpha$, we effectively create a wide capture basin, providing continuous error signals that pull the misaligned target out of local minima. This allows the gradients to backpropagate even when the initial model pose is significantly misaligned with the target image, effectively pulling the model toward the correct orientation without getting trapped in local minima caused by sharp edges.

Conversely, the Laplace kernel decays exponentially at a much faster rate. Far from the surface, its gradients vanish rapidly, which would cause optimization stagnation if used prematurely. Once the global pose is anchored, the Laplace kernel’s sharp peak at the $s=0$ interface provides the high-frequency gradient signals necessary for fine-grained structural deformation. This forces the latent code to optimize for a watertight, structurally sound aircraft mesh that strictly adheres to the observed optical boundaries. Figure 6 presents an example to illustrate the changes in the rendered images during these two processes, clearly showing the functional distinction between the two stages.

Critically, the shape-controlling parameter (beta) of the Laplace density kernel is set to be fully trainable throughout the optimization process, allowing the model to adaptively adjust the sharpness of the implicit density field. As the optimization proceeds, the trainable beta parameter decreases gradually, which corresponds to the continuous hardening of the scene density field. To avoid redundant iterations while guaranteeing sufficient detail convergence, we design an adaptive early termination mechanism for this stage: the optimization process will automatically terminate when the iteration count exceeds the preset base steps and the trainable beta parameter drops below a threshold. This termination criterion directly reflects the hardening level of the model’s implicit density field, ensuring that the optimization stops only when the scene details are sufficiently refined and the model converges to a stable high-fidelity reconstruction result. In this stage, the pre-aligned pose parameters are also set to be trainable with a small learning rate for minor fine-tuning.

3.6. Mesh Extraction

Upon the convergence of the differentiable rendering loop, the optimization framework yields the optimal latent code $z^{\ast }$ and the refined spatial pose ${\theta }^{\ast }$. This optimized code $z^{\ast }$ represents a fundamental breakthrough in our architecture: it serves as a compact, unified representation that seamlessly bridges both generative and discriminative tasks without requiring separate feature extraction pipelines.

In the reconstruction branch, $z^{\ast }$ acts as the generative seed. To recover the explicit 3D surface of the aircraft from the continuous Neural Implicit Field, we evaluate the SDF $f_{\varphi }(z^{\ast },p)$ and employ the classic Marching Cubes algorithm.

The principle of Marching Cubes involves discretizing the 3D bounding volume into a high-resolution regular voxel grid. The SDF value is queried at all eight vertices of each voxel. If a sign change occurs between two adjacent vertices (e.g., one vertex is negative/inside the object, and the other is positive/outside), it indicates that the object’s surface intersects the edge connecting them. By applying linear interpolation based on the magnitude of the SDF values at these vertices, the precise coordinates of the zero-crossing points are computed. These points are then connected according to a predefined lookup table to form triangular facets. Consequently, the continuous implicit field is converted into a high-fidelity, watertight explicit 3D mesh.

3.7. Classification Based on Feature Fusion

In the classification branch, the optimized latent code $z^{\ast }\in {\mathbb{R}}^K$ ($K=8$ in our implementation) functions as a low-dimensional, highly discriminative semantic feature vector. To fully leverage the 3D topological priors transmitted from the reconstruction loop without discarding the fine-grained texture details inherent in the 2D image, we propose a Gated Attention Fusion mechanism.

Specifically, the input optical image is first processed by a pre-trained ResNet-50 backbone. After applying global adaptive average pooling, we obtain a high-dimensional visual feature vector $F_{img}\in {\mathbb{R}}^D$ ($D=2048$). To bridge the dimensional discrepancy between the geometric prior and the visual features, the latent code $z^{\ast }$ undergoes a linear projection to align with the visual feature space:

$$F_z=W_{proj}·z^{\ast }$$

(11)

where $W_{proj}\in {\mathbb{R}}^{D\times K}$ is the projection weight matrix, yielding the aligned geometric feature $F_z\in {\mathbb{R}}^D$.

Rather than employing a naive feature concatenation, which often leads to channel redundancy and suboptimal gradient flows, we fuse $F_{img}$ and $F_z$ through an adaptive gating module. The attention gate $g\in {(0,1)}^D$ is dynamically calculated based on the content of both feature streams:

$$g=\sigma (W_{img}{F}_{img}+W_z{F}_z+b)$$

(12)

where $W_{img},W_z\in {\mathbb{R}}^{D\times D}$ are learnable weight matrices, $b\in {\mathbb{R}}^D$ is the bias term, and $\sigma (·)$ denotes the Sigmoid activation function.

The generated gate g acts as a channel-wise selector, determining the relative importance of visual textures versus 3D geometric structures for each feature channel. The final fused representation $F_{fused}$ is computed via a convex combination of the two feature vectors:

$$F_{fused}=g\odot F_{img}+(1-g)\odot F_z$$

(13)

where ⊙ represents the element-wise Hadamard product. This fused feature is subsequently fed into a linear classifier (MLP) to output the final categorical logits, as illustrated in Figure 7.

Unlike traditional 2D Convolutional Neural Networks that typically rely on superficial image textures or background biases, the proposed gating mechanism acts as a dynamic information filter. When an image suffers from severe shadows or partial occlusions (rendering the 2D textures unreliable), the network can adaptively lower the gate values g, thereby relying more heavily on the robust 3D topological prior $(1-g)\odot F_z$ to make decisions. This consistency between geometry and semantics guarantees that our system maintains high recognition robustness under complex atmospheric and lighting conditions in optical remote sensing scenarios.

4. Experiments and Results

We conduct extensive experiments to validate the proposed joint reconstruction and classification framework. Section 4.1 describes the offline training of the SDF generative model, including dataset preprocessing, network architecture, and the latent space continuity verification. Section 4.2 evaluates the complex-moment-based pose initialization method. Section 4.3 presents reconstruction results on real-world remote sensing images. Section 4.4 evaluates reconstruction quantitatively on simulated images with ground truth geometry and compares against baseline methods. Section 4.5 reports classification benchmarks on the MTARSI dataset. Section 4.6 presents the ablation studies on both the key modules in the reconstruction process and the reconstruction feature fusion branch.

4.1. Offline Training of the SDF Generative Model

We utilize a subset of the ShapeNet dataset [49], specifically focusing on the Airplane category (ID: 02691156). After preprocessing, each model is normalized into a unit sphere with a diameter of 2.0. To sample the signed distance field, we adopt a hybrid point sampling strategy: for each model, we uniformly sample $N=\mathrm{100,000}$ points within the bounding sphere, among which the first $B=\mathrm{30,000}$ are near-surface points ($\left|s\right|<0.05$) to capture fine geometric details, and the remaining points cover the free space to provide regularization. The signed distance values are computed by projecting each query point to the nearest surface triangle.

The network is trained with the truncated signed distance (TSDF) loss:

$${\mathcal{L}}_{\mathrm{rec}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left[{\mathbb{I}}_{|s_i|<\delta }·{(s_i-{\widehat{s}}_i)}^2+{\mathbb{I}}_{|s_i|\ge \delta }·\max (0,|s_i-{\widehat{s}}_i|-\delta )\right],$$

(14)

The truncation threshold $\delta =0.1$ prevents gradient domination by distant points. The joint training objective also includes the clustering regularization loss ${\mathcal{L}}_{\mathrm{clu}}$ described in Section 3.3. The total loss is $\mathcal{L}={\mathcal{L}}_{\mathrm{rec}}+\lambda {\mathcal{L}}_{\mathrm{clu}}$. We train the network for 3000 epochs using the Adam optimizer with an initial learning rate of $1\times {10}^{-3}$, a batch size of 64, and a linear learning rate decay to $1\times {10}^{-6}$. Training takes approximately 24 h on a single RTX 4090 GPU. It can be seen from Figure 8 that the SDF generation network has fully converged after approximately 1000 epochs.

A critical property of the learned SDF generative model is the continuity of its latent manifold. Intuitively, the latent space of the SDF decoder is a high-dimensional curved surface embedded in ${\mathbb{R}}^K$, where each point on this surface corresponds to a distinct 3D shape. During training, the ShapeNet airplane models serve as anchor points that force the manifold to pass through the geometric configurations represented by each training sample. Between these anchor points, the network must learn smooth transitions to minimize the reconstruction loss across the entire training set, thereby creating a continuous surface that interpolates between known shapes.

The continuity of this manifold is of fundamental importance to our approach. It guarantees that for any target aircraft depicted in the input image—even if its exact 3D model was never seen during training—there exists a latent code $z^{\ast }$ on the learned manifold whose decoded shape approximates the observed geometry. This is what enables the differentiable rendering optimization to generate a plausible 3D reconstruction rather than merely retrieving the closest match from a discrete database. Conversely, if the latent space were discontinuous, the optimization would collapse into a nearest-neighbor template matching scheme.

To verify this property empirically, we perform linear interpolation between two latent codes from different aircraft categories. Specifically, we compute $z\left(t\right)=(1-t)·z_A+t·z_B$ for $t\in [0,1]$ at 8 evenly spaced steps, decode each interpolated code through the SDF network, and extract the zero-level set via Marching Cubes.

As shown in Figure 9, the interpolated meshes exhibit plausible intermediate geometries: the wing sweep angle gradually decreases, the fuselage elongates, and the tail configuration transitions smoothly. This confirms that the SDF decoder has learned a continuous and well-regularized latent space, which is essential for the downstream online optimization.

4.2. Test of the Pose Estimation Method

To provide a reliable starting point for the differentiable rendering optimization, we estimate the initial 6-DoF pose ${\theta }_{\mathrm{init}}=[R_{\mathrm{init}},T_{\mathrm{init}}]$ and scale $S_{\mathrm{init}}$ using the method of complex moments described in Section 3.4. The centroid $(c_x,c_y)$ of the mask $M_{\mathrm{obs}}$ provides the in-plane translation. The principal orientation is obtained from the second-order complex moment $C_{20}$, and the heading ambiguity ($\pi$ ambiguity) is resolved using the third-order moment $C_{30}$.

We evaluate the quality of the pose initialization on 100 random samples. As shown in Figure 10, in most of the samples, our method is able to accurately estimate the direction of the plane. The estimated heading direction is within $\pm {30}^{\circ }$ of the ground truth in 84% of cases, confirming that the complex-moment method provides a sufficiently accurate initialization for subsequent fine-tuning.

4.3. Reconstruction on Real-World Remote Sensing Images

The evaluation dataset is constructed based on the Multi-type Aircraft of Remote Sensing Images (MTARSI) dataset [50], a widely recognized open-source benchmark for fine-grained aircraft classification in optical remote sensing. Figure 11 shows some images from the dataset. In this study, real remote sensing image patches covering eight representative aircraft categories were selected from this benchmark. To further enrich the dataset diversity and simulate complex imaging environments, data augmentation was comprehensively applied to these selected patches. Specifically, a series of geometric and radiometric transformations were performed, including rotation, cropping, blurring, and the addition of random noise.

For each real-world input image I, we jointly optimize the latent code z and the camera pose $\theta$ to minimize the silhouette loss between the rendered image $\widehat{I}$ and the observed mask $M_{\mathrm{obs}}$. The optimization proceeds in two stages as described in Methodology: Phase I (coarse alignment, 200 iterations, Logistic kernel, learning rate $1\times {10}^{-2}$) followed by Phase II (fine-grained refinement, up to 1000 iterations, Laplace kernel with adaptive $\beta$, learning rate $1\times {10}^{-3}$).

Figure 12 is the rendered image sequence during the latent vector optimization process. It can be seen that the latent vector used as the starting point for optimization produces an image that is quite different from the original image. As the optimization process continues, the generated SDF of the latent space gradually approaches the actual three-dimensional structure of the target, and thus the rendered images will become increasingly similar to the real ones.

From the reconstruction results shown in Figure 13, our method can completely restore the 3D models of various aircraft. Even if the tails and engines of some images are damaged during the segmentation process, our method still provides complete aircraft models based on prior knowledge.

4.4. Reconstruction on Simulated Dataset

Since ground-truth 3D models are unavailable for real satellite imagery, to enable quantitative comparison with state-of-the-art single-view reconstruction methods, we construct a simulated dataset by rendering ShapeNet airplane models from random viewpoints. For each of the 1200 watertight meshes, we generate 8 rendered views with varying azimuth angles, totaling 9600 simulated images with perfect ground truth for both 3D geometry and camera pose. We compare our method against three 3D reconstruction baselines: AtlasNet, Pix2Vox and InstantMesh. The visualized results of the reconstruction can be seen in Figure 14. All baselines are trained and evaluated on the same simulated dataset. We report three standard metrics: Chamfer Distance (CD), Normal Consistency (NC), and F-Score at threshold 0.01. To facilitate the calculation of parameters, we converted the results of all methods into point cloud format, and the number of sampling points was uniformly set to 10,000.

Chamfer Distance (CD) measures the average bidirectional nearest-neighbor distance between the predicted point cloud $\mathcal{P}$ and the ground truth point cloud $\mathcal{Q}$:

$$\mathrm{CD}(\mathcal{P},\mathcal{Q})={\displaystyle \frac{1}{\left|\mathcal{P}\right|}}\sum _{p\in \mathcal{P}}\underset{q\in \mathcal{Q}}{\min }{\parallel p-q\parallel }_2+{\displaystyle \frac{1}{\left|\mathcal{Q}\right|}}\sum _{q\in \mathcal{Q}}\underset{p\in \mathcal{P}}{\min }{\parallel q-p\parallel }_2$$

(15)

It evaluates overall geometric similarity with high computational efficiency. Lower values indicate better performance. In our experiments, CD values are reported in units of ${10}^{-3}$ for readability.

F-score balances reconstruction precision and recall using a predefined distance threshold $\tau$. A predicted point is counted as a true positive (TP) if its nearest ground truth neighbor is within $\tau$. The F-score is defined as:

$$P={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}},R={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}},\mathrm{F}\text{-}\mathrm{score}=2·{\displaystyle \frac{P·R}{P+R}}$$

(16)

We report the F-score at a threshold $\tau$ set to 1% of the target bounding box diagonal, where higher values indicate better performance.

Normal Consistency (NC) evaluates the preservation of high-frequency geometric details by measuring the alignment between the surface normals of the reconstructed 3D model and the ground truth. For the predicted point cloud $\mathcal{P}$ and ground truth $\mathcal{Q}$, NC is defined as the average absolute dot product of the normals for nearest-neighbor pairs:

$$\mathrm{NC}(\mathcal{P},\mathcal{Q})={\displaystyle \frac{1}{|\mathcal{P}|}}\sum _{p\in \mathcal{P}}|{\mathbf{n}}_p·{\mathbf{n}}_q|+{\displaystyle \frac{1}{|\mathcal{Q}|}}\sum _{q\in \mathcal{Q}}|{\mathbf{n}}_q·{\mathbf{n}}_{p^{\ast }}|$$

(17)

where ${\mathbf{n}}_p$ and ${\mathbf{n}}_q$ denote the unit surface normals at point $p\in \mathcal{P}$ and its nearest neighbor $q\in \mathcal{Q}$, respectively. Similarly, ${\mathbf{n}}_{p^{\ast }}$ is the normal at the nearest neighbor $p^{\ast }\in \mathcal{P}$ for point $q\in \mathcal{Q}$. The NC value ranges from 0 to 1, with 1 indicating perfect normal alignment. Higher values denote superior geometric fidelity and surface smoothness.

As shown in

Table 1. Quantitative reconstruction comparison with Pix2Vox on the simulated dataset. Best results in bold. ↑ and ↓ indicate higher and lower values are preferred, respectively.

ID	F-Score (↑)		CD ($\times {10}^{-3},\downarrow$)		NC (↑)
	Pix2Vox	Ours	Pix2Vox	Ours	Pix2Vox	Ours
1	0.2788	0.6583	100.7	37.6	0.5588	0.8499
2	0.2427	0.5292	127.7	55.9	0.5319	0.7258
3	0.3297	0.6939	88.8	33.6	0.5876	0.8434
4	0.2358	0.4800	130.9	52.2	0.5474	0.6477
5	0.2288	0.7211	108.0	33.0	0.5367	0.8364
6	0.1968	0.3153	116.6	76.4	0.5328	0.6954
Mean	0.2521	0.5663	112.1	48.1	0.5492	0.7665

,

Table 2. Quantitative reconstruction comparison with AtlasNet on the simulated dataset. Best results in bold. ↑ and ↓ indicate higher and lower values are preferred, respectively.

ID	F-Score (↑)		CD ($\times {10}^{-3},\downarrow$)		NC (↑)
	AtlasNet	Ours	AtlasNet	Ours	AtlasNet	Ours
1	0.6196	0.6583	39.7	37.6	0.8458	0.8499
2	0.5918	0.5292	41.4	55.9	0.6950	0.7258
3	0.5971	0.6939	40.9	33.6	0.8131	0.8434
4	0.3494	0.4800	55.1	52.2	0.6536	0.6477
5	0.5316	0.7211	45.8	33.0	0.8200	0.8364
6	0.3384	0.3153	60.5	76.4	0.7333	0.6954
Mean	0.5046	0.5663	47.2	48.1	0.7601	0.7665

and

Table 3. Quantitative reconstruction comparison with InstantMesh on the simulated dataset. Best results in bold. ↑ and ↓ indicate higher and lower values are preferred, respectively.

ID	F-Score (↑)		CD ($\times {10}^{-3},\downarrow$)		NC (↑)
	InstantMesh	Ours	InstantMesh	Ours	InstantMesh	Ours
1	0.4520	0.6583	39.3	37.6	0.8550	0.8499
2	0.4088	0.5292	59.1	55.9	0.7170	0.7258
3	0.8248	0.6939	32.7	33.6	0.8285	0.8434
4	0.4616	0.4800	43.8	52.2	0.7317	0.6477
5	0.7288	0.7211	35.0	33.0	0.8375	0.8364
6	0.1296	0.3153	106.9	76.4	0.6185	0.6954
Mean	0.5008	0.5663	52.8	48.1	0.7647	0.7665

, our proposed method achieves the highest overall reconstruction quality, yielding the top average F-Score (0.5663) and outperforming Pix2Vox (0.2521), AtlasNet (0.5046), and the recent baseline InstantMesh (0.5008).

Specifically, Table 1 highlights a significant and consistent improvement of our method over Pix2Vox across all three metrics, while Table 2 shows that our framework is highly competitive with or superior to AtlasNet in most cases. For instance, in Normal Consistency, our method achieves an average of 0.7665, which is slightly higher than AtlasNet (0.7601) and significantly surpasses Pix2Vox (0.5492). Our method also maintains a competitive Chamfer Distance compared to AtlasNet.

Furthermore, Table 3 provides a sample-by-sample comparison with InstantMesh. Although InstantMesh exhibits strong performance on specific well-posed geometries (such as higher F-Scores on Samples 3 and 5), it suffers from severe performance degradation when encountering highly ambiguous structures. For example, on Sample 6, our framework remarkably outperforms InstantMesh. On average, our approach yields a more favorable Chamfer Distance of 48.1, compared to 52.8 for InstantMesh.

These collective results across multiple baselines firmly validate that our optimization-based paradigm, reinforced by the dual-kernel rendering prior, offers superior robustness and geometric faithfulness for single-view 3D reconstruction.

4.5. Image Classification Experiment

We evaluate the classification performance of the proposed framework on the MTARSI dataset. The classification branch takes the optimized latent code $z^{\ast }$ as input to a lightweight MLP classifier, which predicts the final aircraft category.

4.5.1. Cluster Analysis of the Latent Vectors

To demonstrate the relationship between the latent vectors and target categories, we apply Principal Component Analysis (PCA) to reduce the high-dimensional latent vectors to 2D for visualization. The first principal component accounts for 40% of the total variance, while the second principal component accounts for 17%. The left panel shows the distribution of individual samples, with dashed ellipses representing the 95% confidence intervals of each class. The right panel presents the mean position of each class with standard deviation error bars. As can be seen from Figure 15, the clustering results demonstrate that the model effectively captures class-specific characteristics while revealing inherent similarities between visually similar aircraft categories. The two types of fighter jets have similar shapes and their clustering centers are relatively close. The fuselages of the Boeing series of passenger aircraft and the P-3 are also quite similar, and their distribution in the latent space also overlaps, which is expected given the morphological similarities for closely related aircraft types.

4.5.2. Comparison with State-of-the-Art

To comprehensively evaluate the performance of the proposed method, we conduct extensive experiments on the MTARSI dataset. The detailed sample distribution across the eight distinct aircraft categories for both the training and test splits is summarized in

Table 4. MTARSI dataset distribution across 8 aircraft categories.

Category	Training	Test	Total
B-52	192	48	240
Boeing	192	48	240
C-130	288	72	360
C-17	192	48	240
C-5	288	72	360
F-16	224	56	280
F-18	192	48	240
P-3	128	32	160
Total	1696	424	2120

.

We compare our method against standard image classification baselines. All classifiers are evaluated using 5-fold cross-validation with identical train/validation splits (StratifiedKFold, seed = 42).

We adopt four standard metrics for performance evaluation: Accuracy (Acc.), Precision (Prec.), Recall (Rec.), and F1-Score. Let TP (True Positive), TN (True Negative), FP (False Positive), and FN (False Negative) denote the four elements of the confusion matrix. The metrics are defined as follows:

• Accuracy measures the overall proportion of correctly classified samples:

  $$\mathrm{Acc}.={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$$

  (18)
• Precision evaluates the fraction of true positives among all samples predicted as positive:

  $$\mathrm{Prec}.={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

  (19)
• Recall assesses the fraction of true positives correctly identified among all actual positives:

  $$\mathrm{Rec}.={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

  (20)
• F1-Score is the harmonic mean of Precision and Recall, balancing both metrics:

$$\mathrm{F}1=2·{\displaystyle \frac{\mathrm{Prec}.·\mathrm{Rec}.}{\mathrm{Prec}.+\mathrm{Rec}.}}$$

(21)

As shown in

Table 5. Classification performance comparison on MTARSI. Metrics: Accuracy (%), Precision (%), Recall (%), F1-Score.

Method	Acc. (%)	Prec. (%)	Rec. (%)	F1
ResNet-18	93.77 ± 1.53	91.96	91.41	0.916
YOLOv11n-cls	96.04 ± 2.18	96.47	96.01	0.962
ViT-Base	96.42 ± 1.51	96.40	95.45	0.959
EfficientNetV2-S	97.55 ± 0.72	97.86	97.36	0.975
RSMamba	95.85 ± 0.37	96.04	95.73	0.958
EAM	95.34 ± 0.72	95.67	95.50	0.955
Ours	97.88 ± 0.9	98.38	97.57	0.979

, our full method achieves the highest classification accuracy of 97.88%, surpassing the strongest 2D baseline (97.55%), while RSMamba demonstrates the most stable performance with the lowest standard deviation. Notably, our method achieves this while simultaneously producing 3D reconstructions, offering a unique advantage in interpretability and geometric grounding that pure classifiers cannot provide.

To further look into the detailed classification performance across different aircraft categories, the normalized confusion matrix of our full method is illustrated in Figure 16. As depicted in the matrix, our model achieves perfect classification accuracy for several categories, such as B-52, C-130, F-16, and F-18, while maintaining exceptionally low confusion rates across all other classes.

4.6. Ablation Study

The first ablation substituted the dual-kernel rendering method with a uniform Laplacian kernel for differentiable rendering. In the second ablation setup, the clustering prior loss was excluded from the SDF generation network’s training phase, retaining only the basic reconstruction loss. Figure 17 depicts the visual comparisons of these ablation variants.

As summarized in

Table 6. Ablation study: contribution of the dual-kernel rendering strategy to reconstruction quality. ↑ and ↓ indicate higher and lower values are preferred, respectively.

Configuration	CD ($\times {10}^{-2},\downarrow$)	F-Score (↑)	NC (↑)
Full method	4.24	0.688	0.756
W/o dual-kernel rendering	5.53	0.578	0.739
Degradation	+1.29 (30.4%)	−0.110 (16.0%)	−0.017 (2.1%)

, excluding the dual-kernel rendering leads to a distinct performance degradation across all evaluated geometric metrics. Most notably, the Chamfer Distance increases significantly from 4.24 to 5.53, representing a 30.4% worsening in overall geometric accuracy. Similarly, the F-Score drops by 16.0%, from 0.688 to 0.578, which firmly demonstrates that the dual-kernel rendering is crucial for producing structurally meaningful and accurate 3D reconstructions under the highly constrained single-view setting. Furthermore, the Normal Consistency also exhibits a 2.1% decline. This comprehensive degradation confirms that our dual-kernel strategy plays an indispensable role in mitigating geometric ambiguities and ensuring the joint convergence of camera pose and implicit geometry.

Removing the clustering regularization from the joint training objective also leads to a noticeable degradation across all reconstruction metrics, as shown in

Table 7. Ablation study: contribution of the clustering prior to reconstruction quality. ↑ and ↓ indicate higher and lower values are preferred, respectively.

Configuration	CD ($\times {10}^{-2},\downarrow$)	F-Score (↑)	NC (↑)
Full method	4.24	0.688	0.756
W/o clustering prior	7.65	0.464	0.743
Degradation	+3.41 (80.4%)	−0.224 (32.5%)	−0.013 (1.7%)

. The Chamfer Distance increases by 80.4%, and the F-Score drops by 32.5%. The Normal Consistency is only marginally affected, decreasing by 1.7%. The clustering prior enforces that the optimized latent codes remain close to a learnable class centroid in the latent space, which acts as a structured regularization that prevents overfitting to view-specific artifacts. Without this constraint, the latent codes drift toward image-specific details that may be inconsistent across views, resulting in reconstructed shapes that deviate more from the ground truth geometry. Furthermore, the experiment without the clustering prior exhibited a significant decline in convergence speed. Upon reaching the predefined number of epochs, this ablation variant struggled to reconstruct a complete 3D model.

In addition, we evaluate the effect of the reconstruction feature fusion by comparing the full model against a variant that uses only the image backbone features (removing the attention-based fusion with $z^{\ast }$). The results are shown in

Table 8. Ablation study: contribution of the reconstruction feature branch to classification accuracy.

Configuration	Acc. (%)	Prec. (%)	Rec. (%)	F1
Image backbone only	96.7 ± 1.2	97.19	96.53	0.968
Feature fusion method (proposed)	97.88 ± 0.8	98.38	97.57	0.979
Improvement	+1.18	+1.19	+1.04	+0.011

.

The inclusion of reconstruction feature fusion improves classification accuracy by 1.2%, confirming that the geometric features extracted from the 3D SDF optimization provide complementary and discriminative information beyond 2D appearance features.

4.7. Implementation Details

Table 9. Specifications of the key components.

Component	Parameter	Value
SDF decoder	Architecture	6-layer MLP: $11\to 512\to 512\to 520\to 256\to 256\to 1$
	Latent dim K	8
	Regularization	dropout 0.3
	Params	∼0.80 M
Training	Optimizer	Adam, lr ${10}^{-3}$ → ${10}^{-6}$ linear decay
	Batch/Epochs	64 shapes/2500
	Hardware	Single RTX 4090, ∼24 h
Phase I	Density kernel	Logistic with learnable $\alpha$, init 10.0
	Optimized vars	Latent z, translation t, pose $\theta$
	Steps/Batch	500/${10}^3$ rays per step
	Learning rates	$z:$5 × 10−3, $t,\theta :{10}^{-2}$
Phase II	Density kernel	Laplace with learnable $\beta$, init 0.01
	Optimized vars	Latent z
	Steps	2000/${10}^3$ rays per step
	Learning rates	$z:5\times {10}^{-2}$
	Scheduler	ReduceLROnPlateau (factor 0.8, patience 500)
	Termination	stop when LR $< {10}^{-5}$
Mesh extraction	Algorithm	Marching Cubes, ${256}^3$ grid
	Max batch	$2^{18}=\mathrm{262,144}$ points

summarizes the specifications of key components in the proposed pipeline. These parameter information may be helpful for the reproduction and verification of the proposed approach.

5. Discussion

The experimental results demonstrate that the proposed framework provides a robust solution for joint reconstruction and classification. While the classification accuracy is competitive rather than state-of-the-art, the framework offers distinct advantages in interpretability and multi-task synergy. Our ablation studies confirm that the reconstruction features, integrated via the attention module, effectively inform the classification process. Furthermore, unlike features extracted by standard backbones—which often lack explicit geometric constraints—our features are derived from the target’s intrinsic geometric contours inferred by the SDF generator. Consequently, this representation exhibits superior robustness against environmental variations, such as illumination shifts and partial occlusions.

5.1. Analysis of Failure Cases

However, we observe that the proposed workflow may fail under certain challenging scenarios, and a detailed analysis of selected representative failure cases is provided below. First, the fidelity of the 3D reconstruction is highly correlated with the quality of image preprocessing; consequently, the reconstruction accuracy directly determines whether the latent code obtained from feature fusion can positively contribute to the subsequent classification process. As illustrated in Figure 18, both the foreground segmentation and the initial pose estimation modules exhibit a certain probability of failure. Poor segmentation quality typically arises when the low contrast between the foreground and background causes the segmentation model to produce redundant or incomplete target regions for the aircraft. The pose estimation module is more prone to errors, which are multi-factorially influenced by the remote sensing perspective, aircraft morphology, and segmentation quality. Nevertheless, pose errors within a certain threshold can be effectively rectified during the initial stage of optimization without compromising the final reconstruction quality.

Although the dual-kernel rendering method provides robust correction and the latent code optimization exerts a strong regularization effect to mitigate preprocessing errors, the reconstruction pipeline can still yield degraded 3D models when subjected to extreme input conditions. As shown in Figure 19, these failure modes primarily fall into three categories: (1) Low foreground resolution coupled with substantial preprocessing errors severely compromises the fidelity of the reconstructed results. (2) Images captured at large off-nadir angles deviate significantly from the orthographic projection assumption, inducing heavy self-occlusions between critical feature regions, which poses severe challenges to pose alignment and geometric reconstruction. (3) Aircraft morphologies that are rarely seen in the training dataset make it difficult for the model to establish valid correspondences on the learned shape manifold, ultimately resulting in large reconstruction errors.

5.2. Computational and Memory Efficiency Analysis

Unlike conventional feed-forward networks, our method relies on iterative optimization during inference, which entails considerable computational overhead. The current latency (approximately 2 min per target on an RTX 4090) precludes its use in time-sensitive or high-throughput scenarios. To thoroughly evaluate the resource requirements and practical deployability of our single-view optimization pipeline, we conduct a comprehensive benchmarking of both computational complexity (FLOPs) and GPU memory (VRAM) consumption.

Table 10. Computational cost of the single-view optimization pipeline per input image.

Component	MLP Calls	MACs	FLOPs
Phase 1 decoder	65,536,000	52,059,701,248,000	104,119.40 GFLOPs
Phase 2 decoder	262,144,000	208,238,804,992,000	416,477.61 GFLOPs
Post-processing	2,560,000 rays	—	3.28 GFLOPs
Total	327,680,000	—	520,600.29 GFLOPs

details the computational budget required to process a single remote sensing image across the entire optimization lifecycle. The pipeline consists of 500 iterations in Phase 1 and 2000 iterations in Phase 2. In each optimization step, a batch of 1024 rays is cast, with 128 points sampled per ray, resulting in 131,072 continuous 3D query points processed by the MLP decoder per forward pass.

As reported, the cumulative computational cost for the entire optimization yields a total of 520,600.29 GFLOPs. The mathematical relationship strictly follows $1\mathrm{MAC}\approx 2\mathrm{FLOPs}$. Phase 2 accounts for the vast majority ($80.0\%$) of the computational budget due to its extended alignment and fine-tuning iterations. In contrast, the post-processing stage (surface rendering and mesh extraction via Marching Cubes) requires a negligible computational effort of only $3.28$ GFLOPs (<0.01%), ensuring that once the implicit representation is optimized, the 3D asset can be synthesized almost instantaneously.

To provide a complete picture of the hardware overhead,

Table 11. GPU memory footprint of the single-view optimization pipeline per input image. All values are measured under FP32 precision with batch size 1024 rays.

Component	Memory (MB)	Proportion
Model parameters (FP32)	3.05	0.3%
Resident data †	0.53	<0.1%
Per-step activation (Phase 1 & 2)	1152.5	97.7%
Decoder layer outputs	1035.5	87.8%
Input buffers	11.0	0.9%
Render temporaries	6.0	0.5%
Adam optimizer states	6.1	0.5%
CUDA context overhead	∼20	1.7%
Peak (batch$= 1024$)	1179.2	100%
Mesh extraction (no-grad)	∼2327	—

^† Resident data includes pre-computed ray origins/directions for all pixels and the target image tensor.

breaks down the peak VRAM utilization measured under FP32 precision with a ray batch size of 1024. A prominent architectural characteristic emerges: our framework is highly parameter-efficient but activation-intensive.

Specifically, the weights and biases of the SDF decoder occupy a mere $3.05$ MB, and the Adam optimizer states require only $6.1$ MB, reflecting a compact footprint of approximately $0.8$ M trainable parameters. However, the per-step activation memory dominates the execution, commanding $1152.5$ MB ($97.7\%$) of the peak optimization VRAM. This phenomenon is inherently driven by the nature of coordinate-based implicit neural representations; during the backward pass, the framework must cache the intermediate features of the 6-layer heavy MLP ($\{512,512,512,256,256,256\}$ dimensions) for all 131,072 spatial query points simultaneously to calculate gradients for the latent code z and pose p.

We further analyze the peak memory scalability and post-processing dynamics. Thanks to the lightweight parameter scale, the optimization VRAM scales linearly with the ray batch size. Even when scaling up to a large batch size of 4096 rays to accelerate convergence, the peak optimization VRAM is estimated at only ∼4.69 GB, which fits comfortably within common consumer-grade hardware.

Finally, during the final mesh extraction phase, querying a dense ${256}^3$ voxel grid under torch.no_grad() creates a temporary memory footprint of ∼2327 MB. Because the optimization and mesh extraction are executed in a strictly sequential paradigm, their memory footprints do not stack. Therefore, the absolute peak VRAM across the entire pipeline lifecycle is bounded by the mesh extraction stage (∼2.33 GB), demonstrating that our method is exceptionally lightweight and highly suitable for deployment on standard workstation computers or resource-constrained remote sensing ground stations.

5.3. Technological Outlook

Although the proposed single-view 3D reconstruction framework demonstrates promising results, several intrinsic limitations remain to be addressed. First, the reconstruction fidelity is heavily coupled with the quality of image preprocessing. Under challenging scenarios—such as low foreground resolution, pronounced off-nadir viewing angles, and out-of-distribution or rare aircraft morphologies—the preprocessing modules are prone to non-trivial errors, occasionally leading to degraded or failed 3D meshes. Furthermore, the test-time optimization paradigm introduces a considerable computational bottleneck. The resulting inference latency limits the pipeline’s immediate applicability in strictly real-time or high-throughput remote sensing scenarios.

To mitigate these fidelity and robustness bottlenecks, future research will first focus on expanding the underlying shape database to encompass a wider variety of rare aircraft categories, thereby providing a more comprehensive prior manifold. Concurrently, developing robust coarse-to-fine pose estimation networks and domain-specific segmentation algorithms tailored for complex remote sensing backgrounds can substantially suppress preprocessing errors at the source.

Regarding computational efficiency, several algorithmic and engineering advancements can be integrated to accelerate the pipeline. First, incorporating a more powerful feed-forward encoder network can provide high-quality initial latent codes and pose priors, effectively transitioning the system to an encoder-driven warm-start strategy that drastically reduces the required optimization iterations. Second, replacing the heavy coordinate-based MLP with hybrid neural representations, such as multi-resolution hash grids or tri-planes, would significantly compress the per-iteration forward-pass latency. Crucially, given that our pipeline exhibits an exceptionally lightweight VRAM footprint, high-capacity hardware such as the NVIDIA RTX 4090 is heavily underutilized when processing a single image. Consequently, implementing a multi-target parallel optimization paradigm can markedly elevate the aggregate system throughput, making it highly viable for high-throughput remote sensing data centers.

Finally, the generalizability of the proposed framework can be extrapolated beyond aircraft to a broader spectrum of remote sensing targets, such as buildings and maritime vessels. Beyond optical imagery, adapting this implicit reconstruction paradigm to cross-modal data, particularly Synthetic Aperture Radar images, represents a compelling future direction. Overcoming the inherent speckle noise and geometric distortions of SAR data to achieve robust 3D asset generation would profoundly expand the utility of implicit neural representations in comprehensive Earth observation applications.

6. Conclusions

In this paper, we have presented a unified framework that jointly addresses single-view 3D reconstruction and classification of aircraft targets in optical remote sensing imagery. The core of our approach is a shared latent code $z^{\ast }$, which is iteratively optimized via differentiable rendering against a single input image and serves a dual role: as a generative seed for high-fidelity 3D mesh, and as a compact discriminative feature for target classification. This coupling establishes a principled bridge between geometric reasoning and semantic recognition, which is absent in conventional decoupled pipelines for remote sensing image interpretation.

From a methodological standpoint, we introduce several contributions. A cluster-aware prior learning strategy is employed during offline SDF training to facilitate downstream classification without sacrificing reconstruction interpolability. A two-stage differentiable rendering pipeline, consisting of a coarse logistic occupancy kernel for global pose alignment followed by a sharp Laplace occupancy kernel for fine-grained surface refinement, is designed to address the coupled optimization of pose and geometry under the non-convex landscape of orthographic neural rendering. Furthermore, an analytical complex-moment-based pose initialization method is proposed to reduce the search space of the subsequent optimization.

Extensive experiments validate the effectiveness of the proposed framework. On simulated ShapeNet renderings, our method outperforms both Pix2Vox and AtlasNet across most evaluation metrics. On the real-world MTARSI dataset, the classification branch achieves 97.88% accuracy with a 1.18% improvement attributable to the reconstruction feature fusion, demonstrating that geometric priors learned from 3D data provide complementary information beyond 2D appearance features.

Although the experimental results are generally satisfactory, the current framework still has certain limitations, including complex preprocessing, relatively long optimization time, and poor generalization performance for rare aircraft shapes. We believe that our work has provided a new approach for the interpretation of remote sensing images, which will be beneficial for the future research ideas and practices.