SDKP: A Synthetic Multi-Task Dataset for Monocular 6-DoF Pose Estimation of Non-Cooperative Spacecraft

Abstract

Monocular six-degree-of-freedom (6-DoF) pose estimation of non-cooperative spacecraft is a fundamental technology for on-orbit servicing, space debris removal, and autonomous rendezvous and docking missions. Existing deep learning-based pose estimation methods typically rely on large-scale image datasets with accurate pose annotations for model training and evaluation. However, acquiring real on-orbit images is extremely expensive, while obtaining high-precision 6-DoF pose ground truth is even more challenging, significantly limiting the development and validation of related algorithms. To address these challenges, this work constructs and publicly releases SDKP (Spacecraft Detection, Keypoint and Pose Dataset), a synthetic multi-task dataset specifically designed for monocular pose estimation of non-cooperative spacecraft. SDKP is built on an automated physically based rendering pipeline with simulated multi-source on-orbit lighting conditions to produce 24,000 high-fidelity RGB images. All annotations are automatically generated within a unified coordinate framework to guarantee geometric consistency across multi-task labels. Each image is synchronously annotated within a self-defined JSON format, containing 2D/3D object detection bounding boxes, 2D projections of 3D semantic keypoints, and ground-truth 6-DoF pose annotations. During data generation, camera viewpoints are densely sampled within a spherical observation space, covering diverse azimuth angles, elevation angles, and observation distances to provide rich viewpoint variations. Meanwhile, both deep-space black backgrounds and Earth-background scenarios are incorporated to enhance environmental diversity. Ultimately, SDKP provides a unified annotation format and multi-level supervision information, aiming to serve as a standardized benchmark dataset for the development and evaluation of 2D/3D object detection, keypoint detection, and end-to-end pose estimation algorithms, while facilitating reproducible experiments and fair comparisons among different methods.

Dataset: https://doi.org/10.57967/hf/9797.

Dataset License: The SDKP dataset is distributed under the CC BY 4.0 license. This license applies exclusively to the final composited images generated in this work. Raw Earth background assets collected from public online resources are not released together with the dataset, and we cannot confirm their original license terms.

1. Introduction

Monocular six-degree-of-freedom (6-DoF) pose estimation of non-cooperative spacecraft is a key technology for missions such as on-orbit servicing (OOS) [1], active debris removal (ADR) [2], and rendezvous and docking (RVD) [3]. Unlike cooperative spacecraft equipped with dedicated navigation aids, non-cooperative spacecraft generally cannot provide prior motion state information and may exhibit arbitrary orientations, complex structures, weak textures, and significant appearance variations, making visual perception and accurate pose recovery more challenging.

With the rapid development of deep learning technology, spacecraft pose estimation has gradually evolved from traditional geometric methods toward data-driven visual perception methods [4,5,6]. Existing learning-based methods generally first extract visual features from monocular images and then directly regress the target pose or achieve pose estimation through intermediate representations such as object detection bounding boxes, semantic keypoints, heatmaps, and segmentation masks [7,8,9]. Among them, keypoint-based methods first establish correspondences between 2D and 3D keypoints and then recover the spacecraft pose using geometric solvers such as PnP; detection-assisted methods, on the other hand, use target localization results to constrain the subsequent pose estimation process. While improving the accuracy and robustness of spacecraft pose estimation, these methods have also promoted the integration of object detection, keypoint localization, and geometric pose recovery within a unified visual perception pipeline.

The performance of the aforementioned deep learning methods highly depends on large-scale training data and accurate annotations. Compared with conventional terrestrial vision tasks, the acquisition of spacecraft image data is more difficult. Real on-orbit observations require costly space missions and dedicated imaging equipment, while obtaining high-precision 6-DoF pose ground truth from real observations is even more challenging. Therefore, existing studies largely rely on synthetic images generated from 3D spacecraft models [10]. Compared with real data, synthetic rendering can directly provide accurate pose annotations and enables systematic control over camera viewpoints, observation distances, illumination, and backgrounds. Therefore, such datasets have become an important foundation for training and evaluating spacecraft visual perception algorithms.

As spacecraft visual perception methods continue to develop, the requirements for datasets have gradually expanded from single pose annotations to richer supervision information. Early pose estimation methods mainly required accurate 6-DoF pose annotations [11,12], while subsequent keypoint-based methods additionally required semantic keypoints and their corresponding 3D geometric information [13,14]. In recent years, the development of detection-assisted and multi-task learning frameworks has introduced additional requirements for object detection bounding boxes [15]. Meanwhile, emerging visual perception systems may also require segmentation, depth, event data, or LiDAR-based multimodal information [16,17]. Therefore, datasets for modern spacecraft visual perception tasks need to provide not only accurate pose supervision but also rich intermediate visual annotations and diverse observation conditions.

Although existing spacecraft vision datasets cover a wide range of tasks, including pose estimation, object detection, keypoint localization, and multimodal perception, their task focuses and supervision schemes still vary considerably. For detection-assisted pose estimation and multi-task visual perception, clear task associations and geometric correspondences need to be maintained among object localization, structural feature extraction, and spatial pose recovery. Therefore, beyond providing rich annotations for individual tasks, it is also necessary to organize visual supervision at different levels within a unified data framework, thereby supporting joint training, independent evaluation, and systematic investigation of task interactions across multiple perception tasks.

To address the above requirements, this study constructs and publicly releases SDKP (Spacecraft Detection, Keypoint and Pose Dataset), a benchmark for monocular visual perception and 6-DoF pose estimation of non-cooperative spacecraft. Based on a physically based rendering pipeline implemented in Blender, SDKP contains 24,000 high-fidelity RGB images, with all multi-task annotations generated in a unified coordinate system to maintain strict geometric consistency. Each image is synchronously annotated via a self-defined JSON format, providing 2D/3D bounding boxes, 3D semantic keypoints and their 2D projections, and precise 6-DoF pose ground truth. During data generation, camera viewpoints are densely sampled within a spherical observation space, covering diverse azimuth angles, elevation angles, and observation distances of 100–600 m. Meanwhile, deep-space black backgrounds and Earth-background scenarios are incorporated to increase environmental diversity.

The remainder of this paper is organized as follows: Section 2 introduces existing spacecraft visual perception datasets and related studies; Section 3 introduces the construction method of the SDKP dataset; Section 4 introduces the quality evaluation and benchmark evaluation results; and Section 5 concludes the paper.

2. Related Work

With the development of monocular spacecraft pose estimation research, researchers have successively released several benchmark datasets for this task. Among them, SPEED [18] is one of the earliest publicly available spacecraft pose estimation datasets. It targets the Tango satellite from the PRISMA mission and contains 15,000 synthetic images rendered using OpenGL and 300 real images captured using the TRON hardware-in-the-loop testbed. SPEED was used as the official dataset for the Satellite Pose Estimation Challenge (SPEC2019) in 2019 and has been widely used for the training and evaluation of monocular spacecraft pose estimation methods.

Subsequently, SPEED+ [19] was proposed as an extension of SPEED, increasing the number of synthetic images to nearly 60,000 and additionally providing 9531 unlabeled real images. The dataset further considers two typical illumination domains, namely the lightbox domain simulating Earth albedo and the sunlamp domain simulating direct solar illumination, with target distances ranging from 2.25 to 10 m. SPEED+ was used in SPEC2021 and provided an important benchmark for research on domain adaptation, domain randomization, and sim-to-real generalization.

For close-range relative navigation tasks, the URSO [20] dataset contains approximately 25,000 high-resolution images generated using Blender and covers two typical mission phases: the safety-ellipse circumnavigation phase at distances of 100–600 m and the final approach phase at distances of 8–80 m. In addition to 6-DoF pose annotations, URSO also provides six semantic keypoint annotations, making it suitable for the research and evaluation of keypoint-based spacecraft pose estimation and relative navigation methods.

The AKM [21] dataset focuses on another challenging spacecraft perception problem, primarily targeting textureless and rotationally symmetric objects such as the STAR 24 apogee engine. The dataset is designed to investigate pose estimation under weak visual texture and strong structural symmetry and provides a data basis for analyzing the limitations of conventional visual feature- and keypoint-based methods for such targets.

In addition to dedicated pose estimation benchmarks, researchers have also developed several datasets for broader spacecraft and space-object visual perception. These datasets typically provide annotations such as object detection bounding boxes, segmentation masks, depth information, or multiple sensor modalities.

SPARK [22] is a representative dataset for space situational awareness tasks. It contains approximately 150,000 RGB images generated using Unity3D, covers 11 types of space targets, and provides corresponding depth maps and segmentation masks. Unlike SPEED and URSO, which focus primarily on pose estimation, SPARK places greater emphasis on general visual perception of space objects and can be used for object detection, semantic segmentation, depth estimation, and multimodal space-object perception research.

With the development of event cameras, SPADES [16] was proposed to investigate event-based spacecraft pose estimation. The dataset contains approximately 180,000 simulated poses and 15,000 real poses, covering a measurement range of 3.5–12 m. It is mainly used to investigate the advantages of event-based visual perception and the sim-to-real domain gap in spacecraft pose estimation.

In recent years, SpaceSense-Bench [23] has further expanded spacecraft visual perception benchmarks toward large-scale, multimodal, and zero-shot evaluation. The dataset covers 136 spacecraft categories and simultaneously provides RGB images, depth maps, and LiDAR point clouds. It further divides different spacecraft into non-overlapping training, validation, and test categories, enabling models to be evaluated on previously unseen spacecraft and providing a more challenging evaluation protocol for cross-object generalization.

Overall, existing datasets have provided important data resources for spacecraft pose estimation, object detection, keypoint localization, and multimodal visual perception. As summarized in Table 1, these datasets have been developed for different research objectives, resulting in substantial variations in annotation types, target categories, and supported tasks. Pose estimation-oriented datasets mainly focus on accurate 6-DoF pose supervision, while keypoint-based datasets provide intermediate geometric representations for pose recovery. In contrast, general spacecraft perception datasets mainly emphasize object-level understanding, such as detection, classification, segmentation, or multimodal perception. However, the organization and format of annotations across different tasks still vary substantially. With the recent development of detection-assisted pose estimation and multi-task visual perception, there remains room for further improvement in simultaneously providing object localization, semantic structural representations, and accurate spatial pose annotations within the same observation instance while maintaining geometric consistency among them. To address this need, SDKP synchronously generates 2D/3D object bounding boxes, 3D semantic keypoints and their 2D projections, and 6-DoF pose ground truth within a unified rendering pipeline, camera model, and coordinate system, thereby providing a unified data foundation for different spacecraft visual perception tasks.

Table 1. Comparison of Existing Spacecraft Vision Datasets and Annotation Characteristics.

Dataset	Images	Target(s)	2D KP	3D KP	2D BBOX	3D BBOX	Pose	Supported Tasks
SPEED	15,300	Tango	×	×	×	×	√	Pose Estimation
SPEED+	69,491	Tango	×	×	×	×	√	Pose Estimation, Domain Adaptation
URSO	25,000	Soyuz, Dragon	√	√	×	×	√	Keypoint Detection, Pose Estimation
AKM	10,000	STAR-24 Motor	×	×	×	×	√	Pose Estimation
SPARK	150,000	10 Satellites + Debris	×	×	×	×	×	Detection, Classification, Segmentation
SPADES	194,900	Proba-2	√	√	×	×	√	Event-based Pose Estimation
SpaceSense-Bench	90,000+	136 Spacecraft	√	×	√	√	√	Multi-modal Perception, Pose Estimation
SDKP	24,000	Fermi-GLAST	√	√	√	√	√	2D/3D Object Detection, Keypoint Localization, Monocular 6-DoF Pose Estimation, Multi-task Learning

The following section presents the detailed data construction pipeline, annotation scheme, and dataset organization.

3. Dataset

3.1. Data Collection

SDKP is constructed following a fully automated synthetic rendering pipeline, as illustrated in Figure 1. The overall pipeline consists of five stages: 3D model preparation, scene and camera sampling, physically based rendering, automatic annotation generation, and dataset splitting and output.

Figure 1. Overview of the SDKP dataset construction pipeline. The five-stage process includes 3D asset preparation, spherical camera sampling, physically based rendering, automatic annotation generation, and train/validation/test set splitting.

3.1.1. 3D Model Preparation

The first step of data generation is to prepare the 3D model of the target spacecraft. In this work, a publicly available spacecraft 3D model released by NASA, namely the Fermi-GLAST model, is selected. NASA is acknowledged as the source of this 3D asset, and no NASA insignia or logos appear on the adopted spacecraft model. Its material properties and surface attributes are configured in Blender to ensure that the subsequent rendering process can accurately reproduce the visual appearance of the target in the space environment. To ensure the accuracy of subsequent keypoint annotations, bevels at the semantic keypoint locations are first removed from the models. Empty objects are then placed at the semantic keypoint locations and used to define the positions of the 3D keypoints. During the rendering process, the spacecraft model and the predefined Empty objects maintain a fixed relative structure. The origin of the model coordinate system is set at the geometric center of the spacecraft model and serves as the unified reference frame for all subsequent pose annotations.

3.1.2. Scene and Camera Sampling

After model preparation, the pipeline proceeds to scene configuration and camera sampling. To precisely control the imaging geometry, the virtual camera is modeled using a standard pinhole camera model, with its intrinsic parameters adopted from the publicly available camera model of the SPEED [18] dataset, as listed in Table 2. The spacecraft model is fixed at the origin of the world coordinate system, while camera viewpoints are randomly sampled within a spherical observation space. Specifically, three parameters are sampled independently: the azimuth angle is uniformly sampled over [0°, 360°], and the elevation angle is also uniformly sampled within [−90°, 90°]. Note that the sampling is performed by direct uniform random sampling in the azimuth–elevation parameter space. The observation distance between the camera and the target is randomly selected from 100–600 m. The camera optical axis always points toward the center of the spacecraft. The roll angle (rotation about the camera optical axis) is fixed at 0° throughout sampling; the camera local up-axis is always aligned with the Z-axis of the world coordinate system, and no in-plane image rotation is introduced. Small random angular perturbations are applied along the directions perpendicular to the optical axis to simulate realistic pointing jitter. The angular offsets in the two orthogonal directions both follow zero-mean Gaussian distributions with a standard deviation of 0.5°. A boundary check is implemented during sampling to ensure that the projected silhouette of the spacecraft mostly stays within the camera field of view, avoiding excessive target displacement that could lead to keypoint detection failure or large pose estimation bias. This sampling strategy ensures that the generated images cover diverse observation geometries and target scales, providing comprehensive data for evaluating the generalization capability of algorithms under different observation conditions.

Table 2. Camera intrinsic parameters adopted in the SDKP dataset.

Parameter	Description	Value
Nu	Image width (pixels)	1920
Nv	Image height (pixels)	1200
fx	Horizontal focal length (m)	0.0172
fy	Vertical focal length (m)	0.0172
Px	Horizontal pixel pitch (µm)	5.86
py	Vertical pixel pitch (µm)	5.86

3.1.3. Physically Based Rendering

Once the camera parameters are determined, the pipeline invokes Cycles, the physically based rendering engine in Blender, to generate RGB images. All images are rendered at a resolution of 1920 × 1200 pixels and saved in PNG format.

To realistically reproduce on-orbit imaging conditions, a multi-source illumination system is designed, consisting of five components: sunlight, key fill light, Earth albedo, rim light, and ambient light. Sunlight is modeled as a parallel light with an intensity of 3.5 and a warm-white color (RGB: 1.0, 0.95, 0.9). Since the distance from the spacecraft to the Sun is far larger than the spacecraft’s physical size, the incident light can be approximated as parallel rays. Under this approximation, the incident direction of the parallel sunlight is randomly sampled per frame over the upper hemisphere of the world coordinate system to simulate variations in the solar azimuth under on-orbit conditions, while keeping the parallel nature of the light beam unchanged. The key fill light is an area light whose position is always aligned with the line-of-sight from the camera to the target, ensuring uniform illumination on the front face of the spacecraft. Its intensity is dynamically adjusted between 25 and 40 according to the camera-target distance to compensate for illuminance attenuation. In addition, a large blue-tinted light source (RGB: 0.5, 0.6, 1.0) is used to simulate Earth albedo, accounting for the reflected light from the Earth in low-Earth orbit. A warm-colored rim light is placed behind the spacecraft to enhance edge visibility and separate the foreground from the background. The ambient light comes from the world background node with an intensity of 0.5, providing a low, uniform baseline illumination and preventing excessively dark shadow regions. To match the random camera sampling strategy and guarantee diverse target appearances, the intensity and orientation of the key fill light are updated dynamically with the camera position, reproducing natural illumination variations during spacecraft imaging. This illumination setup is applied consistently across all rendered samples, independent of background type.

Shadow calculation is enabled during rendering to naturally produce self-shadows on the spacecraft surface. The Earth background is composited via alpha blending after rendering; the background layer is excluded from the physical lighting computation of the spacecraft mesh, so the spacecraft casts no shadows onto the background.

For backgrounds, a hybrid background generation strategy is adopted to simulate realistic spacecraft observation scenes and enrich environmental diversity. Two background categories are considered: pure black backgrounds representing unobstructed deep-space observations, and real Earth background images for on-orbit scenes with complex visual distractions. Specifically, 80% of rendered samples use a pure black background, and the remaining 20% are composited with real background images collected from public online resources via alpha blending. This design serves two purposes: black-background samples provide ideal clutter-free conditions for geometry-oriented evaluation, while Earth-background samples introduce realistic visual interference for robustness assessment. The SDKP dataset only distributes the final composited rendered images and does not release the original background source files. It should be noted that this compositing scheme implies no physical light interaction between the spacecraft and the Earth background, which constitutes a limitation of this synthetic dataset.

3.1.4. Automatic Annotation Generation

In parallel with image rendering, the rendering engine automatically records the geometric information associated with each frame through the Blender Python API and uses it as precise supervision. The annotation system comprises three main types of information: (1) Camera extrinsic 6-DoF parameters: rotation quaternion and translation vector representing the pose of the camera in the Blender world coordinate system; (2) 2D image projections of 3D semantic keypoints, where keypoints predefined on the spacecraft 3D model are mapped onto the image plane via the camera projection matrix; (3) 2D and 3D bounding boxes, where the 2D bounding box is an axis-aligned bounding box in the image plane, and the 3D bounding box is an oriented bounding box defined in the camera coordinate system.

Here, the 3D bounding box is parameterized with only the yaw angle to characterize its in-plane rotation, since the silhouette variation of the spacecraft in the image is predominantly governed by yaw rotation, while the influence of pitch and roll on the imaging silhouette is negligible. This simplification pertains solely to the bounding box visualization parameters, whereas the 6-DoF pose ground truth of the dataset still fully preserves the three-axis rotation information. The 2D vertices of the 3D bounding box are obtained by projecting the eight vertices of the 3D bounding box in the camera coordinate system through the camera projection matrix. All three types of annotations are generated synchronously within a single rendering pipeline, thereby ensuring geometric consistency among the different annotations.

3.1.5. Dataset Splitting and Output

The training, validation, and test subsets are generated separately at a ratio of 10:1:1, yielding 20,000, 2000 and 2000 images, respectively, with a total of 24,000 high-quality RGB images after quality checks. Unlike post hoc random splitting from a single image pool, each subset is rendered independently with separate random seeds during viewpoint sampling, which avoids near-duplicate samples and guarantees statistical independence across subsets. Each image is named according to a unified encoding scheme and associated with its corresponding annotation file through an explicit indexing relationship, facilitating subsequent data loading for algorithm training and evaluation as well as the reproduction of experimental results.

3.2. Data Description

The SDKP dataset contains 24,000 high-resolution RGB images and their corresponding annotation files, with a customized directory layout, as illustrated in Figure 2. The dataset is divided into training, validation, and test sets at a ratio of 10:1:1, containing 20,000, 2000, and 2000 images, respectively. At the root level, the dataset provides a 3D semantic keypoint template file (3D_keyworld.json) and a camera intrinsic parameter file (camera.json). Each subset contains an images/folder for storing RGB images and an annotation.json file containing the corresponding annotations, including 2D/3D object bounding boxes, 2D semantic keypoints, and 6-DoF pose information.

Figure 2. Directory structure of the SDKP dataset.

All images are stored in 8-bit RGB PNG format with a resolution of 1920 × 1200 pixels. The dataset includes two representative observation backgrounds: deep-space black backgrounds and Earth backgrounds. The former are used to simulate idealized deep-space observation conditions, while the latter represent on-orbit observation scenarios with more complex visual interference.

The semantic keypoints provided by the dataset establish explicit 2D–3D correspondences between image observations and the three-dimensional geometric structure of the spacecraft. These correspondences can be directly used for keypoint-based pose estimation, PnP solving, and geometry-aware methods.

3.2.1. Annotation Description

The annotation system of SDKP consists of four main components: 3D semantic keypoint models, 2D semantic keypoint coordinates, 6-DoF pose ground truth, and 2D/3D object bounding boxes, as described below.

(1) The dataset provides the 3D coordinates of predefined semantic keypoints as an independent model file, expressed in the spacecraft body coordinate system. This file defines the 3D keypoint template of the spacecraft and establishes a one-to-one mapping with the 2D keypoint annotations in the images. Based on this 2D–3D correspondence, geometric methods such as PnP can be adopted to solve the spacecraft pose, while also supporting keypoint-based geometric optimization and performance evaluation of pose estimation. In this paper, a total of 16 semantic keypoints are defined, each corresponding to a specific hardware component on the spacecraft body. The number of keypoints and their annotation locations on the spacecraft model are illustrated in Figure 3.

Figure 3. Locations of the 16 predefined semantic keypoints defined on the Fermi-GLAST spacecraft body model.

(2) 2D Semantic Keypoint Coordinates. For each image, the 2D pixel coordinates (x,y) of the predefined semantic keypoints are provided. These coordinates represent the image-plane locations of the corresponding structural features of the spacecraft under the current observation viewpoint, thereby establishing explicit correspondences between image features and the spacecraft geometry. They provide fundamental annotations for semantic keypoint detection and keypoint-based pose estimation. Keypoints located outside the image frame are stored with their true continuous projected coordinates, and negative coordinate values are permitted. This design supports keypoint extrapolation for 6-DoF pose regression.

(3) 6-DoF Pose Ground Truth. Each JSON file stores the extrinsic camera pose in the Blender world coordinate system, represented by a 3D translation vector t = [tx, ty, tz] (in meters) and a unit quaternion q = [qw, qx, qy, qz]. This camera-to-world transformation describes the camera’s pose in Blender world space. The derived body-to-camera result serves as reference ground truth for quantitative evaluation of end-to-end pose regression and geometric pose estimation algorithms. All pose annotations in the JSON files represent the body-to-camera transformation under the Blender coordinate convention, where the camera forward direction points along (-). This differs from the OpenCV convention with (+Z) forward.

(4) 2D/3D Object Bounding Boxes. Each image provides both 2D and 3D object bounding boxes for the target spacecraft. The 2D bounding box is stored in the bbox field and specifies the object class, the normalized center coordinates (x_center, y_center), and the box width and height, thereby describing the spacecraft’s location and scale in the image plane. The 3D bounding box is stored in the bbox_3d field and comprises the projected 2D center center_2d, the 3D center center_3d, the 3D dimensions, the yaw angle in radians (yaw) and degrees (yaw_deg), the target depth, and the 2D projections of the eight 3D box vertices vertices_2d. Here, yaw refers to the rotation about the Z-axis of the world coordinate system, and vertices_2d are projected using the full 6-DoF pose of the spacecraft rather than a yaw-only box. Together, these attributes characterize the spacecraft’s spatial extent, position, orientation, depth, and image projection, supporting research on 3D object detection, spatial localization, and multi-task visual perception.

All annotations are generated automatically within the same rendering and coordinate framework and are explicitly indexed to their corresponding images, thereby ensuring geometric consistency among the different annotation types. The training, validation, and test sets adopt a unified annotation format, facilitating fair comparisons, reproducible experiments, and systematic benchmarking of different spacecraft visual perception methods under consistent data conditions.

3.2.2. Background Conditions

To increase environmental diversity and simulate different on-orbit spacecraft observation conditions, SDKP includes two types of backgrounds: deep-space black backgrounds and Earth backgrounds. Specifically, approximately 80% of the images use a pure black background to represent unobstructed deep-space observation conditions, while the remaining 20% use real Earth backgrounds, which are composited with the spacecraft scenes through alpha blending to simulate complex visual interference from the Earth background.

The Earth backgrounds contain visual elements such as planetary boundaries, cloud structures, and surface textures, introducing greater background complexity compared with the pure black deep-space scenes. The two background types maintain a consistent 8:2 ratio across the training, validation, and test sets to avoid significant background distribution differences among the dataset subsets. Figure 4 presents representative samples from the SDKP dataset, illustrating rendered spacecraft images under different background conditions, observation viewpoints, and target scales.

Figure 4. Representative rendered samples of the SDKP dataset under different background conditions. The samples include spacecraft images with pure black backgrounds representing deep-space observation environments and images composited with Earth backgrounds to simulate realistic on-orbit observation scenarios involving planetary edges, cloud structures, and surface texture variations.

3.3. Viewpoint and Distance Distribution

To comprehensively cover the observation geometries of spacecraft, camera viewpoints in the dataset are sampled within a spherical observation space with a wide range of directions and distances. The training, validation, and test sets follow identical viewpoint and distance distributions, while independent random seeds are adopted during sample generation to guarantee statistical independence across the three subsets.

The camera viewpoints span the full azimuth range of [0°, 360°] and elevation range of [−90°, 90°], supporting near-omnidirectional observation around the spacecraft. The camera-to-spacecraft distance ranges from 100 to 600 m, yielding rich variations in target scale and projection geometry. This broad sampling coverage produces diverse spacecraft appearances, visible regions and projected shapes under different viewing directions and imaging scales, providing comprehensive support for evaluating spacecraft detection, semantic keypoint localization, and 6-DoF pose estimation algorithms.

The spatial distribution of camera viewpoints in the training set is illustrated in Figure 5. The projection of camera positions onto the X–Y plane in (a) reflects the horizontal distribution of viewpoints around the spacecraft. The distribution in the X–Z plane in (b) illustrates variations in elevation and radial distance. The histogram in (c) characterizes the distribution of camera positions along the Z-axis. Overall, the sampled viewpoints span a broad spherical observation space, covering diverse observation distances and relative spatial configurations.

Figure 5. Camera viewpoint distribution analysis of the SDKP dataset. The black star marks the position of the target spacecraft at the origin of the world coordinate system. Note that the plots visualize separate X/Y/Z coordinate components of camera positions, not radial distance.

4. Data Evaluation

4.1. Experimental Details and Evaluation Metrics

4.1.1. Experimental Details

To verify the applicability of the proposed SDKP dataset and evaluate the performance of existing methods for monocular 6-DoF spacecraft pose estimation, we conducted benchmark experiments under unified experimental settings. The hardware platform adopted in the experiments consisted of an Intel Xeon Silver 4210R CPU and one NVIDIA RTX A6000 GPU.

In terms of training configurations, identical input settings were adopted for all baseline methods. Input images were uniformly resized to 256 × 256 pixels to satisfy network input requirements while maintaining computational efficiency. The Adam optimizer was utilized for parameter updating, with an initial learning rate set to 1 × 10⁻³, which was gradually decayed down to 1 × 10⁻⁵ via a cosine annealing schedule during training. The batch size was set to 32.

For data augmentation, multiple online augmentation strategies were applied to improve model generalization and simulate the diversity under real-world on-orbit observation conditions. These strategies included random brightness-contrast adjustment to mimic varying illumination conditions, random Gaussian noise injection to emulate sensor noise, random image cropping and scaling to enhance the model’s robustness against target-scale variations.

4.1.2. Evaluation Metrics

The evaluation metrics follow the widely adopted standards from the Satellite Pose Estimation Challenge (SPEC). The deviations between predicted pose and ground-truth pose are quantitatively assessed via translation error and orientation error.

The translation error is defined as the absolute translation error, which measures the discrepancy between the predicted translation vector and the ground-truth translation vector:

$${\mathit{Score}}_p={\displaystyle \frac{{‖t_{\mathit{est}}-t_{\mathit{gt}}‖}_2}{{‖t_{\mathit{gt}}‖}_2}}$$

(1)

where t_est and t_gt denote the predicted translation vector and ground-truth translation vector, respectively, and ‖⋅‖₂ represents the L2-norm.

The orientation error is defined as the geodesic distance between predicted rotation and ground-truth rotation, which is computed based on unit quaternions:

$${\mathit{Score}}_o=2\arccos \left(|⟨q_{\mathit{est}},q_{\mathit{gt}}⟩|\right)$$

(2)

where q_est and q_gt are the unit quaternions corresponding to the predicted rotation and ground-truth rotation, respectively, and ⟨⋅,⋅⟩ denotes the quaternion dot product. This metric effectively computes the minimal angular deviation in the 3-D rotation space and eliminates sign ambiguity caused by the double-cover property of quaternions.

Finally, the overall pose score is defined as the sum of these two errors:

Score = Score_p + Score_o

(3)

4.2. Quality Assessment

To verify the reliability of the generated annotations, we performed both geometric consistency analysis and manual inspection.

First, the accuracy of the keypoint annotations was evaluated using reprojection error analysis. The annotated 2D keypoints were reprojected from the corresponding 3D keypoints using the provided camera intrinsic matrix and ground-truth pose parameters. The average reprojection error was calculated to measure the consistency between the rendered images and generated annotations. The reprojection error is defined as:

$$e_{\mathit{reproj}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{\Vert {\mathrm{p}}_i-\pi (\mathrm{K},\mathrm{T},{\mathrm{P}}_i)\Vert }_2}$$

(4)

where N is the number of keypoints, p_i denotes the i-th annotated 2D keypoint, P_i is the corresponding 3D keypoint, K represents the camera intrinsic matrix, T is the ground-truth pose transformation matrix, and π(⋅) denotes the perspective projection function.

Furthermore, we conduct an independent evaluation based on the PnP algorithm to further verify the pose annotations. The spacecraft pose is solved using the 2D keypoints provided in the dataset and the 3D keypoint model, and the solved pose is compared against the ground-truth pose obtained from the original rendering. Over the 20,000 training samples, the mean rotation error is 0.37°, the mean translation error is 2.35 m, and the mean reprojection error is 0.340 pixels. The non-zero errors mainly stem from pixel quantization: the exported 2D keypoint coordinates are rounded to integer pixel values in the JSON file. Minor numerical noise from the solver also contributes to the error, rather than defects in the ground-truth annotations themselves. In the annotation files, 2D keypoints are stored as integers, while the 3D keypoint coordinates, translation vectors, and quaternion components are stored in double-precision floating-point numbers. These results demonstrate that the generated annotations achieve high geometric accuracy.

In addition, to verify the correctness of the 2D/3D object detection annotations, 500 images were randomly selected for bounding box visualization and manual inspection. During inspection, the eight vertices of each 3D bounding box are projected onto the image plane using ground-truth camera intrinsics and pose parameters. We visually check whether the projected vertices tightly enclose the spacecraft silhouette, while the 2D bounding boxes are validated to ensure they fully cover the projected spacecraft region. Samples with obvious misalignment between projected box vertices and spacecraft contour are regarded as erroneous. The inspection results showed that no annotation errors were found in the automatically generated annotations.

4.3. Benchmark Test Analysis

To evaluate the applicability and challenges of the SDKP dataset for monocular spacecraft pose estimation, benchmark experiments were conducted on four subtasks: 2D object detection, 3D object detection, semantic keypoint detection, and 6-DoF pose estimation. Table 3 summarizes the representative models and their performance for each subtask.

Table 3. Multi-task baseline performance on the proposed SDKP dataset.

(a) 2D Object Detection
Model	Prec. (%)		mAP50 (%)
YOLO26n	0.986		0.994
YOLO26s	0.971		0.993
(b) 3D Object Detection
Model	Backbone	AP3D|R40 (%)	APBEV |R40 (%)
SMOKE	DLA-34	15.72	22.46
MonoDGP	ResNet50	24.88	33.91
(c) Keypoint Detection
Model	Backbone		Reprojection Error (px)
CSKDet	ResNet50		6.230
Motion-Aware ViT	ViT-Base		8.847
(d) Pose Estimation
Model	Backbone	Scoreo(rad)	Scorep(-)	Score
Mobile-URSONet	MobileNetv2	0.118209	0.374166	0.492375
GkNet	UNet	0.924413	0.612374	1.536787

As shown in Table 3a, both YOLO26n and YOLO26s [24] achieve high 2D detection accuracy, with precision above 0.97 and mAP50 above 0.99, indicating that the target regions in SDKP are clearly annotated and relatively easy to detect. This provides a reliable 2D foundation for the subsequent 3D tasks.

As shown in Table 3b, SMOKE [25] and MonoDGP [26] achieve AP_3D of 15.72 and 24.88, and AP_BEV of 22.46 and 33.91, respectively, evaluated at 40 recall positions (R40). Compared with 2D detection, the 3D metrics drop significantly, indicating that the main challenges of SDKP arise from depth estimation and 3D spatial localization rather than 2D object localization.

As shown in Table 3c, CSKDet [27] and Motion-Aware ViT [28] obtain reprojection errors of 6.230 px and 8.847 px, respectively. The gap between them suggests that keypoint detection accuracy is sensitive to model architecture, and that structural symmetry, occlusion, and extreme viewpoints in SDKP impose significant interference on keypoint localization.

As shown in Table 3d, Mobile-URSONet [29] achieves a mean orientation score of 0.118 rad and a normalized translation score of 0.374, yielding a total pose score of 0.492. Conversely, GkNet [30] exhibits significantly larger errors, with an orientation error of 0.924 rad, a translation score of 0.612, and a total pose score of 1.537. To further analyze the error distributions, Figure 6 visualizes the histograms of Score_p and Score_o for both models. As observed in Figure 6a, Mobile-URSONet’s position errors are highly concentrated near zero, whereas GkNet shows a bimodal distribution with a noticeable secondary peak at approximately 2.0, indicating that keypoint-based methods occasionally suffer from severe depth estimation failures. Moreover, Figure 6b reveals that GkNet frequently encounters orientation crashes, evidenced by a prominent spike at around 3.14 rad (180°), which corresponds to complete rotational flip ambiguities. In contrast, Mobile-URSONet maintains a stable and concentrated orientation distribution. This substantial performance gap and the distinct error distributions indicate that keypoint-based approaches encounter greater difficulty in monocular pose estimation on the challenging SDKP dataset compared to direct regression methods.

Figure 6. Comparison of the evaluation metrics distributions on the test set. (a) Distribution of the relative position Score. (b) Distribution of the orientation Score in radians. The blue and orange bars represent the results of Mobile-URSONet and GkNet, respectively. The dashed vertical lines indicate the mean values for each model.

Overall, the above benchmark results show that SDKP is already well-suited for 2D detection tasks, while 3D detection, keypoint detection, and 6-DoF pose estimation remain significantly challenging. The dataset can provide a challenging evaluation platform for future research on monocular spacecraft pose estimation methods.

5. Conclusions

In this paper, we introduce a new large-scale synthetic multi-task dataset, SDKP (Spacecraft Detection, Keypoint and Pose Dataset), designed to support the development and evaluation of visual perception and monocular six-degree-of-freedom (6-DoF) pose estimation methods for non-cooperative spacecraft. The dataset contains 24,000 high-fidelity RGB images, generated through an automated rendering pipeline built with Blender using a physically based rendering engine. The rendering pipeline simulates realistic multi-source orbital lighting, including sunlight, Earth albedo reflection, rim light and ambient illumination, to introduce rich illumination variations. During data generation, camera viewpoints are densely sampled over a spherical observation space, covering diverse azimuth and elevation angles as well as observation distances ranging from 100 to 600 m. To increase environmental diversity, the dataset includes both deep-space black-background and Earth-background scenes.

For each image, SDKP provides synchronized and standardized annotations in a self-defined JSON format, including 2D/3D object bounding boxes, 3D semantic keypoints and their 2D image projections, and accurate 6-DoF pose ground truth. Therefore, the dataset can be used to train and evaluate a wide range of methods, including 2D/3D object detection, semantic keypoint localization, detection-assisted pose estimation, and end-to-end monocular 6-DoF pose estimation. Moreover, the unified annotation framework enables different visual perception tasks to be studied under consistent data splits and annotation protocols, thereby facilitating multi-task learning, reproducible experiments, and fair comparisons among different methods.

Different from conventional post hoc random data division, the training, validation, and test subsets of SDKP are independently generated with separate random seeds, ensuring statistical independence across subsets while maintaining identical geometric distributions.

We further conducted comprehensive annotation quality verification and benchmark evaluations to validate the applicability and effectiveness of SDKP as a reliable spacecraft visual perception benchmark. Experimental results demonstrate that monocular 6-DoF pose estimation for non-cooperative spacecraft remains a challenging task on this dataset. In future work, we will further explore geometry-aware dataset partition strategies. By explicitly dividing the geometric observation space, we can construct out-of-distribution test sets with unseen camera poses and distance ranges, which can better evaluate the generalization ability and robustness of pose estimation algorithms under diverse geometric conditions. We hope that SDKP will provide researchers with a useful and standardized data resource and further promote the development of robust and highly generalizable visual perception methods for non-cooperative spacecraft.