In this section, the NDSIA is compared with Pyramid using the following five metrics:

#### 3.1. Test Conditions

Each test contained 1000 randomly oriented scenes, where a scene consists of performing the Star-ID and estimating the attitude. The random orientations were generated via

$QR$ decomposition of a randomly generated

$3\times 3$ matrix. The seed used to generate the random

$3\times 3$ matrix was fixed such that all tests used the same random orientations. All of the tests included in this section were performed in C++ on a computer running Ubuntu 18.04 with an Intel(R) Core(TM) i5-2400 CPU at 3.10 GHz and 16.0 GB of RAM. All run times were calculated using the system_clock function in the C++ boost chrono library. The parameters of the virtual star tracker used in these tests are shown in

Table 2, where

$U[0,5]$ represents the uniform distribution of integers in the range

$[0,5]$ and

$\sigma $ denotes the standard deviation of a normal distribution.

The centroiding error was applied to the star measurements in the camera frame using the methodology shown in Equation (

2).

where

$\mathcal{N}(\mu ,{\sigma}^{2})$ is the normal distribution with mean

$\mu $ and variance

${\sigma}^{2}$,

$\sigma $ is the centroid error of the camera,

${\widehat{\mathit{b}}}_{t}\in {\mathcal{R}}^{3}$ is a unit vector that points in the true direction of the star in the camera reference frame,

${\widehat{\mathit{b}}}_{e}\in {\mathcal{R}}^{3}$ is the observed unit vector, affected by centroid error,

$\mathit{v}\in {\mathcal{R}}^{3}$ is a random vector, and the function

${C}_{p}(\widehat{\mathit{e}},\theta )$ produces an attitude matrix with principle axis

$\widehat{\mathit{e}}$ and principle angle

$\theta $. The function used to calculate

${C}_{p}$ given

$\widehat{\mathit{e}}$ and

$\theta $ is,

where

${\mathcal{I}}_{3\times 3}$ is the

$3\times 3$ identity matrix and

$[\widehat{\mathit{e}}\times ]$ is the skew-symmetric matrix formed using the components of

$\widehat{\mathit{e}}$.

The focal length and OA offset perturbations are applied to the star measurements in the camera frame using the methodology shown in Equation (

3),

where

${\widehat{\mathit{b}}}_{e}\left(3\right)$ is the third component of the vector

${\widehat{\mathit{b}}}_{e}$,

$\delta f$ is the focal length perturbation,

$\delta x$ and

$\delta y$ are the OA offset perturbations, and

${\widehat{\mathit{b}}}_{p}$ is the unit vector pointing to the star whose centroid coordinates are

$[{x}_{c},{y}_{c}]$ on the imager. The “±” sign appearing in Equation (

3) depends on the imager

x and

y axes directions. Please note that Equation (

3) assumes that the star tracker is modelled as an ideal pin-hole camera.

#### 3.2. Results

Table 3 gives a summary of the performance and results of Pyramid and the NDSIA on the eight tests. For each algorithm, three columns are included that contain the first three metrics described earlier. The first column,

${n}_{id}(\%)$, gives the percentage of scenes where the Star-ID was completed, and the second column,

${n}_{+id}$, shows the percentage of completed Star-ID runs with a successful Star-ID (i.e., a Star-ID wherein all stars identified by the algorithm are identified correctly). The third column shows the average time each algorithm took to perform the Star-ID in milliseconds. In the joint statistics column of the table, the remaining two metrics are reported.

In general,

Table 3 shows that the NDSIA is reliable, even when the camera parameters are perturbed, as it never completes a Star-ID unsuccessfully (i.e., it never incorrectly identifies a star). Furthermore, in every test, the NDSIA is able to successfully complete the Star-ID in more than 50% of the scenes wherein Pyramid completes the Star-ID unsuccessfully. The joint statistics for the NDSIA completing Star-IDs successfully where Pyramid does not complete the Star-ID at all are less impressive, but still show that the NDSIA maintains some benefit in this regard over Pyramid in all tests with focal length perturbations. Finally,

Table 3 shows that the NDSIA takes on the order of tens of milliseconds or less to perform the Star-ID, a speed that is suitable for many real-time applications.

Test 1 shows that when the star tracker is working nominally Pyramid and the NDSIA are each able to identify the stars correctly, with one exception; in one case, Pyramid performs the Star-ID incorrectly. However, in this case, the final attitude error, $0.011$ degrees, is still small. Pyramid is able to identify stars more often than the NDSIA, and Pyramid is more than an order of magnitude faster than the NDSIA. This is expected, as the NDSIA is meant to be used only when the Pyramid algorithm begins to fail.

Test 2 and Test 3 are the first two examples of such situations. These tests were performed with focal length perturbations. Compared to the nominal case, the Pyramid algorithm performance degrades significantly. The most alarming change in the performance of the Pyramid algorithm is the reduction in the percentage of scenes with a successful Star-ID. In Test 2 and Test 3, the Star-ID performed by Pyramid can no longer be trusted. In contrast, the percentage of the NDSIA scenes with a successful Star-ID remains unchanged from the nominal case. However, the number of Star-IDs that can be completed is reduced significantly when compared with the nominal case. In addition, the time required to complete a Star-ID increases due to the focal length perturbations.

In Test 4 and Test 5, the performance of the Pyramid algorithm remains almost unchanged when compared with Test 1, the nominal case. The reason is small OA offset perturbations have little effect on the interstellar angles between stars for most locations on the imager. Therefore, the Pyramid algorithm is still able to perform the Star-ID. Conversely, the small OA offset perturbations have a larger effect on the dihedral angles between the stars. Thus, the number of times that the Star-ID can be completed by the NDSIA is reduced as the OA offset increases. However, the percentage of scenes with a successful Star-ID remains unchanged from the nominal case. Therefore, while the NDSIA may not be able to perform the Star-ID process as often as Pyramid when there is only an OA offset perturbation, the Star-ID provided by the NDSIA can still be trusted.

Test 6 and Test 7 show the performance of the two algorithms when subject to both focal length perturbations and OA offset perturbations. In these two tests, there is again a significant degradation in the performance of the Pyramid algorithm, and the number of Star-IDs that both algorithms can complete is reduced when compared with Test 1, the nominal case.

Test 8 shows that the NDSIA is robust to changes in the centroiding accuracy of the camera, because the performance of the NDSIA in Test 8 is similar to the performance of the NDSIA in Test 6—Test 6 is the same as Test 8 but with a lower centroiding error. Comparing Pyramid and the NDSIA on Test 8 reveals that the NDSIA does better in terms of percentage of Star-IDs completed and percentage of Star-IDs completed successfully. The Pyramid algorithm is slightly faster than the NDSIA in Test 8. Comparing the Pyramid results for Test 8 with the Pyramid results for Test 6 shows that Pyramid is able to perform more Star-IDs in Test 8 than Test 6. The larger centroiding error absorbs some of the error due to the focal length and OA offset perturbations. In other words, in many scenes, the position error of stars in the camera frame due to focal length and OA offset perturbations is less than the typical centroiding error in Test 8. Thus, with the increased centroiding error, and therefore an increased range when performing the associated range searches, Pyramid is able to identify stars in more scenes.

Based on the results of Test 8, one is naturally led to wonder if the performance of Pyramid can be improved by increasing the range it uses when performing range searches in its database. Please note that in Test 8 both the actual camera centroiding error and the range used in Pyramid were modified. In contrast, here, only the range Pyramid uses will be modified, the actual camera centroiding error will be held constant, using the value given in

Table 2.

Table 4 shows how the performance of Pyramid is impacted by changing the size of the range search. The left most column in

Table 4 gives the standard deviation,

$\sigma $, that Pyramid assumes the star tracker camera has; hence, the range that Pyramid uses when searching the database of interstellar angles is

$\pm 3\sigma $. The focal length and OA offset perturbations used to create

Table 4 are the same as those used in Test 6. Hence, the first row of

Table 4 is identical to that of Test 6. Each row in

Table 4 was created using 1000 scenes, the same as in the previous eight tests.

Table 4 shows that as the range that Pyramid uses increases, so does the percentage of tests with completed Star-IDs. Moreover, the percentage of successful star IDs remains approximately constant. As a result, the total number of tests for which Pyramid completes a successful Star-ID increases as the range increases. However, this also means that there are still a notable number of cases for which Pyramid identifies stars incorrectly. Hence, even by varying the range that Pyramid uses to search in the database, the Pyramid algorithm does not outperform the NDSIA.

Histograms of attitude error are provided for each test to quantify how much the perturbations ultimately affect the attitude estimation. The histograms only include the attitude estimation error for scenes where the Star-ID was completed successfully, as almost all scenes with an incorrect Star-ID have a large attitude error. The q-method is used to estimate the attitude for each algorithm [

18]. Therefore, the only variable that affects the attitude estimation error is the number of stars identified by each algorithm in each scene.

Figure 2 shows the histograms of attitude error for Pyramid and the NDSIA for Test 1. This figure shows that the two algorithms have similar attitude error distributions, but that Pyramid completed more Star-IDs than the NDSIA.

Figure 3 and

Figure 4 show histograms of the attitude error for Pyramid and the NDSIA for Test 2 and Test 3. These histograms clearly show that the NDSIA is more accurate on average than the Pyramid algorithm. Moreover, comparing Test 2 and Test 3 to Test 1 reveals that the attitude error of the NDSIA increases slightly as the focal length perturbation increases.

Figure 5 and

Figure 6 show the attitude error histograms for Pyramid and the NDSIA for Test 4 and Test 5. These figures show that the attitude error distributions for the two algorithms are similar for both tests; however, in each case, the NDSIA completed fewer Star-IDs than the Pyramid algorithm. When compared to Test 1, the attitude error of both algorithms increases with the increase in the OA offset perturbation.

Figure 7 and

Figure 8 show the attitude error histograms for Pyramid and the NDSIA for Test 6 and Test 7. These figures show that on average the NDSIA is more accurate than Pyramid. In these tests, the NDSIA is able to complete the Star-ID process in more cases than Pyramid.

Figure 9 shows the attitude error histograms for Pyramid and the NDSIA for Test 8. This figure shows that on average the NDSIA is more accurate than Pyramid. In Test 8, the NDSIA was able to complete the Star-ID process more times than Pyramid. Comparing the results of this test with Test 6 reveals that the performance of the NDSIA is similar between the two tests, whereas Pyramid’s performance improves noticeably in Test 8.