A Parallel Approach for Multi-GNSS Ultra-Rapid Orbit Determination

Abstract

Global Navigation Satellite System (GNSS) ultra-rapid orbit is critical for geoscience and real-time engineering applications. To improve the computational efficiency and the accuracy of predicted orbit, a parallel approach for multi-GNSS ultra-rapid orbit determination is proposed based on Message Passing Interface (MPI)/Open Multi Processing (OpenMP). This approach, compared with earlier efficient methods, can improve the efficiency of multi-GNSS ultra-rapid orbit solution without changing the original observation data and retaining the continuity and consistency of the original parameters to be estimated. To obtain high efficiency, three steps are involved in the approach. First and foremost, the normal equation construction is optimized in parallel based on MPI. Second, equivalent reduction of the estimated parameters is optimized using OpenMP parallel method. Third, multithreading is used for parallel orbit extrapolation. Thus, GNSS ultra-rapid orbit determination is comprehensively optimized in parallel, and the computation efficiency is greatly improved. Based on the data from MGEX and IGS stations, experiments are carried out to analyze the performance of the proposed approach in computational efficiency, accuracy and stability. The results show that the approach greatly improves the efficiency of satellite orbit determination. It can realize 1-h update frequency for the multi-GNSS ultra-rapid orbit determination using 88 stations with four-system observations. The accuracy of the GPS, GLONASS, Galileo and BDS ultra-rapid orbit with 1-h update frequency using the parallel approach is approximately 33.4%,31.4%,40.1% and 32.8% higher than that of the original orbit, respectively. The root mean squares (RMS) of GPS, GLONASS, Galileo and BDS predicted orbit are about 3.2 cm, 5.1 cm, 5.6 cm and 11.8 cm. Moreover, the orbit provided by the proposed method has a better stability. The precision loss of all parallel optimization can be negligible and the original correlation between the parameters is fully preserved.

1. Introduction

Global Navigation Satellite System (GNSS) ultra-rapid orbit is the basis of geodetic research and real-time applications, such as real-time atmosphere monitoring [1,2], real-time PPP [3,4,5], real-time satellite clock estimation [6], and real-time orbit determination of LEO satellite [7]. With the development of Galileo and implementation of BDS global service plan, the number of satellites in the constellation has increased dramatically to more than 130 [8]. Currently, the International GNSS Service (IGS), the Multi-GNSS Experiment and Pilot Project (MGEX) and the International GNSS Monitoring and Assessment System (iGMAS) have provided more than 500 global distributed stations. A combined multi-GNSS solution can effectively reduce systematic errors and improve the accuracy and reliability of satellite orbit. However, on the other hand, the increase in the number of satellite constellations and stations makes the processing model more complicated and results in larger computational burden [9], which decrease the computational efficiency of multi-GNSS ultra-rapid orbit determination [10]. Real-time or quasi-real-time users have higher requirements for the accuracy, stability, and update frequency of GNSS ultra-rapid orbits. Therefore, further research on the rapid determination of multi-GNSS predicted orbits is critical for its high-precision applications.

GNSS orbit products that have been widely used are gradually developing toward multi-system, higher accuracy and higher update frequency. Since its establishment in 1994, IGS has vigorously carried out research on GNSS precise orbit determination, and the accuracy and reliability of the products have been continuously improved [11,12]. At present, the accuracy of the ultra-rapid orbit is better than 5 cm, which is provided for real-time users. The details of the accuracy and latency of orbit products are shown in

Table 1. Accuracy and latency of each IGS orbit products.

Type		Accuracy	Latency	Updates	Sample Interval
Broadcast	orbits	~100 cm	real-time	-	daily
Ultra-rapid	orbits	~5 cm	real-time	4 times	15 min
Ultra-rapid	orbits	~3 cm	3–9 h	4 times	15 min
Rapid	orbits	~2.5 cm	17–41 h	daily	15 min
Final	orbits	~2.5 cm	12–18 days	weekly	15 min

(http://www.igs.org/products, accessed January 2019). IGS also generates experimental final combined multi-GNSS orbits, which include GPS, GLONASS, Galileo, BDS-2, BDS-3, and QZSS [13]. The IGS called for participation in MGEX in 2012 (http://mgex.igs.org, accessed January 2019). Subsequently, five analysis centers (ACs) provided multi-system orbit products, including CNES, TUM, GFZ, CODE and WUH. The later three ACs can provide the orbit product of GPS, GLONASS, Galileo and BDS, as well as QZSS. In recent years, ACs have increased the update frequency of ultra-rapid orbit for real-time applications (ftp://cddis.gsfc.nasa.gov/pub/gps/products/mgex/, accessed January 2019). IGS increased the update frequency from 12-h to 6-h. The frequency of ultra-rapid orbit has been further improved to 3-h by the German Research Centre for Geosciences (GFZ) (ftp.gfz-potsdam.de, accessed January 2019). The analysis center at Wuhan university is the first to provide a 1-h update frequency ultra-rapid orbit product.

The algorithm of precise orbit determination (POD) mainly uses dynamics and reduced dynamics methods [14,15,16,17,18]. The methods above are first applied to GPS satellite orbit determination and are widely used in GLONASS, Galileo, BDS and other satellite systems, as well as the combination of them with the development of GNSS [19,20,21,22,23,24]. Based on this, the predicted orbit of ultra-rapid products is generally obtained by integration of initial orbit and force model parameters [25]. The accuracy of orbit for real-time applications is directly related to the accuracy of satellite orbit initial values and the length of extrapolation time. In order to obtain accurate solutions and achieve higher computational efficiency, many scholars have optimized the theories and methods of orbit determination by dividing subnets, grouping combined systems, dividing arc length of observation, and reducing observations. Beutler (1999), for the first time, proposed the method of merging the past three consecutive single-day arcs to orbit determination in 1999 [26]. Yao (2008) further used a dynamic smoothing algorithm to smooth the orbit which can improve the accuracy and reliability of the comprehensive algorithm solution. In order to improve the efficiency of ultra-rapid orbit determination [27], Lou (2008) deeply studied the problem of ambiguity processing in real-time orbit determination and proposed a sliding window method [28]. Through the superposition of short-arc normal equations, the orbit can be calculated with higher efficiency, and the accuracy is improved when the ambiguities are fixed. In addition, TUM used a two-step approach for generating Galileo and QZSS products [29]. Furthermore, a three-step approach was applied by WHU to orbit product calculation: GPS and GLONASS orbit and ERPs are solved first, then station coordinates, zenith tropospheric delay (ZTD) and receiver clock are estimated, and the orbit of other satellites are solved in the last step. Then, a block recursive least squares method was proposed to improve the accuracy of GNSS ultra-rapid orbits. The update frequency of ultra-rapid orbits was increased from 6-h to 1-h, and the accuracy was significantly improved [30]. Li (2019) realized 1-h update multi-GNSS ultra-rapid orbit determination by multi-threading and changed observation intervals [31]. The accuracy of predicted orbit was improved and the kinematic PPP solutions could achieve a shorter time to first fix and higher positioning accuracy.

To promote the accuracy of the predicted multi-GNSS orbit, we proposed a new approach for multi-GNSS hourly ultra-rapid orbit determination. This approach improves the efficiency of multi-GNSS ultra-rapid orbit solution without changing the original observation data or processing interval, and retains the continuity and consistency of the original parameters to be estimated using the multi-process and multi-thread method based on Message Passing Interface (MPI)/Open Multi Processing (OpenMP). MPI is a message passing library specification which provides a powerful and portable way for expressing parallel programs. OpenMP is a set of compiler directives as well as an API for programs that provides support for parallel programming in shared-memory environments. Therefore, MPI/OpenMP can achieve higher accuracy for multi-GNSS ultra-rapid orbit as the initial orbit is precise, but the length of extrapolation time is shortened significantly. To realize that, first and foremost, the construction of the normal equation for initial value of satellite orbit is divided into multi-arc parallel processing by the multi-process method based on MPI, and each process achieves consistency through processing the arc data and interacting matrix. In addition, the multi-thread method is used in parameter elimination and orbit extrapolation to further improve the efficiency. In this approach, the GPS+GLONASS+Galileo+BDS observations are processed together in one common procedure with the double-differenced network model. The promotion of computational efficiency, accuracy and stability is validated based on the data from MGEX and IGS stations.

In the following sections, the principle and main steps of the modified parallel multi-GNSS obit determination approach will be introduced. Then, some processing strategies are presented. Afterward, using MGEX/IGS data and related products, the improvement of computational efficiency of the new approach is analyzed. At last, the accuracy of multi-GNSS obit using the proposed approach is evaluated, and the results indicate that the proposed parallel approach is viable.

2. Methodology

The double-differenced network solution is used in the proposed parallel multi-GNSS ultra-rapid orbit determination. After the data preprocessing, the basic procedure of multi-GNSS ultra-rapid orbit determination mainly includes three steps. First, the double-differenced normal equations are constructed based on un-differenced observation equations. Second, the normal equations are solved after eliminating the redundant parameters. Finally, the predicted orbits are obtained by extrapolation on the basis of initial orbit and model parameters using orbit integration. We optimize the above core processing to achieve higher computational efficiency using a parallel computing method based on MPI/OpenMP. Since the original observational model and stochastic model have not been changed, the approach retains the original correlation of the estimated parameters and the consistency of the ambiguity parameters.

2.1. Multi-Arc Parallel Normal Equations Construction Based on MPI

In the multi-GNSS ultra-rapid orbit determination, the sequential batch processing method is adopted. The double-differenced carrier observation equation can be expressed as:

$$L{C}_{kl}^{ij}(t)={\rho }_{kl}^{ij}(t)+\mathsf{\Delta }{T}_{kl}^{ij}{}_{trop}(t)+N_{kl}^{ij}\cdot {\lambda }_c+{{\epsilon }^{\prime }}_{\mathsf{\Phi }}$$

(1)

where t represents epoch, $L{C}_{kl}^{ij}$ represents the ionospheric-free phase observation of station k, l and satellite i, j, ${\rho }_{kl}^{ij}$ denotes pseudorange, $\mathsf{\Delta }{T}_{kl trop}^{ij}$ denotes tropospheric delay, $N_{kl}^{ij}$ is ambiguity, λ is the wavelength of ionospheric-free observations, and ${{\epsilon }^{\prime }}_{\mathsf{\Phi }}$ is the observation noise. The observation equation (1) is linearized and expressed as:

$$L{C}_{kl}^{ij}(t)={\rho }_{kl0}^{ij}(t,X_{r0},X^s(t))+B\cdot \delta X_r+H\cdot \delta X_0^s+\mathsf{\Delta }{T}_{kl}^{ij}{}_{trop}(t)+N_{kl}^{ij}\cdot {\lambda }_c+{{\epsilon }^{\prime }}_{\mathsf{\Phi }}$$

(2)

where $B=\partial {\rho }_{kl}^{ij}\left(t\right)/\partial X_r$ is the partial derivative of the measured value with respect to the state parameters, $\Psi \left(t,t_0\right)$ is the transition matrix which can be obtained by solving the variational equation, $X_{r0}$, $\delta X_r$ are the approximate coordinates of the station and their corrections, respectively, $X^s\left(t\right)$ is the state of satellite at time t, which is obtained by solving the differential equation of satellite motion, ${\rho }_{kl0}^{ij}\left(t,X_{r0},X^s\left(t\right)\right)$ is the approximate value of the double-differenced satellite-ground geometric distance at time t, and $\delta X_0^s$ is the correction of the satellite state at the reference time. Other symbols have the same meaning as in (1). Based on the observation (2), the GNSS satellite reduce-dynamic precise orbit determination (RDPOD) can be performed.

Table 2. Multi-GNSS ultra-rapid orbit determination strategies.

Item	Description	Remark
Observations	Double-differenced phase and pseudo-range measurements	External a
Satellite system	GPS, GLONASS, BDS, Galileo	External
Sampling rate	30 s	External
Elevation cut-off angle	3°	External
Ionosphere model	Ionosphere-free combination with higher-order-ionosphere correction	External
Tropospheric model	GMF [34] with 2-h resolution of station-specific troposphere parameters, gradients [35] with daily resolution	Estimated b
PCO/PCV	IGS14.atx	External
A priori orbit	MGEX broadcast ephemeris	External
Stations coordinates	IGS SINEX products	External
Orbital model	3 position and 3 velocity coordinates at the start of the orbital arc, without pseudo-stochastic pulses	Estimated
Solar Radiation Parameter (SRP) model	ECOM1 model with five parameters [36]	External
ERP	xp, yp, xp-rate, yp-rate and LOD are estimated with tight constraint of UT1-UTC [37]	Estimated
Gravity field model	EGM2008, 12 × 12 degree [38]	External
N-body perturbation	JPL DE405 ephemeris	External
Solid earth tides, pole tides/Semidiurnal tide	IERS Conventions 2010 [39]	External
Ocean Tidal model	FES2004 [40]	External
Precession and nutation models	IAU2006/IAU2000A	External
Relativistic effects	IERS Conventions 2010	External

^a The remark of External means the data source from models or products provided by other agencies; ^b The remark of Estimated means the data source generated by our software.

shows the main parameter settings of the RDPOD strategy. In this paper, we have no a priori box-wing model used for the orbit determination, which may generate orbit errors, especially for Galileo satellites due to the elongated shape of the satellite buses. The a priori box-wing model may be a solution for twice-per-revolution empiri-cal parameters from the CODE’s new solar radiation pressure model are used for GNSS orbit determination [32]. In the paper, we adopt the one-step orbit determination of the GPS, GLONASS, Galileo and BDS in order to study that the parallel algorithm improves the efficiency of satellite orbit prediction, which does not affect the computational efficiency of the parallel algorithm. For BDS, no official metadata for a priori box-wing models are available. However, estimated parameters can be still used [33].

Equation (2) is the basic observation equation for ultra-rapid orbit determination. In order to determine the ultra-rapid orbit, the traditional sequential batch processing method traverses the observation data in all arcs by epoch to construct the normal equations. In this study, a parallel method based on MPI is used in this process. The main idea is to establish the multi-arc normal equation in the orbit determination by adopting the multi-processing method. Assuming that the single-day data is divided into three arcs, the length of each arc is 8 h. Each sub-process processes the measured data of the current arc and then calculates the observation equation coefficients. Based on the current time period, sub-normal equations including initial satellite state, force model parameters, earth rotation parameters, atmospheric parameters, station coordinates and ambiguity parameters are generated. Each process sends the sub-normal equation matrices and auxiliary parameters to the main process to obtain the total normal equations.

The parameter sequence of each period is consistent with that of the entire original period. The mathematical model and the parameters have not changed, so there is no additional conversion and connection. Supposing that the entire arc is divided into k sub-arcs, the least squares problem can be solved by accumulating the normal equations. The sub-normal equations and the entire normal equation can be expressed as:

$$\{\begin{array}{c}{N}_1\delta x=W_1\\ N_2\delta x=W_2\\ \cdots \\ N_k\delta x=W_k\end{array}$$

(3)

$$(N_1+N_2+\cdots +N_k)\delta x=(W_1+W_2+\cdots +W_k)$$

(4)

where N_k and W_k are the related matrix of sub-normal equations and $\delta x$ is unknown parameter vector. Based on this entire normal equation, the elimination of parameters and least squares (LS) estimation can be then carried out. It is worth noting that the number of time periods are designed according to the computer hardware and time requirements.

2.2. Parallel Parameter Elimination Based on OpenMP

The parameters to be estimated in the GNSS observation equation of POD include station coordinate parameters, satellite orbit parameters, tropospheric parameters, ambiguity parameters, and earth rotation parameters. Usually, only some of these parameters are of concern to us, and the other parameters can be eliminated to improve the computational efficiency [41]. Equivalent elimination is a classic algorithm that can reduce the dimensionality of normal equations without a precision loss (Yao 2004). The error equation can be expressed as:

$$v=L-(\begin{array}{cc}A& B\end{array})\left[\begin{array}{c}{\widehat{x}}_1\\ {\widehat{x}}_2\end{array}\right],P,$$

(5)

where L is the observation vector of dimension m, A and B are the design matrices of dimension $m\times \left(n-r\right)$ and $m\times r$, ${\widehat{x}}_1$ and ${\widehat{x}}_2$ are unknown parameters of dimension $n-r$ and $r$, $v$ is the residual error matrix of dimension $m$, $n$ is the number of unknowns, $m$ is the number of observations, and $P$ is the weight matrix of dimension $m\times m$.

The parameters in the equation are decomposed into $\left[\begin{array}{c}{\widehat{x}}_1\\ {\widehat{x}}_2\end{array}\right]$ as needed, taking into account the correlation between the observations. The normal equations are:

$$\left[\begin{array}{cc}{N}_{11}& N_{12}\\ N_{21}& N_{22}\end{array}\right]\left[\begin{array}{c}{\widehat{x}}_1\\ {\widehat{x}}_2\end{array}\right]=\left[\begin{array}{c}{b}_1\\ b_2\end{array}\right]$$

(6)

where

$$\left[\begin{array}{cc}{N}_{11}& N_{12}\\ N_{21}& N_{22}\end{array}\right]=\left[\begin{array}{cc}{A}^{T}PA& A^{T}PB\\ B^{T}PA& B^{T}PB\end{array}\right]$$

(7)

$$b_1=A^{T}PL,b_2=B^{T}PL$$

(8)

If only $x_1$ is concerned, $x_2$ can be eliminated in order to reduce the dimension of the normal equation; $\left[\begin{array}{cc}E& -Z\\ 0& E\end{array}\right]$ is defined, where $E$ is the identity matrix, $0$ is the zero matrix, $Z=N_{12}{N}_{22}^{-1}$, and $N_{22}^{-1}$ is the inverse matrix of $N_{22}$. The left and right sides of (8) are multiplied by the matrix as follows:

$$\left[\begin{array}{cc}E& -Z\\ 0& E\end{array}\right]\left[\begin{array}{cc}{N}_{11}& N_{12}\\ N_{21}& N_{22}\end{array}\right]\left[\begin{array}{c}{\widehat{x}}_1\\ {\widehat{x}}_2\end{array}\right]=\left[\begin{array}{cc}E& -Z\\ 0& E\end{array}\right]\left[\begin{array}{c}{b}_1\\ b_2\end{array}\right]$$

(9)

$$\left[\begin{array}{cc}{N}_1& 0\\ N_{21}& N_{22}\end{array}\right]\left[\begin{array}{c}{\widehat{x}}_1\\ {\widehat{x}}_2\end{array}\right]=\left[\begin{array}{c}{R}_1\\ b_2\end{array}\right]$$

(10)

where $N_1=N_{11}-N_{12}{N}_{22}^{-1}{N}_{21}$, $R_1=b_1-N_{12}{N}_{22}^{-1}{b}_2$. Then the normal equation after elimination is:

$$N_1{\widehat{x}}_1=R_1$$

(11)

The solution of the normal equation is:

$${\widehat{x}}_1=N_1^{-1}{R}_1$$

(12)

$$Q_{{\widehat{x}}_1{\widehat{x}}_1}={(N_{11}-N_{12}{N}_{22}^{-1}{N}_{21})}^{-1}=N_{11}^{-1}+N_{11}^{-1}{N}_{21}{(N_{22}-N_{12}{N}_{11}^{-1}{N}_{12})}^{-1}{N}_{12}{N}_{11}$$

(13)

Equation (12) is substituted into (5):

$${\widehat{x}}_2=N_{22}^{-1}(b_2-N_{21}{\widehat{x}}_1)$$

(14)

$$Q_{{\widehat{x}}_2{\widehat{x}}_2}=N_{22}^{-1}+N_{22}^{-1}{N}_{21}{(N_{11}-N_{12}{N}_{22}^{-1}{N}_{21})}^{-1}{N}_{12}{N}_{22}^{-1}$$

(15)

When the number of observation equations and parameter dimensions is very large, the parameter elimination is time-consuming. In order to further improve computational efficiency, parallel optimization using OpenMP technology for parameter elimination is adopted.

When the initial orbit parameters and the force model parameters are determined by using LS estimation, the orbit of the entire arc will be updated in the order of epoch and satellite by using orbit integration. As with the the above-mentioned, we use the parallel orbit integration method to update the final orbit sequence.

2.3. Parallel Orbit Determination Process

The parallel approach proposed for the ultra-rapid orbit determination of the double-differenced network solution mainly adopts the MPI/OpenMP hybrid parallel programming model. The optimized orbit determination process is shown in Figure 1. After reading observation files, navigation ephemeris files, and some auxiliary model correction files, data preprocessing includes gross error detection, basic model correction, forming double-differenced baseline, and floating solution. Then, the entire time period to be calculated is divided into n segments, and multiple processes are developed based on MPI to read and construct sub-normal equations in parallel. Each process read from the first epoch to the last epoch of the current period so that the consistency of the current arc parameter sequence can be maintained. The above multiple sub-normal equations are accumulated to obtain the total normal equation. Next, in order to further improve the efficiency of orbit determination, the OpenMP multithreading model is used for the parameter elimination process. Finally, after obtaining the precise dynamic parameters by the LS estimation, the same multi-thread model is used to realize the parallel orbit integration to obtain the final satellite POD results. The entire parallel optimization does not destroy the correlation between the parameters without the ambiguity segmentation problem. The input and output of orbit determination do not need to be adjusted greatly, which is easy to implement. It can be applied to single satellite POD as well as multi-GNSS POD.

3. Computation and Comparison

In order to verify the feasibility and effectiveness of the proposed GNSS parallel ultra-rapid orbit determination approach, multiple sets of experiments are carried out. The precision loss of the parallel POD, the improvement of computational efficiency, and the orbit accuracy after using the parallel approach are analyzed in detail.

Two sets of tracking stations from IGS/MGEX networks around the world are selected to analyze the accuracy and efficiency of multi-GNSS parallel ultra-rapid POD. The networks are shown in Figure 2. Relevant data from day of year (DOY) 182 to 189, 2018 are selected for the processing. The approach is implemented by Fortran and intel MPI/OpenMP parallel library. The experiments are carried out on a Linux computer cluster.

3.1. Impact of Parallel Approach on the Orbit Solution

Before further analyzing the efficiency and accuracy of the parallel approach, it is necessary to analyze the accuracy of the non-parallel GNSS POD method. The observation data from DOY 182 to 188, 2018 of the above 131 IGS/MGEX stations are used to carry out the multi-GNSS POD. Analysis of the errors in the radial, along and cross directions of the multi-GNSS orbits using the non-parallel method is performed. As a result, the along track direction accuracy is the lowest, followed by the cross track, and the radial is the highest. Except for slightly larger fluctuations in the BDS in the sense of the orbit RMS, the other systems have little change and the accuracy is relatively stable. The accuracy of each system is further analyzed, which is shown in

Table 3. Accuracy statistics of multi-GNSS orbit using the non-parallel method (cm).

Satellite System	R	A	C	RMS
GPS	1.0	1.5	1.2	1.3
GLONASS	1.6	3.2	2.6	2.7
Galileo	2.7	2.8	2.4	2.7
BDS	5.9	5.3	4.2	5.4

. It can be seen from the table that the 1D mean RMS ($1DRMS=\sqrt{\left({\sigma }_R^2+{\sigma }_A^2+{\sigma }_C^2\right)/3}$) of GPS, GLONASS, Galileo and BDS are 1.3 cm, 2.7 cm, 2.7 cm, and 5.4 cm, respectively, which are at the same accuracy level as reported by international institutions [42]. The results show the correctness of the non-parallel POD processing, which lays the foundation for the proposed parallel POD.

It is necessary to further analyze the impact of the parallel approach on the covariance matrix and solution of the normal equations, as well as the precision loss of orbit. Based on MPI, the entire 480 epochs are divided into 2 periods. Statistics of the difference between the matrix elements of the parallel method and those of the non-parallel method is shown in

Table 4. Statistics of the difference between the matrix elements of the parallel method and those of the non-parallel method.

	Max	Min	Mean	STD
Difference of ATPA	1.019 × 10−9	−1.019 × 10−9	−9.633 × 10−18	1.365 × 10−13
Difference of ATPL	1.023 × 10−12	−1 × 10−11	−6.233 × 10−16	1.22 × 10−13
Difference of unknown parameters	2.913 × 10−5	−2.525 × 10−5	7.519 × 10−8	3.274 × 10−6

. It can be seen from these figures that the error impact of the parallel approach on the covariance matrix and unknown parameters are approximately 10⁻¹³ and 10⁻⁶, respectively.

Using the same set of data, the normal equation coefficients of the above-mentioned parallel approach and the normal equation coefficients of the non-parallel approach are saved. After the procedures of parameter estimation, orbit update, and result output, the two sets of orbits are obtained. The comparison of the two sets of orbits show that their difference is below the millimeter level, which indicates that the precision loss caused by parallel method can be negligible.

3.2. Improvement of Computational Efficiency of Parallel Approach

In order to verify the characteristics of the parallel approach in terms of efficiency, the parallel approach is compared with the non-parallel method. The efficiency improvement of the approach in many aspects is further analyzed, such as multi-system, number of threads, and number of processes. In addition, the computational time changing with the number of stations is analyzed, which reflects the advantage of the parallel approach in the improvement of efficiency and provides a reference for the selection of the number of stations in the 1-h update frequency ultra-rapid POD.

3.2.1. Improvement of Efficiency with Different Number of Satellite Systems

The computational efficiency of GPS, GPS+GLONASS and GPS+GLONASS+Galileo+BDS parallel POD are compared to analyze the influence of the number of satellites. Two processes and ten threads are used in the parallel scheme. The statistical results of computational time are shown in

Table 5. Time-consuming statistical results of GNSS parallel and non-parallel POD (unit: min).

Satellite System	Non-Parallel POD	Initial Orbit Estimation	Parallel Initial Orbit Estimation	Parallel POD
GPS	28	12	6	23
GPS + GLONASS	60	46	22	46
GNSS(G/R/E/C)	178	160	46	122

, and the comparison between non-parallel and parallel approach is shown in Figure 3.

It can be seen from Table 5 that the more satellite systems that are involved, the longer it takes for POD. After using MPI parallelism, the calculation time of this process is significantly reduced, indicating that it is reasonable to optimize this process in parallel.

It can be seen from Figure 3 that the efficiency of GPS, GPS+GLONASS and GPS+GLONASS+Galileo+BDS POD have been obviously improved by the proposed parallel approach. As the number of satellites increases, the efficiency of the parallel POD has been improved more significantly, and the efficiency of the four-system combined POD has been improved the most. The parameters to be estimated and the dimension of the equation both increase nonlinearly, so the advantages of the parallel POD are more obvious.

3.2.2. Improvement of Efficiency with Different Number of Threads

The computational efficiency of the parallel parameter elimination process with a different number of threads based on OpenMP is further analyzed. According to the hardware configuration of the platform, the number of threads is sequentially increased from 2 to 16 at an interval of 2, and the computational time for the parameter elimination process of each parallel scheme is compared, which is shown in Figure 4.

It can be seen from Figure 4 that as the number of threads increases, the computational time of parameter elimination in GNSS POD gradually decreases and the scheme with 16 threads has the shortest computational time. However, the speed of efficiency improvement gradually decreases, most likely because when the number of threads is close to or greater than the number of cores, queuing round calculations occur, and the opening and ending of threads take more time. Therefore, further increasing the number of threads beyond the hardware core is of little significance to the improvement of parallel efficiency.

3.2.3. Improvement of Efficiency with Different Number of Processes

In order to further reflect the improvement of efficiency by the parallel POD approach, the influence of the number of processes on efficiency is analyzed. Multi-process is mainly used to construct normal equations in parallel. All schemes use the same number of threads, the same processing strategy, and select the number of processes as the only control variable to analyze the effect of increasing the number of processes on the computational time. The time consumption is shown in

Table 6. Time consumption of multi-process parallel computing.

Number of Process	1	2	3	4	5
Computational Time	824.0s	460.8s	326.3s	265.6s	241.9s

, and the speedup is shown in Figure 5.

As can be seen from Figure 5 and Table 6, compared to a single-process, the speedup ratios of two-process, three-process, four-process, and five-process are 1.788, 2.525, 3.102, and 3.406, respectively. As the number of processes increases, the speedup ratio increases. When five processes are used, the speedup in POD reaches the maximum. The speedup shows a nonlinear trend probably because the cost of opening and closing processes and inter-process communication increases as the number of processes increases, and depends on the proportion of parallel programs to the total program. In general, if the hardware can support more processes, the computational efficiency of POD can be further improved, which is similar to multithreading.

3.2.4. Improvement of Efficiency with Different Number of Stations

Observation data from 40 to 110 IGS and MGEX stations are selected to analyze the improvement of efficiency of parallel multi-GNSS POD. The results are shown in Figure 6.

It can be seen from Figure 6 that the parallel POD is obviously more efficient than the non-parallel POD. As the number of IGS/MGEX stations increases, the serial calculation time of non-parallel and parallel POD both gradually increases. The speedup ratio of parallel POD increases with the increase in the number of stations. The efficiency of POD using 80 to 90 stations has been significantly improved. Moreover, the parallel POD of four satellite system can be controlled within one hour, which provides the possibility to increase the update frequency to 1-h for multi-GNSS parallel POD.

3.3. Analysis of the Accuracy of Parallel Ultra-Rapid POD

Aiming at the problem that the predicted orbit accuracy of GNSS ultra-rapid products becomes worse as the extrapolation time is longer, we apply the parallel approach to the multi-GNSS ultra-rapid POD. Based on the results of the improvement of efficiency with different number of stations in the previous section, 88 IGS/MGEX stations are selected for multi-GNSS ultra-rapid POD. The distribution of stations is shown in Figure 2.

3.3.1. Improvement of Accuracy with Increased Update Frequency by the Parallel POD

The update frequency of the original ultra-rapid orbit is 6 h, and the update frequency can be increased to 1 h after applying the parallel POD. The accuracy of orbit with 6-h update frequency and 1-h update frequency are compared in this section. Observation data from DOY 182 to 189, 2018 are selected for multi-GNSS ultra-rapid POD. The accuracy of GPS, GLONASS, Galileo, and BDS predicted orbit with parallel and non-parallel POD are analyzed, which are shown in Figure 7, Figure 8, Figure 9 and Figure 10.

From Figure 7, Figure 8, Figure 9 and Figure 10, it can be seen that the accuracy of the predicted orbit of the 6-h update frequency fluctuates at different extrapolation times. The accuracies of orbit at 0 o’clock, 6 o’clock, 12 o’clock and 18 o’clock are the highest, and then gradually decrease. After the update frequency is increased to 1-h by parallel approach, the accuracy of predicted orbit is higher and more stable at different extrapolation time. The statistical results of the accuracy of GPS/GLONASS/Galileo/BDS orbit are shown in

Table 7. Accuracy of ultra-rapid predicted Multi-GNSS orbit updated per 6 h.

	R (cm)	A (cm)	C (cm)	1D RMS (cm)	Improvement of Accuracy
GPS_6	2.3	6.8	2.7	4.8
GPS_1	2.3	3.9	2.2	3.2	33.4%
GLO_6	3.2	10.4	4.4	7.4
GLO_1	3.1	6.4	3.6	5.1	31.4%
Gal_6	6.3	12.4	4.4	9.3
Gal_1	5.8	5.2	3.6	5.6	40.1%
BDS_6	14.3	21.0	6.9	16.9
BDS_1	12.2	12.0	5.6	11.3	32.8%

. Selecting the precise orbit of IGS and CODE analysis center as reference, the along, normal, radial and 1D mean RMS values of predicted orbit of 1-h update frequency and 6-h update frequency are compared.

It can be seen from the table that the accuracy of GPS, GLONASS, Galileo, and BDS 1-h updated orbits are improved in normal, radial and along direction, and the radial direction is improved the most compared to the 6-h updated orbit. The RMS of the predicted orbit of GPS, GLONASS, Galileo and BDS are 33.4%, 31.4%, 40.1% and 32.8% higher than those of the original 6-h results, respectively.

3.3.2. Evaluation of Predicted Orbit Accuracy of Ultra-Rapid POD by Parallel Approach

In order to further analyze the performance of the parallel approach, the accuracy of the ultra-rapid predicted orbit is evaluated. The GPS/GLONASS precise orbit of IGS and the Galileo/BDS precise orbit of CODE are selected as reference values. The experiment includes three groups. The first one uses the proposed parallel POD approach with a 1-h updated result (denoted as SDU-1h). The second one uses the non-parallel POD approach with a 6-h updated result (denoted as SDU-6h), and the third one is the 1-h updated result of Wuhan University analysis center (denoted as WHU-1h). The statistical results of accuracy are shown in Figure 11.

From Figure 11, it can be seen that as the extrapolation time increases, the orbit accuracy of SDU-6h decreases in the orbit sequence. The phenomenon that the orbit accuracy fluctuates with extrapolation time does not exist in WUH-1h and SDU-1h. The accuracy of WUH-1h is better than that of SDU-6h result, and it is relatively stable except for occasionally large outliers. The predicted orbits of GPS, GLONASS, Galileo, and BDS of SDU-1h have the highest accuracy and the most stable performance. Combined with the statistical results in

Table 8. Statistics of the accuracy of ultra-rapid predicted orbit (cm).

Satellite System	Scheme 1	Scheme 2	Scheme 3
GPS	3.2	4.8	4.3
GLONASS	5.1	7.3	6.5
Galileo	5.6	9.2	11.2
BDS	11.8	16.9	16.4

, the 1D mean RMS of the ultra-rapid predicted orbit with the proposed parallel approach is 3.2 cm, 5.1 cm, 5.6 cm and 11.8 cm, respectively.

The accuracy of the predicted orbit of each satellite from GPS/GLONASS/Galileo/BDS are plotted in Figure 12, and BDS-2 MEO and BDS-2 IGSO of the BDS system are used in this processing. It can be seen from these figures that the predicted orbits of GPS, GLONASS, Galileo and BDS determined by the parallel approach with 1-h update frequency have the highest accuracy. The accuracies of GPS, GLONASS, Galileo and BDS predicted orbit are mostly better than 3 cm, 5 cm, 6 cm, and 12 cm, respectively. One of the GPS satellites, G18, has poor accuracy, which may be due to its large orbit error during the shadow period. In general, the accuracy of the ultra-rapid predicted orbit of each GNSS satellite with the parallel approach and 1-h updated frequency have greatly improved.

4. Conclusions

A parallel ultra-rapid POD approach has been proposed in this study. The parallel approach properly makes use of parallel computing to improve the efficiency of multi-GNSS orbit determination. The construction of normal equations, parameter elimination and orbit extrapolation in the GNSS POD algorithm are optimized based on MPI/OpenMP. The following conclusions can be drawn:

(1) The precision loss and feasibility of parallel POD are verified. The results show that 1D mean RMS of GPS, GLONASS, Galileo and BDS orbit in non-parallel mode are 1.3 cm, 2.7 cm, 2.7 cm, and 5.4 cm, respectively. Moreover, the precision loss of using the parallel approach is at the sub-millimeter level, which can be ignored, thus verifying the feasibility of the parallel method.

(2) The improvement of efficiency by the parallel approach is analyzed in detail, including multiple factors such as the number of processes, the number of threads, the number of stations and the number of satellite systems. The results show that with the increase of the number of threads and processes, the efficiency of the parallel approach increases more obviously. The approach is more applicable to large reference networks and multiple satellite systems.

(3) The superiority of the parallel approach used in multi-GNSS ultra-rapid POD is verified. The improvement of accuracy of predicted orbit with an increased update frequency of 1-h is analyzed in detail. The 1-h updated orbit determined by the parallel approach is compared with the 6-h updated orbit determined by the non-parallel approach, and the results show that the accuracy of GPS, GLONASS, Galileo and BDS are 33.4%, 31.4%, 40.1% and 32.8% higher than the 6-h results of the non-parallel approach, respectively. Additionally, the accuracy of the 1-h updated orbit is evaluated by comparing with IGS and CODE precise orbit products. The 1-h orbit product of Wuhan University is selected for comparison. The results show that the orbit with 1-h update frequency by the parallel approach has higher accuracy and stability. Further, statistical results show that the 1D mean RMS of the ultra-rapid predicted orbit of GPS, GLONASS, Galileo, and BDS with the proposed parallel algorithm is 3.2 cm, 5.1 cm, 5.6 cm and 11.8 cm, respectively.

In summary, the proposed parallel approach can greatly improve the efficiency of multi-GNSS ultra-rapid orbit determination and enhance the update frequency from 6-h to 1-h, which can help for the improvement of the ultra-rapid orbit accuracy and stability. The proposed parallel approach makes it possible to use a denser network of reference stations for multi-GNSS ultra-rapid POD with high update frequency for real-time users.

It should be noted that we adopted the ECOM1 SRP model for the one-step orbit determination of the GPS, GLONASS, Galileo and BDS in this study. For the specific elongated shape of Galileo satellites, orbit accuracy will be improved by using a prior model. In addition, clock determination is not related in this study, making it difficult to get the Signal In Space Range Error (SISRE). In the next studies, we will use an a priori model to reduce the orbit perturbations for Galileo, and we will combine the orbit and clocks through the processing in order to calculate clocks and orbital SISRE for the analysis of the algorithm.