A Novel Closed-Loop Structure for Drag-Free Control Systems with ESKF and LQR

Space-borne gravitational wave detection satellite confronts many uncertain perturbations, such as solar pressure, dilute atmospheric drag, etc. To realize an ultra-static and ultra-stable inertial benchmark achieved by a test-mass (TM) being free to move inside a spacecraft (S/C), the drag-free control system of S/C requires super high steady-state accuracies and dynamic performances. The Active Disturbance Rejection Control (ADRC) technique has a certain capability in solving problems with common perturbations, while there is still room for optimization in dealing with the complicated drag-free control problem. When faced with complex noises, the steady-state accuracy of the traditional control method is not good enough and the convergence speed of regulating process is not fast enough. In this paper, the optimized Active Disturbance Rejection Control technique is applied. With the extended state Kalman filter (ESKF) estimating the states and disturbances in real time, a novel closed-loop control structure is designed by combining the linear quadratic regulator (LQR) and ESKF, which can satisfy the design targets competently. The comparative analysis and simulation results show that the LQR controller designed in this paper has a faster response and a higher accuracy compared with the traditional nonlinear state error feedback (NSEF), which uses a deformation of weighting components of classical PID. The new drag-free control structure proposed in the paper can be used in future gravitational wave detection satellites.


Introduction
Gravitational wave astronomy provides a new tool to explore black holes, dark matter, the early universe, and the evolution of the universe. To detect gravitational waves in space, a strategy involves deploying multiple satellites in mega-satellite formations to measure tiny changes in the relative distances between satellites when the gravitational wave goes through. However, the challenge lies in the weak characteristics (to the order of 10 −21 ) in the changes caused by gravitational waves, which can be easily influenced by extraneous perturbations and noise. To address this issue, the typical approach is to employ drag-free satellites: an inner test mass is shielded and free-falls along the geodesic of spacetime, and the outer satellite counteracts non-conservative forces and tracks the test mass in a sensitive axis. This setup creates an ultra-static and ultra-stable platform, with the test mass serving as an inertial reference for the measurement of relative distances in space. Using this approach, it is possible to obtain accurate and reliable measurements of distance variation. And the variation represents the effect of gravitational waves.
In the drag-free control loop, there are many factors that can impact the effect of the controller, such as external environmental disturbances, sensor measurement noise, process noise, and other inevitable disturbances and noises. Models of these disturbances and noises are hard to be built precisely, which makes it difficult to determine the appropriate models for compensation. To address this challenge, the Extended State Observer (ESO) method has emerged as an effective solution for modeling, estimating, and identifying the estimation and compensation of perturbations without considering the suppression of noise.
The accuracy required for drag-free and attitude control in gravitational wave detection is crucial, so the impact of noise must be considered. Previous methods are insufficient in dealing with the noise affecting gravitational wave detection satellites, and cannot estimate the perturbations effectively. Those methods also suffer from longer setting times. To design a successful control system, it is necessary to develop effective strategies to reduce the noise impact on control performance. In this study, we propose a novel approach that combines ESKF with LQR control. We use the state and disturbance estimated by ESKF as the input information for the controller, ensuring optimized control. Our analysis and simulations show that this new approach outperforms traditional solutions. It effectively shortens the adjustment time, reduces the number of oscillations, compensates for disturbances, and suppresses noise, ultimately achieving the desired design specifications.
The paper is organized as follows: in Section 2, a dynamic model of a single test mass drag-free satellite is established. In Sections 3 and 4, the design process and calculation methods of the ESKF and LQR are presented, respectively. In Section 5, the performance of the system using the LQR controller and NSEF controller is compared through numerical simulations, indicating that the overall performance of the LQR controller is superior to that of the NSEF controller when using ESKF as the estimation method. The conclusion is given in Section 6.

Dynamics Modeling
This paper takes a single test mass, a drag-free satellite in geocentric orbit as the research subject, as is shown in Figure 1, where C indicates the center of mass of an object, h represents the sensitive cavity, which is fixed to the satellite, then the position vector r h from the center of the sensitive cavity to the center of mass (CoM) of the satellite is constant, and the position vector of test mass relative to the satellite is r = r h + r rel . The relative translation equations of motion in the inertial system are first transformed into the satellite body coordinate system, similarly the relative attitude equations of motion are projected into the TM body coordinate system, which is illustrated in Figure 2. Then a comprehensive drag-free satellite dynamics model can be established as follows.
where w T Csc indicates input noise, here assume that all input forces and moments acting on the satellite, as well as the measurement output of the sensor, are subject to noise, T Dsc denotes the disturbance moment to the satellite, F Dtm and F Dsc denote the test mass and the disturbance force on the satellite, respectively, and T Dtm indicates the disturbance moment to the test mass. The sc, tm, rel, C, D, and G subscripts indicate the S/C, Test Mass, measurements RELated to the sensitive cavity, Control command, Disturbance, and Gravity. The h superscript indicates the components in the sensitive cavity frame, and A TS is the coordinate transformation from the satellite frame to the test mass frame. The dynamic model is expressed in the form of state space equations which are presented as follows:Ẋ where where K trans , D trans , K rot , D rot are the coupling coefficient matrices for translation and rotation, respectively. u is the system control variable, w is input noise, d is measurement noise, and f represents the total perturbation affecting the system, including the known part and the unmodeled part. The Gravitational Wave Detector-TianQin requires detection satellites in deep-space orbit. The main disturbance on the satellite comes from solar pressure. To ensure a steady power supply and minimize fluctuations in the satellite's internal thermal environment for ultra-stability, the drag-free satellite uses a body-attached battery array.
The expression of this perturbation is shown below: where C R indicates surface reflection coefficient, normally 1-1.44, ρ SR indicates the solar pressure near the Earth, 4.56 × 10 −6 N/m 2 , S R m is the surface-to-mass ratio of the spacecraft, S R is the projected area of the spacecraft facing the sun, r s is the unit vector indicating the direction from the center of the Earth to the sun. The sun exposure factor, denoted as k, is assumed to be 1 for the light area and 0 for the ground shadow area. The amplitude spectral density of the solar pressure on the satellite is shown in Figure 3. In a drag-free satellite with a single test mass, the displacement measurements between the CoM of the TM and the CoM of the satellite, as well as the attitude measurements of the TM relative to the satellite, are obtained by an inertial sensor. The attitude of the satellite is determined through a star sensor, and the micro-propulsion provides the necessary control forces and moments to maintain the desired position and attitude of the satellite. At present, the typical micro-propulsion systems have a noise power spectral density of 1 × 10 −6 N/ √ Hz under open-loop condition [22]. Their corresponding power spectral densities are presented in   Based on the data of Ref. [23], it can be inferred that electrostatic actuation noise can be treated as white noise in the frequency band needed for the detection of gravitational waves by the TianQin detector. In the case of capacitive displacement sensors, the noise levels for displacement measurements are equal in the x, y, and z directions, while the noise levels for angle measurements in the θ direction are one order of magnitude lower than those in the η and φ directions [23]. Figure 6 displays the measurement noise in each direction. The spectral density curves of perturbation and noise are given in Figures 3-6, which will be used as the basis for the modeling and simulation calculations later.

Extended State Kalman Filter Design
Achieving high accuracy of relative displacement and relative attitude control in a noisy and disturbed environment requires multiple steps, including disturbance estimation, noise suppression, and state control.
The Extended State Kalman Filter (ESKF) can estimate nonlinear uncertainty. In cases of initial error, uncertain dynamics (perturbation), and bounded noise, the perturbation is estimated and compensated for by the extended state, and noise effects can be suppressed. This paper uses ESKF to estimate disturbance forces and moments, such as solar pressure on the satellite and anomalous electromagnetic forces and moments on the test mass. First, we present the design scheme of ESKF, then apply it to uncertain disturbance estimation in drag-free control.

Extended State Kalman Filter
For the following discrete system containing uncertain perturbations where W k is the system state, A k , B k are system matrices, C k is the measurement matrix, f (W k , k) is the nonlinear uncertain part in the system (6), w k , n k are the process noise and measurement noise, respectively, and Y k is the system measurement output. Treating f (W k , k) as an additional state variable f k , which is then estimated and compensated for. The extended system is described as assume w k , n k are unrelated zero-mean Gaussian random series and E n k n k is the initial value of the nominal part of f (W k , k), P 0 is a known constant matrix, and According to the classical state observer theory, the extended state observer for the extended state Equation (7) is shown below, where X k+1 = W k+1 f k+1 Based on the given initial estimationX 0 and initial value of covariance matrix P 0 , we obtain the middle value of the estimated quantityX − k and update the value of the covariance matrix P k where θ = tr(Q 1,0 ) tr(P 0 ) is used to decouple the cross terms of estimation error and uncertainty, by calculating K k , update the estimationX k and the covariance matrix P k After that, calculate the control variable u 0 based on the error between the state estimate and the reference, then the calculated control variable u 0 and the estimated valuê f of the disturbance are used to calculate the final control variable u, and the flow chart is shown in Figure 7.

Extended State Design of Drag-Free Control System
The complex space environment presents a challenge in accurately modeling and describing perturbations affecting on satellite and test masses. Conventional control methods are model-dependent, and their performance can be severely affected by the inaccuracies of the perturbation model. To address this issue, Section 3.1 proposes a method to expand the perturbations into new states and create a new filter model. Then the perturbation can be compensated and the noise can be suppressed.
The uncertain disturbance term is f = [T Dsc , F Dsc , T Dtm ] T . T Dsc represents the disturbance moments that the satellite is subjected to, F Dsc represents the disturbance forces that the satellite is subjected to, and T Dtm represents the disturbance moment that TM is subjected to. Those are treated as extended states. The state vector is taken as X = [ϕ sc ,φ sc , r rel ,ṙ rel , ϕ rel ,φ rel , T Dsc , F Dsc , T Dtm ] T , Y = [ϕ sc , r rel , ϕ rel ] T indicates related attitude and displacement that can be measured.
According to Section 3.1, the extended state differential equation is obtained aṡ where T , symbols has the same physical meaning as in Equation (4). The extended state Equation (15) is discretized to obtain the discrete difference equation model (16) where G k ∆ = f k+1 − f k . The ESKF is designed according to the flowchart shown in Figure 7 to estimate the kinematic state parameters and the unknown disturbances, while implementing feedback control. Where q i = (max| f i+1 − f i |) 2 , Q 0 = idiag(q i ), i is the number of state vector.

Feedback Controller Design
Once the estimation of states and perturbations has been successfully obtained, the next step is the selection of an appropriate controller. The Linear Quadratic Regulator (LQR) is a control strategy designed to minimize a cost function. The optimal control law is obtained through the design of the state feedback controller, which allows for the completion of closed-loop optimal control in a fast, stable, and accurate manner.
The performance index for LQR control reflects the requirements for both state and control quantities, and the cost function used in this paper is The weighting matrix Q is semi-positive definite and R is positive definite, which are set as a diagonal matrix in the subsequent simulation. For the first term in the cost function J, each component is required to be small in the control process. The larger the weight in Q means the stricter the constraint on the components; while the second term in the cost function indicates the requirement for the control output, which is weighted according to the different characteristics of each component.
The Ricatti equation PA + A T P − PBR −1 B T P + Q = 0 is used to calculate P, then based on K = R −1 B T P, the feedback gain matrix is calculated, the control law of LQR is chosen as u(k) = −Kx(k).
In order to maximize the effective time of gravitational wave detection and minimize the output of thrusters (actuators), it is necessary to ensure that the detection satellite is maintained in a free and stable flight for as long a time as possible. To achieve this goal, the adjustment transition process of drag-free control needs to be as fast as possible, allowing the detection system to quickly reach an ultra-static and ultra-stable state. Our research shows that the LQR control strategy can satisfy the optimal control requirements, enabling a rapid transition to a steady state and achieving the required control accuracy. By appropriately designing the weights of each component, in addition to the values of Q and R, it is possible to achieve a more refined control.
Nonlinear error feedback is used in classical Active Disturbance Rejection Control [1], mainly by rewriting the weighting of classical PID control into a nonlinear combination, as shown in Equation (18) f al(e, α, δ) = e δ α−1 , |e| ≤ δ sgn(e)|e| α , |e| > δ It is a continuous power series function with linear segments near the origin, δ > 0 denotes the length of the interval of the linear segment, and e indicates the amount of error.
This function f al( ) is characterized by increasing the gain when the error is small and using a small gain when the error is large, which prevents high-frequency chattering due to excessive gain calculated when the error is small [24,25]. The control law for nonlinear error feedback is where e 1 (k) = r −x(k), e 2 (k) = k ∑ 0 (r −x(k)), e 3 =ṙ −x(k). Parameter triples γ 1 , γ 2 , γ 3 determine the final control variable, the value of δ is generally selected as the sampling time for discrete systems, the value of α satisfies α ∈ (0, 1). A simulation program using the combination of ESKF and LQR is developed, then the comparison results of the combination of ESKF+NSEF and the combination of ESKF+LQR is presented and analyzed in the next section.

Simulation Analysis
According to the discrete model (16), a block diagram of the drag-free satellite control system is designed and presented in Figure 8. The measurement mechanism provides information on the attitude angle of the satellite, the displacement, and the attitude of the test mass relative to the center of the cavity. The output command of actuators and output measurement information of the sensors are both inputs of ESKF. The ESKF estimates the attitude, angular velocity, displacement, velocity, disturbance forces, and disturbance moments. By selecting an appropriate controller with the ESKF, high-accuracy anti-disturbance control of the drag-free satellite/test mass dynamic system is achieved. First, we consider the LQR controller. The ESKF filter design discussed in Section 3 was simulated in MATLAB with a time step of 0.01 s. The ESKF consists of 27 states, which are divided into three groups of nine states each. After the ESKF output was stable and tracking accurately. The LQR control algorithm was then employed for designing the control law, and the corresponding simulation results were obtained.
Initial conditions of simulation: The perturbation forces and moments of the satellite are modeled as constant superposition sinusoidal perturbations with phase differences in each axial direction. Specifically: Hz. The simulation program will achieve real-time estimation of the extended state for the above perturbations, and the results of the error analysis of the estimation are given later. The expectation of the input noise of the thrusters providing force and moment to the satellite are 1 × 10 −9 N/ √ Hz and 1 × 10 −9 N · m/ √ Hz, respectively. The input noise expectation of the electrostatic actuator providing the test mass control torque is 1 × 10 −15 N · m/ √ Hz, the expectation of the measurement noise of the star sensor providing satellite attitude measurement is set as 1 × 10 −7 rad/ √ Hz. And the expectation of displacement measurement noise of inertial sensor is 1 × 10 −9 m/ √ Hz, the expectation of attitude measurement noise is 1 × 10 −8 rad/ √ Hz. To meet the requirements of gravitational wave detection, the control loop's design objective is set as follows: the setting time should be less than 1 min, and the amplitude spectral density of the relative displacement between test mass and satellite should both be less than 10 −8 m/ √ Hz within the detection frequency band.
The performance of the combination of LQR and ESKF is evaluated initially. Subsequently, the performance of using the combination of LQR and ESKF is compared with the performance of the combination of NSEF and ESKF in terms of control accuracy and setting time. The intrinsic reasons for any differences observed are analyzed, and recommendations for engineering design are provided.

ESFK+LQR
By utilizing the ESKF-estimated states as input, we designed an LQR controller based on Equation (17). The initial value of state estimation wasX 0 = [0, 0, 0, 0, 0, 0, 0, 0, 0] T , and the values of Q and R were selected as R = 1 × 10 −4 diag(I 3 , 10I 3 , I 3 ). It should be noted that a larger value of Q can facilitate faster convergence of the states for the LQR controller. Figure 9 demonstrates the effectiveness of the ESKF and LQR controllers in controlling the attitude angle of the satellite. The results show that the attitude angle was successfully controlled from the initial ϕ sc = [7 × 10 −5 , 0, 0] T rad to ±3 × 10 −8 rad for all three attitude angles with a setting time of about 7 s, achieving the desired control target. The estimation error of the attitude angle was measured to be ±2 × 10 −8 rad, while the estimation error of the attitude angular velocity and disturbance moment were ±2 × 10 −10 rad/s and ±1 × 10 −5 N · m, respectively. The amplitude spectral density curves for each attitude angle control error are presented in Figure 10. Based on the results shown in Figure 10, it can be observed that within the measurement bandwidth, the amplitude spectral density of the satellite's attitude control error conforms to the design requirements of 10 −7 rad/ √ Hz.

Test Mass and Satellite Relative Displacement
The results in Figure 11 show that the relative displacement was successfully controlled from r rel = [0.0005, 0.0009, −0.0006] T m to ±2 × 10 −9 m for all three axes within an adjustment time of 20 s, achieving the desired control target. The position estimation error was measured to be ±2.2 × 10 −9 m, while the velocity estimation error and disturbance estimation error were ±4 × 10 −10 m/s and ±4 × 10 −8 N, respectively.
The amplitude spectral density curves for each axial direction are presented in Figure 12. Based on the observations from Figure 12, it can be concluded that the kinematic indexes of the translation within the frequency band satisfy the design requirements and achieve 10 −8 m/ √ Hz.  Figure 13 shows the simulation results of ESKF and LQR controller dealing with the relative attitude between test mass and satellite. Based on the results from Figure 13, it can be concluded that the ESKF and LQR controllers designed in this paper were capable of controlling the relative attitude between the test mass and the satellite from ϕ rel = [0, 4 × 10 −5 , 0] T rad to ±5 × 10 −8 rad for all three axes. The control result was found to be essentially oscillation-free, with an adjustment time of about 6s, achieving the desired control target. The estimation error of the attitude angle was measured to be ±5 × 10 −8 rad, while the estimation errors of the attitude angular velocity and disturbance moment were ±6 × 10 −8 rad/s and ±2 × 10 −11 N · m, respectively. The amplitude spectral density curves for each attitude angle are presented in Figure 14. The amplitude spectral densities are lower than 5 × 10 −8 rad/ √ Hz in the whole frequency band.

Comparison Evaluation
To demonstrate the efficacy of the designed method, we present a simulation to compare ESKF+LQR with ESKF+NSEF. This allows us to illustrate the functions of our objectives and evaluate their performance against an established method. Simulation results is given in Since the convergence facility of the f al( ) function used in NSEF is mainly based on the value of α, while the proof of certain physical relation is extremely complicated, the value of α is determined by rule of thumb. Based on the method in Ref. [26], we take α 1 = α 2 = α 3 = 0.5 in simulation. The simulation results of relative displacement of the x-axis are shown in Figure 16, while the control accuracy is satisfied, and the convergence process of ESKF+LQR was found to be much faster than that of ESKF+NSEF. The RMS of the two approaches is listed in Table 1. Additionally, the LQR controller was observed to produce negligible oscillations in the system, which can be advantageous in terms of reducing energy consumption and extending the effective time of gravitational wave signal detection. Table 1. RMS of two different control approaches.

Conclusions
Integrating perturbations into states vector, the ESKF method demonstrates the capability to accurately estimate the state and disturbance of the drag-free satellite dynamics, and effectively suppressing handling noise. This approach lays the foundation for the controller to achieve accurate adjustment of the system state, and it is recommended to consider the ESKF method as a viable alternative for estimating uncertain disturbances in future drag-free engineering designs.
The LQR controller's feedback parameter matrix can be rigorously derived using the generalized index to ensure optimal control performance. By combining the reasonable values of the state covariance matrix, the designed gain matrix guarantees the convergence speed of the system state, while the accurate estimation of ESKF enhances the relative attitude control accuracy by an order of magnitude.
For the relative displacement, the adjustment time of NSEF fails to reach the control task requirements. This is mainly due to the relatively slow control process of the relative kinematic parameters between the satellite and the test mass in NSEF, and the lack of a complete theoretical method to adjust and optimize the nonlinear error feedback parameters.
In summary, the ESKF+LQR control approach enables the system to reach a steady state rapidly and smoothly, thereby increasing the free flight time available for gravitational wave observation. This is more in line with the desired observation duration for gravitational wave detection. In comparison to the combination of ESKF+NSEF, the combination of ESKF+LQR is capable of adjusting to the reference target swiftly and reducing oscillations, leading to a reduction in thruster mass consumption.