Estimation of Vehicle Attitude, Acceleration, and Angular Velocity Using Convolutional Neural Network and Dual Extended Kalman Filter

The acceleration of a vehicle is important information in vehicle states. The vehicle acceleration is measured by an inertial measurement unit (IMU). However, gravity affects the IMU when there is a transition in vehicle attitude; thus, the IMU produces an incorrect signal output. Therefore, vehicle attitude information is essential for obtaining correct acceleration information. This paper proposes a convolutional neural network (CNN) for attitude estimation. Using sequential data of a vehicle’s chassis sensor signal, the roll and pitch angles of a vehicle can be estimated without using a high-cost sensor such as a global positioning system or a six-dimensional IMU. This paper also proposes a dual-extended Kalman filter (DEKF), which can accurately estimate acceleration/angular velocity based on the estimated roll/pitch information. The proposed method is validated by real-car experiment data and CarSim, a vehicle simulator. It accurately estimates the attitude estimation with limited sensors, and the exact acceleration/angular velocity is estimated considering the roll and pitch angle with de-noising effect. In addition, the DEKF can improve the modeling accuracy and can estimate the roll and pitch rates.


Introduction
In recent decades, the vehicle controller has been developed significantly for stability and user convenience. Typically, the electronic stability controller (ESC) or active roll stabilization (ARS) are used for ensuring vehicle stability in chassis controller, adaptive cruise control (ACC), or lane keeping system (LKS) as well as for ensuring convenience in advanced driver assistant system (ADAS). To improve the performance of these controllers, vehicle states must be estimated with a high accuracy. The commonly required and important state information of aforementioned controllers are acceleration and angular velocity. In a vehicle, the inertial measure unit (IMU) measures the acceleration and angular velocity using an inertial force. However, if a transition in attitude (roll and pitch) occurs, the gravitational force is reflected in the sensor value. The IMU cannot distinguish between gravitational force and inertial forces; therefore, a change in the attitude of the vehicle causes a fatal error in the IMU.
Several studies have been conducted to overcome the errors in these accelerometers and to estimate the exact state of the vehicle. In [1,2], the adaptive Kalman filter was designed to minimize the effect of the accelerometer offset errors. In addition, the Kalman filter was used to estimate the accelerometer offset and vehicle velocity [3][4][5]. However, the accuracy of estimating vehicle states was limited because of a lack of information of the vehicle attitude. The remainder of this paper is organized as follows. Section 2 describes the related literature review. Section 3 explains the methodology including neural network and DEKF. Section 4 describes the results of performance verification. Section 5 summarizes the study and contributions.

Literature Review
Several studies have been conducted to estimate the accurate vehicle attitude in various ways. In [8], authors proposed an observer which can estimate the land vehicle's roll and pitch by using an IMU and the kinematics model. Also, the adaptive Kalman filter was proposed based on IMU aided by vehicle dynamics [9]. However, these methods have a low accuracy in dynamic situations. This problem was attempted to be solved in [10][11][12][13], based on the Kalman filter by compensating for the external acceleration that interfered with the estimation of the attitude, but the accuracy was limit.
Therefore, many researchers have proposed sensor fusion method using not only IMU but fusion with global positioning system (GPS). Representatively, the method in which IMU and GPS fusion with Kalman filter [14][15][16] or sliding mode observer [17] can be used.
These methods can increase the accuracy, but they require high-cost sensors such as GPS and are highly affected by GPS performance. Accordingly, vehicle attitude estimator studies were conducted without the use of GPS, based on the characteristics of the vehicle dynamics. Using vehicle roll dynamics, a dynamic observer design based on a reliable rollover index [18] was proposed. Also, the Kalman filter [19] and the robust observer [20] based on vehicle roll dynamics which estimate roll angle were proposed. Alternatively, using lateral dynamics, the research which estimate road bank angle [21,22] was proposed. However, these methods exhibited limited accuracy in transient situations. A method for estimating both roll and pitch attitudes using a six-dimensional IMU and bicycle model [23,24] was also studied. These methods exhibited a high accuracy, but they have a short validation range and required hard-to-get data such as six-dimensional IMU.
Several recent studies have also been conducted to estimate the vehicle roll and pitch angle using neural network. In [25][26][27], the vehicle roll angle estimation method using sensor fusion with a neural network and Kalman filter was proposed. Furthermore, a vehicle roll and road bank angle estimation method based on a deep neural network was introduced [28]. However, the pitch attitudes were not estimated, and the six-dimensional IMU data such as roll rate or vertical acceleration were required, which complicated the estimation. Literature reviews are summarized in Table 1. Table 1. Literature review summarizing.

Feature Selection
Feature selection is based on vehicle dynamics, sensor usability, and attention mechanism. The purpose of neural network is to design a regression model for calculating the roll and pitch angles, so select the primary feature candidates for the vehicle's roll and pitch dynamics. Vehicle roll dynamics [29] can be written as: where M x is the moment respect to the roll axis, h R is the height of the from center of gravity to roll center, m s is the vehicle sprung mass, g is the gravity acceleration, φ is the roll of vehicle, k s is the stiffness of the suspension, l s is the length of the wheel track, b s is the damping coefficient of the suspension, and h CG is the height of the center of gravity from the ground. The lateral force applied to the vehicle can be expressed by (2) based on the bicycle model. C α f and C αr are the cornering stiffness values of each front and rear tire, respectively, δ is the steering angle, v x and v y are the velocities of each x and y axes, respectively, l f and l r are the length of each front and rear wheel axis from the center of gravity, and . ψ is the yaw rate.
The vehicle pitch dynamics can be written as [29]: where M y is the moment with respect to the pitch axis and θ is the pitch of the vehicle. The traction force applied to the vehicle can be expressed by (4) based on the vehicle drive-line dynamics. N t f is the gear ratio of the transmission and differential gears, η t f is the efficiency of the power transfer from the engine to the wheel axis, r e f f is the effective radius of the tire, T e is the engine torque, I e , I t , I d , I w are the inertia of each engine, transmission, differential gear, and wheel, respectively, and a x is the longitudinal acceleration of the vehicle.
Brake force applied to the vehicle is given by (5). P brake is the pressure of the master cylinder, A caliper is the area of the brake pad caliper, µ pad is the friction coefficient of the brake pad, and R b is the distance from the wheel center to the brake pad.
Based on (1)-(5), we can choose only variables, except for static parameters such as the vehicle mass. As a result, a total of 13 feature candidates were selected, including the acceleration and yaw rate. Subsequently, seven features were selected considering the availability of sensors in the experimental vehicle. Table 2 shows the selected features. Based on these, we conducted an analysis of importance based on the attention mechanism [30]. The results of the importance analysis are presented in Appendix A. According to Appendix A, the final features were selected same as Table 2. Before configuring a 2-d input to the CNN, we calculate φ static,pseudo and θ static,pseudo reflecting the static roll and pitch angles using feature sensor values. The pseudo values are written as: where a,b,c,d are the constant design parameters. (5) and (6) are similar to the road bank and the slope angle, respectively when the vehicle's wheel speed is zero. If the vehicle's wheel speed increases, (5) and (6) become zero. The neural network architecture is shown in Figure 2. The network is composed of CNN part and fully connected layer (FCL) part in parallel configuration. The input array is composed of a mux of each sequential feature sensor data including (5), (6). The sequential information is 2 s for 0.01 s, therefore the input array size is (200 × 9). CNN part uses all inputs, namely all-time series data in the past 2 s, but FCL part uses only the last row of the input array, meaning only current step data. This means that the CNN part is designed with the intention of estimating dynamic vehicle body changes while driving and FCL part is designed to estimate the roll and pitch angles in static scenarios. The CNN part comprises four convolution layers and two fully connected layers. The first layer converts the input matrix into a square matrix and shuffles the sensor placement order. Then, it passes through the three convolution layers, and then pass the one convolution layer with large size filter and wide strides for compressing the data. Next, unfold to fully connected layer and make the last layer's size (256 × 1). In FCL part, there are four layers and final layer's size is also (256 × 1). The two final layers are concatenated, and then the regression model is constructed, which calculates two outputs using one fully connected layer. Hyperparameters of the neural network such as the number of filters or activation function are described in Appendix B.

State Space Model
The state space model is based on the vehicle 6-DOF sprung mass model [31,32] for expressing the six-dimensional acceleration and angular velocity. The dynamics are composed based on the Euler rigid body equation and the force or moment of each axis which is given by each dynamic [29]. Figure 3 shows the six-dimensional motion of vehicle sprung mass. The forces or moment of each axis are given by: where ΣF x , ΣF y , ΣF z are the sum of forces of the x, y, and z axes, respectively. ΣM x , ΣM y , ΣM z are the sum of moments of the x, y, and z axes, respectively. I x , I y , I z are the moments of inertia along the x, y, and z axes, respectively. ΣF x , ΣF y , ΣF z , ΣM x , ΣM y , and ΣM z can be derived from a vehicle model such as a bicycle model. Therefore, the nonlinear state space equation can be written as: . . . . .
. ΣF drive , ΣF brake , and ΣF y have been described previously. The state vector ψ φ θ , state input u = [T e P brake δ]. ρ is the density of the air, C d is the drag coefficient and A is the frontal area of the vehicle.
State outputs include longitudinal acceleration, lateral acceleration, and yaw rate from IMU. The roll and pitch angles from the neural network are also included. Thus, the state output y = a x a y . ψ φ θ and it can be written as: . ψ = x 6 (24)

Observability Check
Before designing the estimator, the observability must be checked. The observability of the nonlinear state space model can be checked by the Lie derivative [33]. When the state space equation is expressed as . x = f (x, u), y = h(x), the Lie derivative and observability matrix can be written as: where O is the observability matrix. Using the rank of the observability matrix, the system's observability can be seen locally. As a result of checking the rank of the observability matrix, the observability matrix has full rank in range of v x = 0; therefore, this system is locally observable for the range except v x = 0.

Dual Extended Kalman Filter Module
Among the vehicle dynamics parameters, the cornering stiffness varies under conditions such as the vehicle load. To reduce the errors while modeling, the cornering stiffness should be estimated.
This study adopts the DEKF as an estimator for reducing the error and increasing the modeling accuracy. Figure 4 shows the DEKF scheme. The state vector, state input, state output, and state space equation have been discussed in Section 3.2.1. The DEKF module works according to the following recursive algorithm [34]: State prediction: State update: Parameter update: where the parameter vectorx p = C α f C αr ; P p and P s are the error covariance matrices for parameters and states, respectively; Q p and Q s are the process noise covariance matrices for parameter and state estimators, respectively; R p and R s are the output noise covariance matrices for parameter and state estimators, respectively. R p and R s are the same because both the parameter and state estimator have same output y. K p and K s are the Kalman gain matrices for parameter and state estimators, respectively. F s and H s are the Jacobian matrices for the state and output equations, respectively, and are expressed as follows: In Equations (36) and (38), H p is the Jacobian matrix of state output for the parameter estimator and can be expressed as: R p and R s can be determined by the engineer's tuning based on sensor noise. Q s and Q p can also be determined by engineer's tuning but Q p 's square of each elements is tuned to approximately 1% of the initial values of the each actual parameter.

Results and Analysis
This section presents and discusses the experimental verification results of the algorithms mentioned in Section 3. Neural network and DEKF are discussed separately in Sections 3.1 and 3.2, respectively. The performance is compared with that of commercial sensors.

Dataset
Sensor data from real-car experiment data were used for training and validating the neural network. The neural network input data set contained information of a car's chassis sensors, and the label data set of roll and pitch angles was obtained from the high-accuracy GPS-inertial navigation system (INS) RT-3002 (Oxford Technical Solutions Ltd., Bicester, UK). For training the neural network, a total of 176,259 data sets were used, which logged about 30 min at 10 ms intervals in various situations and offline validation was performed in scenarios as shown in Table 3 with the same vehicle. The software used Python and the framework used TensorFlow 1.6.

. Validation Result and Analysis
The estimation performance was validated using offline sensor data logged in various cases, as shown in Table 3. The root mean square error (RMSE) was calculated by comparison with RT-3002, which was treated as a reference, and with the datasheet of SST810, which is a commercial inclinometer sensor (Vigor Technology Co., Ltd., Xiamen, China). Figures 5-8 show the scenario and roll/pitch estimation results for Case 1-4, respectively.     Table 4 shows the accuracy of the SST810 datasheet and RMSE calculation results of the estimation results for Case 1-4. Figures 5-8 show that the value of pitch and roll angles between 0 and 2 s is fixed at zero. This is due to the structure described in Section 3.1.2, requiring 2 s of sequential data for the input; thus, the values in the initial 2 s cannot be calculated. The RMSE was therefore calculated in the time zone excluding the initial 2 s. Case 1 shows that the roll RMSE is approximately 0.1 • and the pitch RMSE is approximately 0.4 • . There was some offset error, although the vehicle's speed was zero, i.e., a stationary scenario. For commercial sensors, the error rate is 0.05 • at the static scenario, but the estimation results show a larger error compared with the commercial sensors. The neural network estimator in this study uses data of only the chassis sensors of the vehicle; thus, the performance of sensors has a significant impact on the estimation performance. In particular, it can be expected that the IMU's characteristic bias error affected the offset errors in the estimation results.
Cases 2-4 include scenarios in which rapid changes in vehicle attitude occur in areas where acceleration or deceleration occurs. The validation results show high estimation accuracy in these scenarios. In addition, the noise reduction effect compared to RT-3002 is noticeable in the 15-60 s duration for Case 3; thus, the neural network estimator can estimate stable output. The performance of the estimator is thus superior compared to the commercial sensors.

Acceleration and Angular Velocity Estimator (DEKF) 4.2.1. Validation Environment
The acceleration and angle velocity estimator was validated by simulation using CarSim, which is a commercial vehicle simulator software. To create an environment similar to the actual vehicle, Gaussian white noise was added to the sensor value, as shown in Table 5. To ensure that the errors of the acceleration sensor due to roll and pitch have been corrected, simulations were conducted in the scenarios shown in Table 6. Table 5. Sensor configuration for simulation.

Sensor
Noise Unit

Validation Results and Analysis
The correctness performance of a x , a y , and . ψ was validated by comparing IMU sensor values, DEKF estimates, and reference values, while the estimates of roll rates and pitch rates without sensors were validated by comparing with only the reference values. The RMSE of the estimated results is calculated based on the reference and compared with the datasheet of SMI860, which is a commercial six-dimensional IMU from BOSCH Co., Ltd. In addition, to validate the effects of DEKF's parameter estimator, the estimated results of C α f and C αr for each case were noted, and the estimated results with and without cornering stiffness estimation for Case 3 were compared. Figures 9-12 show the simulation scenarios and results of DEKF.     Table 7 shows the accuracy of the commercial sensor, obtained from its datasheet, and RMSE calculation results of the estimation values for case 1-3. In case 1, there was a fatal error of sensor value of a y due to roll angle, and case 2 exhibited considerably varying roll and pitch angles; therefore, the value of a x and a y from sensors may not be accurate. Figures 9b and 10b confirm that these errors in sensor values are successfully corrected to obtain estimates close to the reference. In addition, in normal driving scenarios, as in the 60-65 s duration in Case 3, an a x sensor error by pitch is noted, which has also been successfully corrected. Furthermore, in all cases, filter effects that reduce noise from existing sensors can be checked through a x , a y , and . ψ. Compared with a commercial sensor, a x and a y show similar accuracy, whereas . ψ shows a significantly higher accuracy. However, in the case of commercial sensors, there may be errors caused by roll and pitch angles; therefore, DEKF can have commercial sensor level or higher accuracy even though correcting those errors. The roll and pitch rates could be estimated because they were included in the state vector, although the sensor values were not included. The roll rate accuracy was occasionally lower than that of commercial sensors depending on the case, and in the case of pitch rate, it is not comparable because there is no output of commercial sensors, but it can be confirmed from the RMSE values that the estimated values have a fairly high accuracy. Figures 9d, 10d and 11d show the estimated cornering stiffness. Figure 12 and Table 8 show the results with and without cornering stiffness estimation in Case 3, which can improve accuracy by approximately 3-5%, particularly with a greater effect on a y .

Conclusions and Future Work
In this paper, we proposed a CNN-based neural network to estimate the roll and pitch angles of a vehicle. A DEKF was used to correct the gravitational effect caused by the roll and pitch for estimating the exact acceleration and angular velocity.
By using the vehicle's chassis sensor data as a time series, the neural network could estimate the roll and pitch angles of the vehicle without a GPS or six-dimensional IMU. Based on the estimated roll and pitch angels, we designed an extended Kalman filter (EKF) using the 6-DOF vehicle model. Another EKF was designed, and the two EKFs were used to estimate the cornering stiffness. We then constructed the DEKF.
Using experimental data obtained using a real car, the proposed roll and pitch estimator was validated, and the DEKF was validated in the CarSim simulation environment. The roll and pitch estimator showed an improved performance compared to the commercial sensors in dynamic scenarios and also reduced the noise. However, the performance in static scenarios was weaker. The acceleration and angular velocity estimator could effectively correct the acceleration sensor error due to roll and pitch with a de-noising effect. In addition, the roll and pitch rates that could not be obtained from sensors could be estimated with significant accuracy. By comparing the results before and after including the cornering stiffness, we found that the accuracy is improved if the cornering stiffness is considered.
On the other hand, our work has limitations and challenges that should be further discussed. We plan to consider fusion with other algorithms to improve the attitude estimation performance in static scenarios. In addition, the proposed method has not been checked in case of a change in the vehicle weight. It will also be necessary to verify the performance of the algorithm due to changes in vehicle weight. Furthermore, our proposed algorithm is hard to apply as an embedded system in vehicle because the neural network has large capacity. The future work should be conducted to enable algorithm to operate as real-time in vehicle through simplification and optimization.  All features have an importance weight higher than 0.8; therefore, feature reduction is not performed.