Enhanced Trajectory Forecasting for Hypersonic Glide Vehicle via Physics-Embedded Neural ODE
Abstract
:1. Introduction
- We propose PhysNODE, a novel physics-embedded neural ODE framework that integrates physical knowledge directly into the model architecture for improved the HGV trajectory prediction.
- We demonstrate the effectiveness of PhysNODE through extensive experiments, showcasing its enhanced performance compared to the state-of-the-art data-driven and physics-informed methods, particularly on small datasets.
- We provide a detailed analysis of how physics integration influences the model, focusing on its impact on prediction accuracy, stability, and generalization ability.
2. Related Work
2.1. Data-Driven Approaches
2.2. Hybrid Approaches
2.3. Integrating Physical Laws into Neural Networks
2.4. Neural ODEs for Trajectory Prediction
3. Background
3.1. Motion Equations
3.2. Problem Description
3.3. Neural ODEs
4. The PhysNODE Model
4.1. Model Formulation
4.2. Physics-Embedded Network
4.3. Neural Network Architecture
4.4. Algorithm Workflow
Algorithm 1 PhysNODE |
|
5. Data
5.1. Dataset Generation
5.2. Data Normalization
5.3. Training and Testing Sets
- 100-samples: 100 trajectories randomly selected trajectories for limited data scenarios.
- 600-samples: 600 trajectories randomly selected trajectories for medium data scales.
- 1200-samples: 1200 trajectories randomly selected for sufficient data conditions.
6. Experiments
- Improved Accuracy: We quantify the prediction accuracy of PhysNODE compared to state-of-the-art trajectory prediction models, especially under limited data scenarios.
- Enhanced Stability: We investigate the generalization ability of PhysNODE trained with different data scales, demonstrating its stability in predicting trajectories.
- Effective Physics Integration: We analyze the impact of incorporating physics knowledge on the prediction performance, particularly in capturing the complex aerodynamic characteristics of HGVs.
6.1. Experimental Setup
6.1.1. Evaluation Metrics
- Mean Absolute Error (MAE): Measures the average absolute difference between the predicted and ground truth values for longitude, latitude, and altitude.
- Mean Squared Error (MSE): Measures the average squared Euclidean distance between the predicted and ground truth trajectories, reflecting the overall accuracy of trajectory prediction.
6.1.2. Compared Methods
- Informer [27]: Informer is a Transformer-based long sequence prediction model. It enhances the efficiency and accuracy of Transformer for long sequence prediction by introducing the probsparse self-attention mechanism and self-distillation technique. Informer can capture long-range dependencies in sequences, making it suitable for predicting hypersonic trajectories with complex aerodynamics.
- LSTM [28]: As a widely adopted and classical deep learning model, LSTM has become a prominent benchmark for evaluating the performance of sequence models, particularly in the realm of trajectory prediction. Its ability to understand long-term patterns from past movements and recognize complex temporal trends makes it effective for this task.
- LSTNet [29]: LSTNet is an integrated time series prediction model that combines 1D CNN, LSTM, and periodic regression components. By utilizing multi-scale convolutions to capture both short-term and long-term features, and leveraging LSTM to memorize long-term dependencies, the model’s performance is further enhanced through skip connections and periodic components. LSTNet is capable of effectively handling complex time series data, making it suitable for predicting glider trajectories.
- Transformer [8]: Transformer, known for its success in language processing, has also proven useful for predicting time series, like trajectories. This is because its special attention mechanism helps it find hidden patterns and connections within data, even across long ranges, enhancing its ability to accurately interpret complex trajectories.
- PhyTransformer [10]: PhyTransformer is a variant of Transformer model that incorporates physics equations as regularization terms in the loss function to penalize prediction results that violate physical laws, thus enhancing the accuracy of the HGV trajectory prediction.
- NODE [15]: NODE is a trajectory prediction model based on Neural ODEs. It utilizes a variational encoder to learn the differential equations of trajectories and generates trajectory predictions through ODE solvers. NODE effectively captures the continuity and smoothness of trajectories but do not directly utilize physics knowledge constraints during the modeling process, relying on a data-driven learning approach.
- PhysNODE-Path: This model extends PhysNODE by incorporating additional path constraints derived from intention inference. Assuming the defender has knowledge of the HGV’s intended target (a realistic assumption in certain scenarios), the following information can be inferred:
- -
- : the azimuth angle of the HGV’s flight direction relative to the target at time t
- -
- : the distance between the HGV and the target at time t.
The two quantities and are then concatenated with the original observed state vector (Equation (5)) to form an augmented state vector:This augmented state vector , which includes both the original observed features and the inferred path constraints, is then fed into the Neural Network Prediction module of the PhysNODE (Algorithm 1). By incorporating these additional path constraints, PhysNODE-Path aims to improve the prediction accuracy. This model can be considered a specialized variant of PhysNODE. Therefore, its performance will be analyzed alongside PhysNODE in the subsequent experimental sections. We utilize PhysNODE-Path to showcase the extensibility of the PhysNODE framework, demonstrating its capacity to integrate additional domain-specific information for enhanced prediction.
6.2. Model Convergence Analysis
6.3. Case Study
6.4. Model Performance Comparison Analysis
- Informer: The performance of Informer varies noticeably with different data sizes. While it shows relatively accurate altitude predictions, the average altitude error increase from 0.539 km (1200 samples) to 0.809 km (600 samples) and finally to 1.348 km (100 samples). However, Informer encounters troubles in accurate longitude and latitude predictions, particularly with limited training data. With 1200 samples, the average longitude and latitude errors were 0.00067 and 0.00058 radians, respectively. These errors increased to 0.001697 and 0.001465 radians with 100 samples. There is a significant performance gap. The box plots for Informer get wider with less data, meaning its predictions are more uncertain. This suggests that Informer lacks the ability in learning the complex physical dependencies inherent in hypersonic trajectories when trained with limited data.
- LSTM: LSTM consistently underperforms across all evaluation metrics and data scales, indicating a significant sensitivity to limited training data. This suggests a poor ability to generalize and learn the complex dynamics of hypersonic trajectories. Specifically, the average altitude error is already high with 1200 samples at 0.546 km, but it increases to 0.814 km with 600 samples and then to 42.838 km with only 100 samples. Similarly, both longitude and latitude predictions exhibit poor accuracy across all data scales, with average errors remaining above 0.002 radians even when trained on 1200 samples. This poor performance is mirrored in the Euclidean distance errors, which are consistently the highest among all models, reaching 42.84 km with 100 samples. Meanwhile, its box plots are very wide, meaning its predictions are highly dispersed.
- LSTNet: LSTNet demonstrates some improvement over LSTM, particularly in longitude and latitude predictions when trained on larger datasets. However, its overall performance remains inadequate, especially with limited training data. With 1200 samples, LSTNet achieves average errors of 0.001814 radians for longitude and 0.001064 radians for latitude, outperforming LSTM in these aspects. However, it faces difficult in altitude prediction, yielding an average error of 0.548 km. This inconsistency across different prediction targets is further amplified with smaller datasets. In 600 samples dataset, LSTNet’s performance advantage over LSTM diminished, with average errors increasing to 0.003365 radians (longitude), 0.002187 radians (latitude), and 0.830 km (altitude). This trend continued with 100 samples, where LSTNet’s errors are 0.003981 radians (longitude), 0.002846 radians (latitude), and 1.353 km (altitude). The Euclidean distance errors for LSTNet are consistently high across all data scales, indicating poor overall prediction accuracy. Additionally, its wider error distributions in the box plots suggest a lack of stability in its predictions. Overall, while LSTNet shows some potential with larger datasets, its performance degrads significantly with limited data.
- Transformer: While Transformer achieves a higher accuracy in altitude prediction, especially with larger datasets (0.202 km average error for 1200 samples), its performance in predicting longitude and latitude remains moderate, the box plots is wider and exhibited instability with smaller datasets. For example, with 1200 samples, the average longitude and latitude errors are 0.002384 and 0.001394 radians, respectively. These errors grow to 0.003216 and 0.001761 radians with 600 samples and further increase to 0.002426 and 0.004292 radians with 100 samples. This pattern is also reflected in the Euclidean distance error, which rises from 17.52 km (1200 samples) to 24.25 km (600 samples) and 33.50 km (100 samples). This indicates that while Transformer shows initial success for altitude prediction, its ability to model the full features of hypersonic trajectories is restricted.
- PhyTransformer: The physics-informed PhyTransformer consistently outperformed the standard Transformer model in all metrics and across all data scales. This highlights the benefits of incorporating physics-based constraints to guide the learning process, especially when data is limited. For instance, with only 100 samples, PhyTransformer achieves remarkably low average errors of 0.001391 radians (longitude), 0.001235 radians (latitude), and 0.475 km (altitude) compared to the standard Transformer’s errors of 0.002426 radians, 0.004292 radians, and 0.539 km, respectively. Similarly, the Euclidean distance errors are significantly lower for PhyTransformer across all data scales and the box plot is very narrow, demonstrating its superior ability to accurately capture the dynamics of hypersonic trajectories.
- NODE: Benefiting from its neural ODE framework, NODE outperforms the data-driven models (Informer, LSTM, LSTNet) in most cases. However, its overall performance falls short of the physics-informed PhyTransformer and PhysNODE, especially with small dataset. For instance, while it achieves relatively low average errors in longitude (0.000511 radians), latitude (0.000385 radians), and altitude (0.927 km) with 1200 samples, these errors increase considerably when trained on 600 samples and even more so with 100 samples. This trend is also apparent in the Euclidean distance errors. Meanwhile, its box plot is very wide in altitude attribute. These results highlight the potential of neural ODEs for trajectory prediction, while also emphasizing the importance of exploring further integration of physics-based constraints within this framework to enhance accuracy and stability.
- PhysNODE: PhysNODE consistently displays remarkable performance in predicting hypersonic trajectories across various training data scales, demonstrating its ability to maintain accuracy even with limited data. With 1200 training samples, PhysNODE achieves impressive results, including average errors of 0.000207 radians for longitude, 0.000180 radians for latitude, and 0.574 km for altitude, resulting in the lowest average Euclidean distance error of 1.850 km among all models. Decreasing the training data to 600 samples still yields strong performance, with average errors of 0.000332 radians (longitude), 0.000289 radians (latitude), and 0.919 km (altitude), resulting in an average Euclidean distance error of 2.961 km. Even with further reduction to 100 training samples, PhysNODE continues to excel, achieving average errors of 0.000452 radians (longitude), 0.000396 radians (latitude), and 1.163 km (altitude), with an average Euclidean distance error of 5.553 km. Although it does not achieve optimal results in altitude prediction, and the box plot is wider, the results across various data scales still highlight the robustness of PhysNODE in leveraging data-driven learning with physical knowledge, enabling accurate predictions of hypersonic trajectory dynamics and effective generalization with limited training data.In addition, PhysNODE-Path consistently outperforms the standard PhysNODE. With 100 training samples, PhysNODE-Path achieves an average Euclidean distance error of 5.442 km. This trend of slight improvement with PhysNODE-Path continues with 600 training samples (2.827 km) and 1200 training samples (1.784 km). This small but consistent improvement highlights the flexibility of the PhysNODE framework, which allows for the integration of additional domain-specific information, such as knowledge about the target, to enhance predictive accuracy.
6.5. Forecast Accuracy over Time
- Informer: Informer demonstrates inconsistent performance in predicting hypersonic trajectories over time. With 1200 training samples, the Euclidean distance error initially remains relatively low but exhibits noticeable fluctuations and a gradual upward trend as the prediction horizon extended. Reducing the training data to 600 samples let to a higher initial error and greater instability, with more pronounced fluctuations throughout the prediction horizon. This instability is further exacerbated with only 100 training samples, where the error increases rapidly and fluctuated significantly over time.
- LSTM: LSTM consistently exhibited poor performance across all data scales. With 1200 training samples, the error is already high at the start and rapidly increased with increasing prediction time. This trend is even more pronounced with 600 and 100 training samples, indicating a severe lack of generalization ability, regardless of data availability.
- LSTNet: LSTNet also displays high errors and large fluctuations across all data scales. While increasing the training data from 100 to 1200 samples let to a slight reduction in the overall error, the model still difficult to maintain accuracy over longer prediction horizons.
- Transformer: Transformer shows a very unstable error pattern across all training sets and throughout the entire prediction horizon. This volatility, resembling that observed with LSTM, suggests that neither model possesses the capability for stable and consistent prediction.
- PhyTransformer: PhyTransformer consistently exhibits more stable performance than the standard Transformer across all data scales. With 1200 samples, it maintained a relatively low and stable error throughout the prediction horizon. While a slight increase in error is observed with 600 and 100 samples, the overall error remained significantly lower and more stable than the standard Transformer, indicating the benefit of incorporating physics constraints.
- NODE: With 1200 training samples, NODE exhibited a relatively low initial error and a relatively smooth error trajectory over the prediction horizon. However, as the training data decreased to 600 and 100 samples, both the initial error and the rate of error increase over time became more pronounced. This suggests that while the neural ODE framework can capture some temporal dependencies, its performance is still sensitive to data availability.
- PhysNODE: Both PhysNODE and PhysNODE-Path consistently show the smallest errors and the most reliable predictions across all dataset sizes. Even with limited training data (only 100 samples), both models maintain accurate and consistent predictions throughout the entire forecasting period. This robust stability, along with their high accuracy, demonstrates the advantage of combining physics-based knowledge with deep learning for dependable and accurate hypersonic trajectory prediction. Additionally, PhysNODE-Path, which incorporates additional path constraints, shows slightly better accuracy than standard PhysNODE, especially in the first half of the flight. This result reveals the benefit of including path information for improving the reliability of predictions.
6.6. Benefits of Physics-Informed Modeling
6.7. Relationship between Prediction Accuracy and Variable Complexity
7. Conclusions
- Decomposing the HGV motion equations into observable and predictable components, allowing for targeted prediction of unobservable aerodynamic parameters.
- Utilizing a hybrid neural network architecture (CNN + MLP) to effectively learn and predict these aerodynamic parameters.
- Encapsulating the learned physical knowledge within an ODE block, integrated with an ODE solver for continuous-time trajectory prediction.
- Investigating the incorporation of additional physical constraints, such as thermodynamics, to enhance the model’s accuracy and physical consistency. Exploring alternative methods for embedding physics knowledge within the Neural ODE framework, such as Hamiltonian or Lagrangian mechanics, could also be beneficial.
- Evaluating PhysNODE’s robustness to noise, uncertainties, and incomplete data is crucial for its practical applicability.
- Extending the principles of PhysNODE to predict trajectories for other vehicles, such as the subsonic unpowered gliding vehicle (SUGV) [30]. Adapting the model to handle various flight dynamics and incorporating relevant physical constraints would be key challenges in this extension.
Author Contributions
Funding
Data Availability Statement
DURC Statement
Conflicts of Interest
References
- Li, G.; Zhang, H.; Tang, G. Maneuver characteristics analysis for hypersonic glide vehicles. Aerosp. Sci. Technol. 2015, 43, 321–328. [Google Scholar]
- Law, Y.W.; Gliponeo, J.J.; Singh, D.; McGuire, J.; Liang, J.; Ho, S.Y.; Slay, J. Detecting and tracking hypersonic glide vehicles: A cybersecurity-engineering analysis of academic literature. In Proceedings of the 18th International Conference on Cyber Warfare and Security, Baltimore County, MD, USA, 9–10 March 2023; pp. 189–198. [Google Scholar]
- Le, W.; Liu, H.; Zhao, R.; Chen, J. Attitude control of a hypersonic glide vehicle based on reduced-order modeling and NESO-assisted backstepping variable structure control. Drones 2023, 7, 119. [Google Scholar] [CrossRef]
- Zhang, H.B.; Xie, Y.; Chan, K.J. Investigation on trajectory prediction of maneuverable target. Mod. Def. Technol. 2011, 39, 26–31. [Google Scholar]
- Sun, L.; Yang, B.; Ma, J. Trajectory prediction in pipeline form for intercepting hypersonic gliding vehicles based on LSTM. Chin. J. Aeronaut. 2023, 36, 421–433. [Google Scholar] [CrossRef]
- Halim, C.J.; Kawamoto, K. 2D convolutional neural markov models for spatiotemporal sequence forecasting. Sensors 2020, 20, 4195. [Google Scholar] [CrossRef] [PubMed]
- Kiranyaz, S.; Avci, O.; Abdeljaber, O.; Ince, T.; Gabbouj, M.; Inman, D.J. 1D convolutional neural networks and applications: A survey. Mech. Syst. Signal Process. 2021, 151, 107398. [Google Scholar] [CrossRef]
- Han, K.; Xiao, A.; Wu, E.; Gou, J.; Xu, C.; Wang, Y. Transformer in Transformer. In Proceedings of the 35th Advances in Neural Information Processing Systems, Online, 6–14 December 2021; pp. 15908–15919. [Google Scholar]
- Lim, B.; Zohren, S. Time-series forecasting with deep learning: A survey. Phys. Eng. Sci. 2021, 379, 20200209. [Google Scholar] [CrossRef] [PubMed]
- Ren, J.; Wu, X.; Liu, Y.; Ni, F.; Bo, Y.; Jiang, C. Long-term trajectory prediction of hypersonic glide vehicle based on physics-informed transformer. IEEE Trans. Aerosp. Electron. Syst. 2023, 59, 9551–9561. [Google Scholar] [CrossRef]
- Djeumou, F.; Neary, C.; Goubault, E.; Putot, S.; Topcu, U. Neural networks with physics-informed architectures and constraints for dynamical systems modeling. In Proceedings of the 4th Learning for Dynamics and Control Conference, Palo Alto, CA, USA, 23–24 June 2022; pp. 263–277. [Google Scholar]
- Hao, Z.K.; Liu, S.M.; Zhang, Y.; Ying, C.Y.; Feng, Y.; Su, H.; Zhu, J. Physics-informed machine learning: A survey on problems, methods and applications. arXiv 2022, arXiv:2211.08064. [Google Scholar]
- Meng, C.; Niu, H.; Habault, G.; Legaspi, R.; Wada, S.; Ono, C.; Liu, Y. Physics-informed long-sequence forecasting from multi-resolution spatiotemporal data. In Proceedings of the 31th International Joint Conference on Artificial Intelligence, Vienna, Austria, 23–29 July 2022; pp. 2189–2195. [Google Scholar]
- Chen, Y.; Huang, D.; Zhang, D.; Zeng, J.; Wang, N.; Zhang, H.; Yang, J. Theory-guided hard constraint projection (HCP): A knowledge-based data-driven scientific machine learning method. J. Comput. Phys. 2021, 445, 110624. [Google Scholar] [CrossRef]
- Chen, R.T.Q.; Rubanova, Y.; Bettencourt, J.; Duvenaud, D.K. Neural ordinary differential equations. In Proceedings of the 32th Advances in Neural Information Processing Systems, Montreal, QC, Canada, 2–12 December 2018; p. 31. [Google Scholar]
- Berkani, S.; Guermah, B.; Zakroum, M.; Ghogho, M. Spatio-temporal forecasting: A survey of data-driven models using exogenous data. IEEE Access 2023, 11, 75191–75214. [Google Scholar] [CrossRef]
- Zhang, J.; Xiong, J.; Li, L.; Xi, Q.; Chen, X.; Li, F. Motion state recognition and trajectory prediction of hypersonic glide vehicle based on deep learning. IEEE Access 2022, 10, 21095–21108. [Google Scholar] [CrossRef]
- Xie, Y.; Zhuang, X.; Xi, Z.; Chen, H. Dual-channel and bidirectional neural network for hypersonic glide vehicle trajectory prediction. IEEE Access 2021, 9, 92913–92924. [Google Scholar] [CrossRef]
- Liu, Y.; Xie, Y.; Li, W.; Zhang, Y. Real-time trajectory prediction for hypersonic glide vehicle based on 3-D flight corridor. In Proceedings of the 6th International Conference on Automation, Control and Robotics Engineering, Dalian, China, 15–17 July 2021; pp. 598–603. [Google Scholar]
- Li, M.; Zhou, C.; Shao, L.; Lei, H.; Luo, C. Intelligent trajectory prediction algorithm for reentry glide target based on intention inference. Appl. Sci. 2022, 12, 10796. [Google Scholar] [CrossRef]
- Azencot, O.; Erichson, N.B.; Lin, V.; Mahoney, M.W. Forecasting sequential data using consistent koopman autoencoders. In Proceedings of the 37th International Conference on Machine Learning, Online, 12–18 July 2020; pp. 475–485. [Google Scholar]
- Jin, M.; Zheng, Y.; Li, Y.F.; Chen, S.H.; Yang, B.; Pan, S.R. Multivariate time series forecasting with dynamic graph neural odes. IEEE Trans. Knowl. Data Eng. 2022, 35, 9168–9180. [Google Scholar] [CrossRef]
- Asikis, T.; Böttcher, L.; Antulov-Fantulin, N. Neural ordinary differential equation control of dynamics on graphs. Phys. Rev. Res. 2022, 4, 013221. [Google Scholar] [CrossRef]
- Lu, P.; Forbes, S.; Baldwin, M. Gliding guidance of high L/D hypersonic vehicles. In Proceedings of the AIAA Guidance, Navigation, and Control (GNC) Conference, Boston, MA, USA, 19–22 August 2013; p. 4648. [Google Scholar]
- Falck, R.; Gray, J.S.; Ponnapalli, K.; Wright, T. A Python package for optimal control of multidisciplinary systems. J. Open Source Softw. 2021, 6, 2809. [Google Scholar] [CrossRef]
- Zhang, K.; Xiong, J.; Fu, T. Multi-model estimation of HGV based on coupled aerodynamic parameters. J. Beijing Univ. Aeronaut. Astronaut. 2018, 44, 2156–2164. [Google Scholar]
- Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.X.; Xiong, H.; Zhang, W.C. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the 35th AAAI conference on artificial intelligence, Online, 2–9 February 2021; p. 1110. [Google Scholar]
- Yu, Y.; Si, X.; Hu, C.; Zhang, J.X. A Review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 2019, 31, 1235–1270. [Google Scholar] [CrossRef]
- Lai, G.; Chang, W.C.; Yang, Y. Modeling long-and short-term temporal patterns with deep neural networks. In Proceedings of the 41th international ACM SIGIR conference on research & development in information retrieval, New York, NY, USA, 8–12 July 2018; pp. 95–104. [Google Scholar]
- Mahmood, A.; Rehman, F.; Bhatti, A.I. Trajectory optimization of a subsonic unpowered gliding vehicle using control vector parameterization. Drones 2022, 6, 360. [Google Scholar] [CrossRef]
Parameter | Symbol | Value |
---|---|---|
Altitude | h | 70 to 80 |
Velocity | v | 5000 m s−1 to 6000 m s−1 |
Flight Path Angle | 0° to 0.2° | |
Mass | m | kg |
Surface Area | S | m2 |
Atmospheric Density | The U.S. Standard Atmosphere 1976 | |
Aerodynamic Drag Coef. | ||
Aerodynamic Lift Coef. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lu, S.; Qian, Y. Enhanced Trajectory Forecasting for Hypersonic Glide Vehicle via Physics-Embedded Neural ODE. Drones 2024, 8, 377. https://doi.org/10.3390/drones8080377
Lu S, Qian Y. Enhanced Trajectory Forecasting for Hypersonic Glide Vehicle via Physics-Embedded Neural ODE. Drones. 2024; 8(8):377. https://doi.org/10.3390/drones8080377
Chicago/Turabian StyleLu, Shaoning, and Yue Qian. 2024. "Enhanced Trajectory Forecasting for Hypersonic Glide Vehicle via Physics-Embedded Neural ODE" Drones 8, no. 8: 377. https://doi.org/10.3390/drones8080377
APA StyleLu, S., & Qian, Y. (2024). Enhanced Trajectory Forecasting for Hypersonic Glide Vehicle via Physics-Embedded Neural ODE. Drones, 8(8), 377. https://doi.org/10.3390/drones8080377