Transformer-Based Global Zenith Tropospheric Delay Forecasting Model
Abstract
:1. Introduction
2. Materials and Methods
2.1. Transformer Model
2.2. Study Area and Data
2.3. Data Preprocessing
2.4. Construction of Transformer ZTD Forecast Model
3. Results
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Rózsa, S.; Ambrus, B.; Juni, I.; Ober, P.B.; Mile, M. An advanced residual error model for tropospheric delay estimation. GPS Solut. 2020, 24, 1–15. [Google Scholar] [CrossRef]
- Böhm, J.; Möller, G.; Schindelegger, M.; Pain, G.; Weber, R. Development of an improved empirical model for slant delays in the troposphere (GPT2w). GPS Solut. 2015, 19, 433–441. [Google Scholar] [CrossRef] [Green Version]
- Landskron, D.; Böhm, J. VMF3/GPT3: Refined discrete and empirical troposphere mapping functions. J. Geod. 2018, 92, 349–360. [Google Scholar] [CrossRef]
- Wabbena, G.; Schmitz, M.; Bagge, A. PPP-RTK: Precise point positioning using state-space representation in RTK networks. In Proceedings of the 18th International Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS 2005), Long Beach, CA, USA, 13–16 September 2005; pp. 2584–2594. [Google Scholar]
- Sun, Z.; Zhang, B.; Yao, Y. Improving the estimation of weighted mean temperature in China using machine learning methods. Remote Sens. 2021, 13, 1016. [Google Scholar] [CrossRef]
- Shi, J.; Xu, C.; Guo, J.; Gao, Y. Local troposphere augmentation for real-time precise point positioning. Earth Planets Space 2014, 66, 1–13. [Google Scholar] [CrossRef] [Green Version]
- Douša, J.; Eliaš, M.; Václavovic, P.; Eben, K.; Krč, P. A two-stage tropospheric correction model combining data from GNSS and numerical weather model. GPS Solut. 2018, 22, 1–13. [Google Scholar] [CrossRef]
- Lagler, K.; Schindelegger, M.; Böhm, J.; Krásná, H.; Nilsson, T. GPT2: Empirical slant delay model for radio space geodetic techniques. Geophys. Res. Lett. 2013, 40, 1069–1073. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Jiang, P.; Ye, S.; Chen, D.; Liu, Y.; Xia, P. Retrieving Precipitable Water Vapor Data Using GPS Zenith Delays and Global Reanalysis Data in China. Remote Sens. 2016, 8, 389. [Google Scholar] [CrossRef] [Green Version]
- Osah, S.; Acheampong, A.A.; Fosu, C.; Dadzie, I. Deep learning model for predicting daily IGS zenith tropospheric delays in West Africa using TensorFlow and Keras. Adv. Space Res. 2021, 68, 1243–1262. [Google Scholar] [CrossRef]
- Yao, Y.; Xu, C.; Shi, J.; Cao, N.; Zhang, B.; Yang, J. ITG: A new global GNSS tropospheric correction model. Sci. Rep. 2015, 5, 1–9. [Google Scholar] [CrossRef] [Green Version]
- Hopfield, H.S. Two-quartic tropospheric refractivity profile for correcting satellite data. J. Geophys. Res. 1969, 74, 4487–4499. [Google Scholar] [CrossRef]
- Black, H.D. An easily implemented algorithm for the tropospheric range correction. J. Geophys. Res. Solid Earth 1978, 83, 1825–1828. [Google Scholar] [CrossRef]
- Penna, N.; Dodson, A.; Chen, W. Assessment of EGNOS tropospheric correction model. J. Navig. 2001, 54, 37–55. [Google Scholar] [CrossRef] [Green Version]
- Saastamoinen, J. Contributions to the theory of atmospheric refraction. Bull. Géodésique (1946–1975) 1972, 105, 279–298. [Google Scholar] [CrossRef]
- Collins, J.P.; Langley, R.B. A Tropospheric Delay Model for the User of the Wide Area Augmentation System; Department of Geodesy and Geomatics Engineering, University of New Brunswick: Fredericton, NB, Canada, 1997. [Google Scholar]
- Leandro, R.; Santos, M.; Langley, R. UNB neutral atmosphere models: Development and performance. In Proceedings of the 2006 National Technical Meeting of the Institute of Navigation, Monterey, CA, USA, 18–20 January 2006; pp. 564–573. [Google Scholar]
- Shamshiri, R.; Motagh, M.; Nahavandchi, H.; Haghshenas Haghighi, M.; Hoseini, M. Improving tropospheric corrections on large-scale Sentinel-1 interferograms using a machine learning approach for integration with GNSS-derived zenith total delay (ZTD). Remote Sens. Environ. 2020, 239, 111608. [Google Scholar] [CrossRef]
- Anantrasirichai, N.; Biggs, J.; Albino, F.; Bull, D. A deep learning approach to detecting volcano deformation from satellite imagery using synthetic datasets. Remote Sens. Environ. 2019, 230, 111179. [Google Scholar] [CrossRef] [Green Version]
- Gao, W.; Gao, J.; Yang, L.; Wang, M.; Yao, W. A Novel Modeling Strategy of Weighted Mean Temperature in China Using RNN and LSTM. Remote Sens. 2021, 13, 3004. [Google Scholar] [CrossRef]
- Miao, K.; Han, T.; Yao, Y.; Lu, H.; Chen, P.; Wang, B.; Zhang, J. Application of LSTM for short term fog forecasting based on meteorological elements. Neurocomputing 2020, 408, 285–291. [Google Scholar] [CrossRef]
- Wu, Q.; Guan, F.; Lv, C.; Huang, Y. Ultra-short-term multi-step wind power forecasting based on CNN-LSTM. IET Renew. Power Gener. 2021, 15, 1019–1029. [Google Scholar] [CrossRef]
- Tsai, Y.T.; Zeng, Y.R.; Chang, Y.S. Air pollution forecasting using RNN with LSTM. In Proceedings of the 2018 IEEE 16th International Conference on Dependable, Autonomic and Secure Computing, 16th International Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), Athens, Greece, 12–15 August 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1074–1079. [Google Scholar]
- Ma, Y.; Zhang, Z.; Ihler, A. Multi-lane short-term traffic forecasting with convolutional LSTM network. IEEE Access 2020, 8, 34629–34643. [Google Scholar] [CrossRef]
- Yu, R.; Li, Y.; Shahabi, C.; Demiryurek, U.; Liu, Y. Deep learning: A generic approach for extreme condition traffic forecasting. In Proceedings of the 2017 SIAM International Conference on Data Mining, Houston, TX, USA, 27–29 April 2017; Society for Industrial and Applied Mathematics: Philadelphia, PA, USA, 2017; pp. 777–785. [Google Scholar]
- Dalgkitsis, A.; Louta, M.; Karetsos, G.T. Traffic forecasting in cellular networks using the LSTM RNN. In Proceedings of the 22nd Pan-Hellenic Conference on Informatics, Athens, Greece, 29 November–1 December 2018; pp. 28–33. [Google Scholar]
- Zhu, X.; Fu, B.; Yang, Y.; Ma, Y.; Hao, J.; Chen, S.; Liu, S.; Li, T.; Liu, S.; Guo, W.; et al. Attention-based recurrent neural network for influenza epidemic prediction. BMC Bioinform. 2019, 20, 575. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Q.; Li, F.; Zhang, S.; Li, W. Modeling and Forecasting the GPS Zenith Troposphere Delay in West Antarctica Based on Different Blind Source Separation Methods and Deep Learning. Sensors 2020, 20, 2343. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Sherstinsky, A. Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network. Phys. D Nonlinear Phenom. 2020, 404, 132306. [Google Scholar] [CrossRef] [Green Version]
- Li, S.; Jin, X.; Xuan, Y.; Zhou, X.; Chen, W.; Wang, Y.-X.; Yan, X. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Adv. Neural Inf. Processing Syst. 2019, 32, 5243–5253. [Google Scholar]
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on vision transformer. arXiv 2020, arXiv:2012.12556. [Google Scholar] [CrossRef]
- Kondo, K.; Ishikawa, A.; Kimura, M. Sequence to sequence with attention for influenza prevalence prediction using google trends. In Proceedings of the 2019 3rd International Conference on Computational Biology and Bioinformatics, New York, NY, USA, 17–19 October 2019; pp. 1–7. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30, 6000–6010. [Google Scholar]
- Wu, N.; Green, B.; Ben, X.; O’Banion, S. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv 2020, arXiv:2001.08317. [Google Scholar]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef]
- Frederickson, G.N. An optimal algorithm for selection in a min-heap. Inf. Comput. 1993, 104, 197–214. [Google Scholar] [CrossRef] [Green Version]
- Zhao, Q.; Yao, Y.; Yao, W.; Zhang, S. GNSS-derived PWV and comparison with radiosonde and ECMWF ERA-Interim data over mainland China. J. Atmos. Sol.-Terr. Phys. 2019, 182, 85–92. [Google Scholar] [CrossRef]
- Grubbs, F.E. Procedures for detecting outlying observations in samples. Technometrics 1969, 11, 1–21. [Google Scholar] [CrossRef]
- García-Laencina, P.J.; Sancho-Gómez, J.L.; Figueiras-Vidal, A.R.; Verleysen, M. K nearest neighbours with mutual information for simultaneous classification and missing data imputation. Neurocomputing 2009, 72, 1483–1493. [Google Scholar] [CrossRef]
- Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, Virtual, 2–9 February 2021. [Google Scholar]
- Shaw, P.; Uszkoreit, J.; Vaswani, A. Self-attention with relative position representations. arXiv 2018, arXiv:1803.02155. [Google Scholar]
- Yibin, Y.; Na, C.; Chaoqian, X.; Junjian, J. Accuracy assessment and analysis for GPT2. Acta Geod. Cartogr. Sin. 2015, 44, 726. [Google Scholar]
- Xue, N.; Triguero, I.; Figueredo, G.P.; Landa-Silva, D. Evolving deep CNN-LSTMs for inventory time series prediction. In Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand, 10–13 June 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1517–1524. [Google Scholar]
- Siami-Namini, S.; Tavakoli, N.; Namin, A.S. A comparison of ARIMA and LSTM in forecasting time series. In Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA, 17–20 December 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1394–1401. [Google Scholar]
Bias | STD | MAE | RMSE | R |
---|---|---|---|---|
−0.08 | 1.00 | 0.96 | 1.19 | 0.95 |
Model | Bias | STD | MAE | RMSE | R | Time Spent |
---|---|---|---|---|---|---|
GPT3 | 0.0 | 3.7 | 3.0 | 3.7 | 0.62 | - |
CNN | 0.0 | 3.5 | 2.8 | 3.5 | 0.85 | 5 h 27 min |
RNN | 0.0 | 2.1 | 1.5 | 2.1 | 0.93 | 8 d 7 h 12 min |
LSTM | 0.0 | 1.8 | 1.4 | 1.9 | 0.94 | 10 d 4 h 53 min |
Transformer | 0.0 | 1.7 | 1.3 | 1.8 | 0.95 | 11 h 56 min |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, H.; Yao, Y.; Xu, C.; Xu, W.; Shi, J. Transformer-Based Global Zenith Tropospheric Delay Forecasting Model. Remote Sens. 2022, 14, 3335. https://doi.org/10.3390/rs14143335
Zhang H, Yao Y, Xu C, Xu W, Shi J. Transformer-Based Global Zenith Tropospheric Delay Forecasting Model. Remote Sensing. 2022; 14(14):3335. https://doi.org/10.3390/rs14143335
Chicago/Turabian StyleZhang, Huan, Yibin Yao, Chaoqian Xu, Wei Xu, and Junbo Shi. 2022. "Transformer-Based Global Zenith Tropospheric Delay Forecasting Model" Remote Sensing 14, no. 14: 3335. https://doi.org/10.3390/rs14143335