Optimizing Traffic Scheduling in Autonomous Vehicle Networks Using Machine Learning Techniques and Time-Sensitive Networking
Abstract
1. Introduction
2. Network Modeling for Traffic Scheduling
2.1. Architecture Modeling
2.2. Application Modeling
3. Reinforcement Learning Based Traffic Scheduling
4. Performance Evaluation
4.1. Evaluation Environment
4.2. Deep Q Network (DQN)
4.3. Advantage Actor-Critic (A2C)
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- J3016; Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. SAE: Pittsburgh, PA, USA, 2018.
- Wang, S.; Zhou, H.; Zhao, H.; Wang, Y.; Cheng, A.; Wu, J. A Zero False Positive Rate of IDS Based on Swin Transformer for Hybrid Automotive In-Vehicle Networks. Electronics 2024, 13, 1317. [Google Scholar] [CrossRef]
- Hank, P.; Suermann, T.; Müller, S. Automotive Ethernet, a holistic approach for a next generation in-vehicle networking standard. Adv. Microsyst. Automot. Appl. 2012, 2012, 79–89. [Google Scholar]
- Park, C.S.; Park, S.K. Performance Evaluation of Zone-Based In-Vehicle Network Architecture for Autonomous Vehicles. Sensors 2023, 23, 669. [Google Scholar] [CrossRef] [PubMed]
- Kim, M.H.; Lee, S.; Ha, K.N.; Lee, K.C. Implementation of a fuzzy-inference-based, low-speed, close-range collision warning system for the urban areas. Proc. Inst. Mech. Eng. D 2013, 227, 234–245. [Google Scholar] [CrossRef]
- Ma, X.; Li, S.; Guan, Z.; Li, J.; Sun, H.; Wang, Y.; Guo, H. Time-Sensitive Networking Mechanism Aided by Multilevel Cyclic Queues in LEO Satellite Networks. Electronics 2023, 12, 1357. [Google Scholar] [CrossRef]
- Guo, M.; Shou, G.; Liu, Y.; Hu, Y. Software-Defined Time-Sensitive Networking for Cross-Domain Deterministic Transmission. Electronics 2024, 13, 1246. [Google Scholar] [CrossRef]
- Kwon, J.H.; Kim, M.H.; Kim, H.J.; Lee, M.W.; Lee, S. Driving Profile Optimization Using a Deep Q-Network to Enhance Electric Vehicle Battery Life. J. Sens. 2023, 1, 6684018. [Google Scholar] [CrossRef]
- Sommer, J.; Gunreben, S.; Feller, F.; Kohn, M.; Mifdaoui, A.; Sass, D.; Scharf, J. Ethernet—A survey on its fields of application. IEEE Commun. Surv. Tutor. 2010, 12, 263–284. [Google Scholar] [CrossRef]
- Bruckner, D.; Stănică, M.P.; Blair, R.; Schriegel, S.; Kehrer, S.; Seewald, M.; Sauter, T. An introduction to OPC UA TSN for industrial communication systems. Proc IEEE 2019, 107, 1121–1131. [Google Scholar] [CrossRef]
- IEEE 802.1Qbv—Bridges and Bridged Networks—Amendment 25: Enhancements for Scheduled Traffic. 2015. Available online: https://standards.ieee.org/ieee/802.1Qbv/6068/ (accessed on 7 July 2024).
- IEEE 802.1Qbu—Bridges and Bridged Networks—Amendment 26: Enhancements for Frame Preemption. 2016. Available online: https://standards.ieee.org/ieee/802.1Qbu/5464/ (accessed on 7 July 2024).
- Wan, T.; Ashwood-Smith, P. A performance study of CPRI over Ethernet with IEEE 802.1Qbu and 802.1Qbv enhancements. In Proceedings of the 2015 IEEE Global Communications Conference (GLOBECOM), San Diego, CA, USA, 6–10 December 2015; Volume 2015, pp. 1–6. [Google Scholar]
- Smirnov, F.; Glaß, M.; Reimann, F.; Teich, J. Optimizing message routing and scheduling in automotive mixed-criticality time-triggered networks. In Proceedings of the 54th ACM/EDAC/IEEE Design Automation Conference, Austin, TX, USA, 18–22 June 2017; pp. 1–6. [Google Scholar] [CrossRef]
- Gavriluţ, V.; Zhao, L.; Raagaard, M.L.; Pop, P. AVB-aware routing and scheduling of time-triggered traffic for TSN. IEEE Access 2018, 6, 75229–75243. [Google Scholar] [CrossRef]
- Steiner, W. An evaluation of SMT-based schedule synthesis for time-triggered multi-hop networks. In Proceedings of the 31st IEEE Real-Time Systems Symposium, San Diego, CA, USA, 30 November–3 December 2010; pp. 375–384. [Google Scholar] [CrossRef]
- Laursen, S.M.; Pop, P.; Steiner, W. Routing optimization of AVB streams in TSN networks. SIGBED Rev. 2016, 13, 43–48. [Google Scholar] [CrossRef]
- Craciunas, S.S.; Oliver, R.S.; Chmelík, M.; Steiner, W. Scheduling real-time communication. in IEEE 802.1Qbv time sensitive networks. In Proceedings of the 24th International Conference on Real-Time Networks and Systems, Brest, France, 19–21 October 2016; pp. 183–192. [Google Scholar] [CrossRef]
- Craciunas, S.S.; Oliver, R.S. SMT-based task- and network-level static schedule generation for time-triggered networked systems. In Proceedings of the 22nd International Conference on Real-Time Networks and Systems, Versaille, France, 8 October 2014; pp. 45–54. [Google Scholar] [CrossRef]
- Tămaş-Selicean, D.; Pop, P.; Steiner, W. Design optimization of TTEthernet-based distributed real-time systems. Real-Time Syst. 2015, 51, 1–35. [Google Scholar] [CrossRef]
- Gavriluţ, V.; Pop, P. Scheduling in time sensitive networks (TSN) for mixed-criticality industrial applications. In Proceedings of the 14th IEEE International Workshop on Factory Communication Systems, Imperia, Italy, 13–15 June 2018; pp. 1–4. [Google Scholar] [CrossRef]
- Dürr, F.; Nayak, N.G. No-wait packet scheduling for IEEE time-sensitive networks (TSN). In Proceedings of the 24th International Conference on Real-Time Networks and Systems, Brest, France, 19–21 October 2016; pp. 203–212. [Google Scholar] [CrossRef]
- Chen, J.; Zuo, Q.; Xu, Y.; Wu, Y.; Jin, W.; Xu, Y. Study of Fixed Point Message Scheduling Algorithm for In-Vehicle Ethernet. Electronics 2024, 13, 2050. [Google Scholar] [CrossRef]
- Zheng, L.; Wei, G.; Zhang, K.; Chu, H. Traffic Classification and Packet Scheduling Strategy with Deadline Constraints for Input-Queued Switches in Time-Sensitive Networking. Electronics 2024, 13, 629. [Google Scholar] [CrossRef]
- Huang, C.; Wang, Y.; Zhang, Y. Time-Sensitive Network Simulation for In-Vehicle Ethernet Using SARSA Algorithm. World Electr. Veh. J. 2024, 15, 21. [Google Scholar] [CrossRef]
- Tindell, K.; Burns, A. Guaranteeing message latencies on controller area network (CAN). In Proceedings of the 1st International Can Conference, Geneva, Switzerland, 25–27 May 1994; pp. 1–11. [Google Scholar]
Source | Destination | Period (ms) | Size (byte) | Transmission Delay (µs) | Route | |
---|---|---|---|---|---|---|
1 | LRR | DCU | 50 | 125 | 10 | [LRR, SW1], [SW1, DCU] |
2 | CAM1 | DCU | 20 | 750 | 60 | [CAM1, SW4], [SW4, SW3], [SW3, DCU] |
3 | CAM2 | DCU | 20 | 750 | 60 | [CAM2, SW4], [SW4, SW3], [SW3, DCU] |
4 | CAM3 | DCU | 20 | 750 | 60 | [CAM3, SW4], [SW4, SW3], [SW3, DCU] |
5 | CAM4 | DCU | 20 | 750 | 60 | [CAM4, SW4], [SW4, SW3], [SW3, DCU] |
6 | MRR1 | DCU | 50 | 125 | 10 | [MRR1, SW2], [SW2, DCU] |
7 | MRR2 | DCU | 50 | 125 | 10 | [MRR2, SW2], [SW2, DCU] |
8 | MRR3 | DCU | 50 | 125 | 10 | [MRR3, SW2], [SW2, DCU] |
9 | MRR4 | DCU | 50 | 125 | 10 | [MRR4, SW2], [SW2, DCU] |
10 | MRR5 | DCU | 50 | 125 | 10 | [MRR5, SW2], [SW2, DCU] |
11 | LIDAR | IVN | 20 | 363 | 29 | [LIDAR, SW1], [SW1, IVN] |
12 | LIDAR | DCU | 20 | 363 | 29 | [LIDAR, SW1], [SW1, DCU] |
13 | LIDAR | ADR | 20 | 363 | 29 | [LIDAR, SW1], [SW1, SW3], [SW3, ADR] |
14 | MAP | DCU | 100 | 625 | 50 | [MAP, SW3], [SW3, DCU] |
15 | IVN | DCU | 10 | 250 | 20 | [IVN, SW1], [SW1, DCU] |
16 | IVN | LRR | 10 | 250 | 20 | [IVN, SW1], [SW1, LRR] |
17 | IVN | MRR1 | 10 | 250 | 20 | [IVN, SW1], [SW1, SW3], [SW3, SW2], [SW2, MRR1] |
18 | IVN | MRR2 | 10 | 250 | 20 | [IVN, SW1], [SW1, SW3], [SW3, SW2], [SW2, MRR2] |
19 | IVN | MRR3 | 10 | 250 | 20 | [IVN, SW1], [SW1, SW3], [SW3, SW2], [SW2, MRR3] |
20 | IVN | MRR4 | 10 | 250 | 20 | [IVN, SW1], [SW1, SW3], [SW3, SW2], [SW2, MRR4] |
21 | IVN | MRR5 | 10 | 250 | 20 | [IVN, SW1], [SW1, SW3], [SW3, SW2], [SW2, MRR5] |
22 | DCU | IVN | 10 | 250 | 20 | [DCU, SW1], [SW1, IVN] |
23 | DCU | IVN | 20 | 250 | 20 | [DCU, SW1], [SW1, IVN] |
24 | V2X | MAP | 20 | 250 | 20 | [V2X, SW4], [SW4, MAP] |
25 | V2X | DCU | 100 | 250 | 20 | [V2X, SW4], [SW4, SW3], [SW3, DCU] |
26 | ADR | DCU | 100 | 500 | 40 | [ADR, SW3], [SW3, DCU] |
27 | HVI | DCU | 100 | 125 | 10 | [HVI, SW3], [SW3, DCU] |
E2E Delay | Jitter | BUGB | ||
---|---|---|---|---|
DQN-based scheduling | 6.1815 × 10−3 (−2.2842) | 1.5293 (−2.8830) | 9.3000 × 10−3 (−1.2000) | −2.2470 |
E2E Delay | Jitter | BUGB | ||
---|---|---|---|---|
A2C-based scheduling | 3.9375 × 10−3 (−2.4054) | 1.5074 (−2.8880) | 9.2000 × 10−3 (−1.4000) | −2.3491 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kwon, J.-H.; Kim, H.-J.; Lee, S. Optimizing Traffic Scheduling in Autonomous Vehicle Networks Using Machine Learning Techniques and Time-Sensitive Networking. Electronics 2024, 13, 2837. https://doi.org/10.3390/electronics13142837
Kwon J-H, Kim H-J, Lee S. Optimizing Traffic Scheduling in Autonomous Vehicle Networks Using Machine Learning Techniques and Time-Sensitive Networking. Electronics. 2024; 13(14):2837. https://doi.org/10.3390/electronics13142837
Chicago/Turabian StyleKwon, Ji-Hoon, Hyeong-Jun Kim, and Suk Lee. 2024. "Optimizing Traffic Scheduling in Autonomous Vehicle Networks Using Machine Learning Techniques and Time-Sensitive Networking" Electronics 13, no. 14: 2837. https://doi.org/10.3390/electronics13142837
APA StyleKwon, J.-H., Kim, H.-J., & Lee, S. (2024). Optimizing Traffic Scheduling in Autonomous Vehicle Networks Using Machine Learning Techniques and Time-Sensitive Networking. Electronics, 13(14), 2837. https://doi.org/10.3390/electronics13142837