Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer

Xiong, Yang; Wang, Shangwen; Tian, Hongjun; Liu, Guijie; Shan, Zihao; Yin, Yijie; Tao, Jun; Ye, Haonan; Tang, Ying

doi:10.3390/jmse13081593

This is an early access version, the complete PDF, HTML, and XML versions will be available soon.

Open AccessArticle

Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer

by

Yang Xiong

^1,†,

Shangwen Wang

^1,†,

Hongjun Tian

^1,*

,

Guijie Liu

²

,

Zihao Shan

¹,

Yijie Yin

¹,

Jun Tao

¹,

Haonan Ye

¹

and

Ying Tang

³

¹

Engineering College, Shanghai Ocean University, Shanghai 201306, China

²

College of Engineering, Ocean University of China, Qingdao 266100, China

³

College of Marine Science and Ecological Environment, Shanghai Ocean University, Shanghai 201306, China

^*

Author to whom correspondence should be addressed.

^†

These authors contributed equally to this work.

J. Mar. Sci. Eng. 2025, 13(8), 1593; https://doi.org/10.3390/jmse13081593

Submission received: 28 July 2025 / Revised: 14 August 2025 / Accepted: 18 August 2025 / Published: 20 August 2025

(This article belongs to the Special Issue Advanced Control Strategies for Autonomous Maritime Systems)

Download Versions Notes

Abstract

Coordinating Multi-Unmanned Surface Vehicle (USV) swarms in complex, adversarial maritime environments is a significant challenge, as existing multi-agent reinforcement learning (MARL) methods often fail to capture intricate spatiotemporal dependencies, leading to suboptimal policies. To address this, we propose Adv-TransAC, a novel Spatio-Temporal Meta-Reinforcement Learning framework. Its core innovation is a hybrid GAT-transformer architecture that decouples spatial and temporal reasoning: a Graph Attention Network (GAT) models instantaneous tactical formations, while a transformer analyzes their temporal evolution to infer intent. This is combined with an adversarial meta-learning mechanism to enable rapid adaptation to opponent tactics. In high-fidelity escort and defense simulations, Adv-TransAC significantly outperforms state-of-the-art MARL baselines in task success rate and policy robustness. The learned policies demonstrate the emergence of complex cooperative behaviors, such as intelligent risk-aware coordination and proactive interception maneuvers. The framework’s practicality is further validated by a communication-efficient federated optimization architecture. By effectively modeling spatiotemporal dynamics and enabling rapid adaptation, Adv-TransAC provides a powerful solution that moves beyond reactive decision-making, establishing a strong foundation for next-generation, intelligent maritime platforms.

Keywords: multi-agent reinforcement learning; Unmanned Surface Vehicles (USVs); spatiotemporal modeling; meta-learning; Graph Attention Tetworks (GATs); transformer; adversarial coordination

Share and Cite

MDPI and ACS Style

Xiong, Y.; Wang, S.; Tian, H.; Liu, G.; Shan, Z.; Yin, Y.; Tao, J.; Ye, H.; Tang, Y. Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer. J. Mar. Sci. Eng. 2025, 13, 1593. https://doi.org/10.3390/jmse13081593

AMA Style

Xiong Y, Wang S, Tian H, Liu G, Shan Z, Yin Y, Tao J, Ye H, Tang Y. Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer. Journal of Marine Science and Engineering. 2025; 13(8):1593. https://doi.org/10.3390/jmse13081593

Chicago/Turabian Style

Xiong, Yang, Shangwen Wang, Hongjun Tian, Guijie Liu, Zihao Shan, Yijie Yin, Jun Tao, Haonan Ye, and Ying Tang. 2025. "Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer" Journal of Marine Science and Engineering 13, no. 8: 1593. https://doi.org/10.3390/jmse13081593

APA Style

Xiong, Y., Wang, S., Tian, H., Liu, G., Shan, Z., Yin, Y., Tao, J., Ye, H., & Tang, Y. (2025). Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer. Journal of Marine Science and Engineering, 13(8), 1593. https://doi.org/10.3390/jmse13081593

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Spatiotemporal Meta-Reinforcement Learning for Multi-USV Adversarial Games Using a Hybrid GAT-Transformer

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI