Abstract
Traditional decision-making methods based on reinforcement learning (RL) for autonomous vehicles rely heavily on environmental interaction samples. These models learn driving policies only implicitly from data and cannot explicitly utilize traffic regulations, driving ethics, or prior human driving experience, which may lead to unsafe control behaviors. A large language model (LLM) has powerful semantic understanding and knowledge reasoning capabilities; however, its high inference latency makes it difficult to directly achieve continuous closed-loop vehicle control. To address these issues, this paper proposes a reinforcement-learning-based autonomous driving decision-making framework with LLM-based knowledge reasoning. In the knowledge reasoning module, a vision-language model (VLM) interprets visual scenes and generates natural-language scene descriptions. Structured prompts that integrate traffic regulations, driving ethics, and prior human driving experience are first constructed and subsequently fed into the LLM, which performs high-level knowledge reasoning and generates actions constrained by human-like rules. The reinforcement learning component combines expert policies with knowledge reasoning policies to iteratively update the policy and value networks. Experimental results demonstrate that the proposed framework incorporates commonsense knowledge, traffic regulations, and risk assessment into the closed-loop training process of reinforcement learning. Compared with conventional deep reinforcement learning (DRL) algorithms, the proposed framework achieves a lower collision rate and significantly fewer traffic violations, while improving decision-making safety and the task completion rate. This study provides a feasible technical pathway for empowering autonomous driving decision-making with large models.