DriveTDPA: Trajectory-Decision Preference Alignment for Vision-Language Autonomous Driving Planning

Abstract

Autonomous driving planning requires not only accurate trajectory prediction but also coherent semantic alignment across perception, decision making, and motion generation. Existing vision-language-based approaches predominantly focus on improving trajectory accuracy, which may lead to limited behavioral consistency. In this paper, we reformulate planning as a structured autoregressive generation task, where reasoning, actions, and future trajectories are jointly produced from multimodal observations. Based on this formulation, we propose Trajectory-Decision Joint Preference Optimization (TDJPO), which is a rollout-based alignment framework equipped with a unified reward that simultaneously captures physical trajectory quality and decision-level coherence. Starting from a supervised fine-tuned model, we construct preference pairs through stochastic rollouts and optimize the model using direct preference optimization. Experimental results on the NuScenes-TP benchmark demonstrate that our approach consistently enhances both trajectory accuracy and semantic consistency compared with supervised fine tuning, trajectory-only optimization, and lightweight vision-language baselines. These findings emphasize the necessity of jointly aligning physical feasibility and decision-level reasoning for achieving coherent and human-like autonomous driving behavior.

1. Introduction

Autonomous driving planning aims to generate safe, feasible, and reliable future motions for the ego vehicle in complex and dynamic traffic environments. Modern autonomous driving systems are not limited to trajectory generation alone but involve a set of tightly coupled problems, including trajectory prediction, motion planning, path tracking, traffic management, and cooperative vehicle control. Classical autonomous driving pipelines usually decompose the driving task into perception, prediction, planning, and control modules, where high-level motion plans must eventually be executed by downstream tracking controllers [1]. In this sense, high-quality planning requires not only geometrically accurate future waypoints but also consistency with driving intent, safety constraints, and execution feasibility.

Recent studies further show that autonomous driving becomes more challenging in connected and interactive traffic scenarios. For intelligent and connected vehicles, integrated motion planning and robust output-feedback trajectory tracking have been studied in unsignalized intersections, where cooperative crossing order, collision-free speed planning, and constrained tracking control must be jointly considered [2]. At the traffic-system level, signalized traffic management has been investigated for vehicles with heterogeneous powertrains, emphasizing the joint optimization of energy efficiency, driver preferences, and traffic safety [3]. Vehicle platooning is another important cooperative driving problem, where robust output-feedback control is required to maintain stable inter-vehicle coordination under uncertainties such as vehicle mass variations [4]. These works indicate that autonomous driving planning should be evaluated from a broader perspective than trajectory error alone.

Traditional end-to-end planning approaches typically learn a direct mapping from sensor observations to future trajectories or control-related outputs. Representative planning-oriented methods such as ST-P3 and UniAD demonstrate the effectiveness of unified perception–prediction–planning architectures [5,6]. However, such methods often focus primarily on geometric trajectory accuracy and do not explicitly model language-level reasoning, discrete driving decisions, or the consistency between high-level intent and low-level motion execution.

Recent advances in vision-language models (VLMs) provide a new paradigm for autonomous driving. By introducing language-conditioned reasoning into driving systems, VLM-based methods can enhance scene understanding, long-tail reasoning, and interactive decision making. For example, VLP explores language-guided planning for autonomous driving [7], while DriveLM formulates driving reasoning as graph-structured visual question answering [8]. Recent survey studies also suggest that VLMs are becoming increasingly important for perception, navigation, decision making, and end-to-end autonomous driving [9]. In addition, VLM-based cooperative driving has been explored in V2X settings [10], and large vision-language-action models have been studied for end-to-end driving [11].

Despite these promising developments, existing VLM-based planners still face an important limitation: planning quality is often optimized mainly from the trajectory perspective, while the alignment between generated reasoning, discrete action tokens, and continuous trajectories is insufficiently enforced. As a result, a model may generate a trajectory that is numerically close to the reference, but its predicted actions or reasoning descriptions may be inconsistent with the actual motion. This inconsistency is particularly problematic in autonomous driving, because multiple trajectories may be physically plausible in the same scene while differing significantly in behavioral appropriateness, goal consistency, and safety implications.

From an optimization perspective, autonomous driving planning can be viewed as a structured multi-objective problem. A desirable plan should simultaneously satisfy physical trajectory quality, goal consistency, action–trajectory coherence, safety awareness, and output validity. Optimizing only one aspect may lead to degenerate solutions: a trajectory may be close to the reference but inconsistent with the generated action tokens, while a semantically plausible decision may still correspond to unsafe or physically inaccurate motion. Therefore, trajectory-level and decision-level objectives should be aligned within a unified preference optimization framework rather than treated as independent post-processing criteria.

To address these limitations, we propose Trajectory-Decision Joint Preference Optimization (TDJPO), a rollout-based preference learning framework for vision-language autonomous driving planning. Specifically, we formulate planning as a structured autoregressive generation task, where the model jointly produces reasoning, discrete driving actions, and future trajectories. We then construct preference pairs from stochastic rollouts using a unified reward that integrates trajectory accuracy, motion smoothness, goal alignment, action–trajectory consistency, safety regularization, and unsupported-behavior suppression. The model is optimized with direct preference optimization, allowing it to learn a consistent preference direction over complete structured planning outputs.

Different from conventional trajectory prediction methods, TDJPO does not only predict future coordinates. Different from control-oriented planning and platooning studies, TDJPO does not design a low-level tracking controller. Different from existing VLM-based driving methods, TDJPO explicitly aligns high-level decision semantics with low-level trajectory realization through preference optimization. Therefore, our method is complementary to classical planning and control approaches: it improves high-level structured planning outputs, which can later be tracked by downstream vehicle controllers. In addition to improving planning quality, we further investigate the deployment efficiency of VLM-based planners under selective LLM quantization. Since structured autonomous driving planning requires both valid output generation and reliable task performance, we introduce task-aware efficiency metrics that jointly consider output validity, planning quality retention, and inference throughput.

Our main contributions are summarized as follows.

• We propose Trajectory-Decision Joint Preference Optimization (TDJPO), which is a unified rollout-based preference learning framework that jointly models trajectory generation and decision-level reasoning in vision-language autonomous driving planning.
• We design a unified reward function that integrates physical trajectory quality with semantic decision consistency, including goal alignment, action–trajectory consistency, and safety constraints, enabling more coherent and human-like driving behavior.
• We introduce a comprehensive evaluation protocol that extends beyond traditional trajectory metrics by quantifying system-level efficiency, including improvements in memory usage and inference latency, providing a more holistic assessment of planning performance.

2. Related Work

2.1. Motion Planning, Trajectory Tracking, and Cooperative Control

Autonomous driving has long been studied from the perspectives of motion planning, trajectory tracking, and vehicle control. Classical planning and control studies usually decompose the autonomous driving pipeline into perception, prediction, planning, and control, where the planner generates feasible future motions and the controller tracks the planned path under vehicle dynamics and environmental constraints [1]. This modular perspective remains important because a planned trajectory is useful only if it can be safely and reliably executed by the vehicle.

Recent studies further extend planning and control to connected and cooperative driving scenarios. Viadero-Monasterio et al. [2] study motion planning and robust output-feedback trajectory tracking for multiple intelligent and connected vehicles in unsignalized intersections, jointly considering cooperative traversal order, collision-free speed planning, and constrained tracking control. Signalized traffic management has also been investigated for vehicles with heterogeneous powertrains, where energy efficiency, driver preferences, and traffic safety are jointly optimized [3]. In addition, vehicle platooning has been studied as a representative cooperative driving task, where robust static output–feedback control improves platoon stability under mass uncertainty [4].

These control- and optimization-oriented studies highlight important issues such as tracking feasibility, robustness, cooperation, and system-level safety. However, they usually assume that high-level planning objectives or reference trajectories are already available, and they do not explicitly address language-level reasoning or the alignment between generated decision tokens and continuous trajectories. In contrast, this paper focuses on high-level vision-language planning. TDJPO improves the structured planning output produced by a VLM, while downstream controllers can be used subsequently for closed-loop trajectory tracking.

2.2. Trajectory Prediction and End-to-End Planning

Trajectory prediction and motion forecasting are fundamental components of autonomous driving. Early learning-based approaches model the future motion distribution of traffic participants using structured representations. Multi-path represents multimodal future trajectories with probabilistic anchor trajectories, enabling efficient behavior prediction under uncertainty [12]. TNT further formulates prediction as a target-driven process, where plausible target states are first estimated and then used to guide trajectory generation [13]. More recent Transformer-based approaches improve interaction modeling and multimodal prediction. For example, Motion Transformer models global intention localization and local movement refinement within a unified framework [14], while convolutional Transformer networks have been explored for vehicle trajectory prediction in urban traffic scenarios [15]. Graph- and map-based methods further exploit structured road information. VectorNet encodes HD maps and agent dynamics using vectorized representations [16], while LaneGCN models lane topology and actor–map interactions through lane graph representations [17].

End-to-end autonomous driving planning aims to directly map sensory observations to future trajectories or planning outputs. ST-P3 introduces spatial–temporal feature learning for end-to-end vision-based autonomous driving [5], and UniAD further unifies perception, prediction, and planning in a planning-oriented framework [6]. TransFuser further investigates Transformer-based sensor fusion for end-to-end autonomous driving [18], and VAD introduces a vectorized scene representation to improve planning efficiency and safety [19]. These methods demonstrate the effectiveness of learning-based planning architectures and have significantly advanced autonomous driving research.

However, most trajectory prediction and end-to-end planning methods primarily evaluate geometric trajectory accuracy, such as displacement error or collision-related metrics. They usually do not explicitly generate language-level reasoning or discrete action sequences and therefore cannot directly enforce consistency between high-level driving decisions and low-level trajectory execution. In contrast, our method treats planning as a structured vision-language generation problem and optimizes not only trajectory quality but also decision-trajectory consistency.

2.3. Vision-Language Models for Autonomous Driving

Vision-language models have recently attracted increasing attention in autonomous driving because they can connect visual perception with language-level reasoning and decision making. A recent survey summarizes the role of VLMs in autonomous driving and intelligent transportation systems, covering scene understanding, navigation, planning, decision making, and end-to-end driving [9]. VLP introduces a vision-language-planning framework that leverages language models to improve contextual understanding and autonomous driving planning [7]. DriveLM proposes graph visual question answering for driving, encouraging multi-step reasoning over perception, prediction, and planning questions [8]. Beyond single-vehicle settings, V2X-VLM studies cooperative autonomous driving through large vision-language models in vehicle-to-everything scenarios [10]. OpenDriveVLA further explores large vision-language-action models for end-to-end autonomous driving [11].

These studies show that language can provide a useful semantic structure for autonomous driving. Nevertheless, many existing VLM-based driving methods emphasize either scene-level reasoning, question answering, or trajectory generation, while the explicit alignment between generated actions and continuous motion remains insufficiently studied. This paper addresses this gap by jointly modeling reasoning, action tokens, and trajectories as well as by using a unified preference objective to align these components.

2.4. Preference Optimization for Structured Planning

Preference optimization has become an important technique for aligning large models with desired behaviors. Direct Preference Optimization (DPO) provides a stable and efficient way to optimize a policy from preference pairs without explicitly training a separate reward model or performing online reinforcement learning [20]. This makes it suitable for structured generation tasks where multiple candidate outputs can be ranked according to task-specific criteria.

In autonomous driving planning, preference optimization is attractive because a single scalar metric is often insufficient to describe planning quality. A useful driving plan should be accurate, smooth, goal-consistent, action-consistent, safe, and structurally valid. However, directly applying generic preference optimization to autonomous driving is non-trivial, because the output contains heterogeneous components, including textual reasoning, discrete actions, and continuous trajectories. Existing trajectory-level optimization methods may improve geometric accuracy, but they do not necessarily improve decision-level coherence.

TDJPO differs from generic preference optimization in that the preference relation is constructed over complete structured planning outputs. Each candidate is evaluated by a unified trajectory–decision reward that combines physical trajectory quality and semantic decision consistency. Therefore, the model learns to prefer outputs that are not only geometrically close to the reference trajectory but also behaviorally coherent and consistent with high-level driving intent.

2.5. Summary and Difference from Existing Work

Overall, existing autonomous driving studies cover several complementary directions. Motion planning and control methods emphasize feasibility, tracking robustness, and cooperative driving. Trajectory prediction and end-to-end planning methods focus on future coordinate estimation and planning accuracy. VLM-based driving methods introduce semantic reasoning and language-conditioned planning. Preference optimization methods provide a mechanism for aligning model outputs with desired criteria.

This paper differs from these directions in its objective and output structure. TDJPO does not aim to replace low-level tracking controllers or classical cooperative-control methods. Instead, it improves the high-level structured planning output of a VLM. Compared with trajectory prediction and end-to-end planning methods, TDJPO explicitly models reasoning and discrete driving actions in addition to trajectories. Compared with existing VLM-based driving methods, TDJPO explicitly enforces trajectory-decision consistency through rollout-based preference optimization. This positioning clarifies the scope of our contribution and shows that TDJPO is complementary to, rather than a replacement for, existing autonomous driving planning and control techniques.

3. Method

3.1. Problem Formulation

The goal of autonomous driving planning is to generate future behaviors for the ego vehicle that are safe, dynamically feasible, and semantically aligned with the surrounding environment. This requires not only understanding complex visual scenes but also producing motion that is both physically plausible and logically consistent.

In this paper, we formulate planning as a structured autoregressive generation task within a vision-language modeling framework.

At each time step, the input is defined as shown below:

$$X=\{I_{1:6},S_{\mathrm{ego}},H_{\mathrm{ego}},G,O\},$$

(1)

where $I_{1:6}$ denotes multi-view camera images, $S_{\mathrm{ego}}$ represents the current ego state, $H_{\mathrm{ego}}$ corresponds to historical trajectories, G indicates the high-level navigation goal, and $O={\left\{o_i\right\}}_{i=1}^M$ denotes surrounding objects with attributes such as category, position, and velocity.

Given X, the model produces a token sequence:

$$Y=(y_1,y_2,\dots ,y_T),$$

(2)

which is further structured into

$$Y=\{R,A,T\},$$

(3)

where R denotes the intermediate reasoning process, A represents discrete action tokens, and T corresponds to the predicted future trajectory.

The generation follows an autoregressive formulation:

$$p_{\theta }\left(Y\right|X)={\displaystyle \prod _{t=1}^T}{p}_{\theta }(y_t\mid y_{<t},X).$$

(4)

Here, the reasoning component R serves as an explicit semantic bridge that connects perception and motion generation, enabling alignment between high-level intent and low-level trajectory outputs. This formulation provides a foundation for subsequent reward-driven optimization.

3.2. Overall Framework

Different from conventional approaches that focus solely on trajectory accuracy, we propose a unified rollout-based preference optimization framework tailored for vision-language autonomous driving planning. Our framework explicitly aligns physical trajectory quality with high-level decision consistency under a single optimization objective.

Given multimodal input X, the model is first trained using supervised fine tuning (SFT) to produce structured outputs. Subsequently, the model is further improved via rollout-based preference learning guided by a unified reward that jointly evaluates trajectory quality and decision-level coherence.

As summarized in Algorithm 1, TDJPO consists of four main steps: stochastic rollout generation, unified reward evaluation, preference pair construction, and DPO-based policy optimization.

Structured Planning Generation. Planning is modeled as a structured multimodal generation process. For each input X, the model generates

$$Y=\{R,A,T\},$$

(5)

where R encodes semantic reasoning, A represents discrete driving actions, and T denotes continuous trajectories. This unified representation enables the joint modeling of perception, reasoning, and motion generation, and it provides an explicit mechanism to enforce consistency across different levels.

Algorithm 1 Overall Trajectory-Decision Joint Preference Optimization Pipeline

Require: Training set $\mathcal{D}=\left\{(x_i,y_i)\right\}$, SFT model ${\pi }_{\theta }$, rollout number K Ensure: Optimized policy ${\pi }_{{\theta }^{★}}$   1: Initialize policy ${\pi }_{\theta }$ by supervised fine tuning on structured targets $y=\{R,A,T\}$   2: for each training sample $x\in \mathcal{D}$ do   3:       Sample K candidate outputs $\{y_1,\dots ,y_K\}\sim {\pi }_{\theta }(·|x)$   4:       for each candidate $y_k$ do   5:             Parse $y_k$ into reasoning $R_k$, action sequence $A_k$, and trajectory $T_k$   6:             Compute unified reward $r\left(y_k\right)$ using Algorithm 2   7:       end for   8:       Rank all candidates according to reward values   9:       Construct preference pairs ${\mathcal{P}}_x=\{(y^{+},y^{-})\mid r\left(y^{+}\right)>r\left(y^{-}\right)\}$ 10:       Filter invalid or low-quality pairs if necessary 11: end for 12: Collect all preference pairs $\mathcal{P}={\bigcup }_x{\mathcal{P}}_x$ 13: Optimize ${\pi }_{\theta }$ with DPO using $\mathcal{P}$ and the SFT model as reference 14: return  ${\pi }_{{\theta }^{★}}$

Algorithm 2 Unified Reward Computation

Require: Candidate output $y=\{R,A,T\}$, input x, ground-truth trajectory $T^{*}$ Ensure: Unified reward $r\left(y\right)$   1: Parse predicted trajectory $T={\left\{p_t\right\}}_{t=1}^H$ and action sequence A   2: Compute format validity reward $r_{\mathrm{format}}$ Physical-level reward   3: Compute $\mathrm{ADE}$, $\mathrm{FDE}$, and $\mathrm{Late}$ between T and $T^{*}$   4: Normalize them into ${\mathrm{ADE}}^{\prime }$, ${\mathrm{FDE}}^{\prime }$, and ${\mathrm{Late}}^{\prime }$   5: Compute trajectory accuracy reward: $$r_{\mathrm{traj}}=0.4e^{-{\mathrm{ADE}}^{\prime }/2.0}+0.4e^{-{\mathrm{FDE}}^{\prime }/2.0}+0.2e^{-{\mathrm{Late}}^{\prime }/2.0}$$   6: Compute velocity variation $\Delta v$ and oscillation penalty $p_{\mathrm{osc}}$   7: Compute smoothness reward: $$r_{\mathrm{smooth}}=0.8e^{-\Delta v/0.5}+0.2(1-p_{\mathrm{osc}})$$   8: Compute physical reward: $$r_{\mathrm{phy}}={\lambda }_{\mathrm{traj}}{r}_{\mathrm{traj}}+{\lambda }_{\mathrm{smooth}}{r}_{\mathrm{smooth}}$$ Decision-level reward   9: Compute goal reward $r_{\mathrm{goal}}$ from the final waypoint and mission goal 10: Infer action labels $\widehat{A}$ from segmented trajectory T 11: Compute action–trajectory consistency reward $r_{\mathrm{act-traj}}$ 12: Compute safety reward $r_{\mathrm{safe}}$ from minimum predicted distance 13: Compute unsupported-motion penalty $p_{\mathrm{unsup}}$ 14: Compute decision reward: $$r_{\mathrm{dec}}={\lambda }_{\mathrm{goal}}{r}_{\mathrm{goal}}+{\lambda }_{\mathrm{cons}}{r}_{\mathrm{act-traj}}+{\lambda }_{\mathrm{safe}}{r}_{\mathrm{safe}}-{\lambda }_{\mathrm{unsup}}{p}_{\mathrm{unsup}}$$ Overall reward 15: Compute final reward: $$r\left(y\right)={\lambda }_{\mathrm{fmt}}{r}_{\mathrm{format}}+{\lambda }_{\mathrm{phy}}{r}_{\mathrm{phy}}+{\lambda }_{\mathrm{dec}}{r}_{\mathrm{dec}}$$ 16: return  $r\left(y\right)$

Unified Preference Optimization. As illustrated in Figure 1, starting from the SFT-initialized model, we adopt a rollout-based pipeline that includes (1) stochastic candidate generation, (2) reward evaluation, (3) preference pair construction, and (4) preference-based optimization.

For each input, multiple candidate outputs are sampled and evaluated using a unified reward function. Preference pairs are constructed based on reward comparisons, and the model is optimized using preference learning objectives such as DPO.

Trajectory-Decision Joint Alignment. The key idea of our approach is to design a joint reward that simultaneously enforces (i) the physical feasibility of trajectories (e.g., accuracy and smoothness), and (ii) semantic consistency across reasoning, actions, and trajectories.

Instead of optimizing these aspects independently, we integrate them into a single objective, allowing the model to learn coherent planning behavior in an end-to-end manner. In this process, the reasoning component R plays a critical role in bridging high-level decision making and low-level motion generation.

3.3. Multi-Objective Interpretation of TDJPO

Although TDJPO is implemented through reward-based preference construction, its underlying objective can be interpreted as a structured multi-objective optimization problem. For each input x, the model generates a structured planning output $y=\{R,A,T\}$, where R denotes reasoning, A denotes discrete driving actions, and T denotes the future trajectory. A desirable output should satisfy multiple criteria simultaneously, including output validity, physical trajectory quality, decision consistency, and safety-related constraints.

We formulate the ideal planning objective as maximizing the expected utility of the policy while keeping it close to the supervised reference policy:

$$\underset{{\pi }_{\theta }}{\max }{\mathbb{E}}_{x\sim \mathcal{D},y\sim {\pi }_{\theta }(·|x)}\left[r(y,x)\right]-\beta D_{\mathrm{KL}}\left({\pi }_{\theta }(·|x)\parallel {\pi }_{\mathrm{ref}}(·|x)\right),$$

(6)

where ${\pi }_{\mathrm{ref}}$ is the SFT-initialized reference policy, $\beta$ controls the strength of KL regularization, and $r(y,x)$ is a scalarized multi-objective reward.

Specifically, the reward is decomposed as

$$r(y,x)={\lambda }_{\mathrm{fmt}}{r}_{\mathrm{fmt}}+{\lambda }_{\mathrm{phy}}{r}_{\mathrm{phy}}+{\lambda }_{\mathrm{dec}}{r}_{\mathrm{dec}},$$

(7)

where $r_{\mathrm{fmt}}$ enforces structural validity, $r_{\mathrm{phy}}$ measures physical trajectory quality, and $r_{\mathrm{dec}}$ measures decision-level behavioral consistency. This scalarization does not imply that the objectives are independent. Instead, it provides a practical way to align multiple planning criteria within a single preference ordering.

The decision-level term can be further interpreted as a soft constraint regularizer:

$$r_{\mathrm{dec}}={\lambda }_{\mathrm{goal}}{r}_{\mathrm{goal}}+{\lambda }_{\mathrm{act}}{r}_{\mathrm{act-traj}}+{\lambda }_{\mathrm{safe}}{r}_{\mathrm{safe}}-{\lambda }_{\mathrm{unsup}}{p}_{\mathrm{unsup}}.$$

(8)

$r_{\mathrm{goal}}$ encourages consistency with the high-level navigation goal, $r_{\mathrm{act-traj}}$ aligns discrete action tokens with continuous motion, $r_{\mathrm{safe}}$ regularizes risky trajectories, and $p_{\mathrm{unsup}}$ suppresses unsupported or weakly grounded behaviors.

This formulation can also be viewed as a Lagrangian relaxation of a constrained planning problem:

$$\underset{{\pi }_{\theta }}{\max }\mathbb{E}\left[r_{\mathrm{traj}}+r_{\mathrm{smooth}}+r_{\mathrm{goal}}+r_{\mathrm{act-traj}}\right],$$

(9)

subject to

$$\mathbb{E}\left[p_{\mathrm{unsafe}}\right]\le {ϵ}_s,\mathbb{E}\left[p_{\mathrm{unsup}}\right]\le {ϵ}_u.$$

(10)

In this view, the safety and unsupported-behavior terms act as soft penalties for violating planning constraints. TDJPO therefore differs from purely trajectory-level preference optimization: it does not rank candidates only by geometric closeness but by a joint utility that reflects whether the generated trajectory is physically accurate, behaviorally coherent, and compatible with safety-related constraints.

The benefit of joint optimization over sequential two-stage optimization is that the preference relation is constructed over complete structured outputs. If trajectory quality and decision consistency are optimized separately, the second stage may overwrite or weaken the preference learned in the first stage. In contrast, TDJPO evaluates each candidate output under the same unified objective, allowing the model to learn a single consistent preference direction over reasoning, action, and trajectory tokens.

3.4. Supervised Fine Tuning

We first initialize the model via supervised fine tuning (SFT) to enable structured planning generation.

Given an input X, the model is trained to predict a target sequence $Y=\{R,A,T\}$ that includes reasoning, action, and trajectory components. The objective follows a standard autoregressive language modeling loss:

$${\mathcal{L}}_{\mathrm{SFT}}=-{\mathbb{E}}_{(X,Y)}{\displaystyle \sum _{t=1}^T}\log p_{\theta }(y_t\mid y_{<t},X).$$

(11)

This stage equips the model with the capability to generate syntactically valid and structurally coherent outputs, including reasonable reasoning, feasible action sequences, and basic trajectory patterns.

However, due to the limitations of supervised signals, the model may still produce suboptimal trajectories or exhibit inconsistencies between high-level decisions and low-level motion. Therefore, we further introduce reward-driven optimization to refine the model.

3.5. Trajectory-Decision Joint Preference Optimization (TDJPO)

Building upon the SFT-initialized model, we perform preference-based optimization to improve structured planning outputs. Instead of treating trajectory quality and semantic consistency separately, we optimize them jointly using a unified reward.

Rollout and preference construction. Given an input x, we sample K candidate outputs via stochastic decoding:

$$\{y_1,y_2,\dots ,y_K\}\sim {\pi }_{\theta }\left(y\right|x).$$

(12)

Each candidate is parsed into structured components, including actions and trajectories. A unified reward function $r\left(y\right)$ is used to evaluate each candidate, and preference pairs are constructed as

$$(y^{+},y^{-})\mathrm{s}.\mathrm{t}.r\left(y^{+}\right)>r\left(y^{-}\right).$$

(13)

The detailed reward computation procedure is summarized in Algorithm 2.

These pairs are then used to optimize the model through Direct Preference Optimization (DPO), which increases the likelihood of preferred outputs.

Given a preference pair $(y^{+},y^{-})$ for input x, the DPO loss is written as

$${\mathcal{L}}_{\mathrm{DPO}}=-{\mathbb{E}}_{(x,y^{+},y^{-})\sim \mathcal{P}}\left[\log \sigma \left(\beta \log {\displaystyle \frac{{\pi }_{\theta }\left(y^{+}\right|x)}{{\pi }_{\mathrm{ref}}\left(y^{+}\right|x)}}-\beta \log {\displaystyle \frac{{\pi }_{\theta }\left(y^{-}\right|x)}{{\pi }_{\mathrm{ref}}\left(y^{-}\right|x)}}\right)\right],$$

(14)

where ${\pi }_{\mathrm{ref}}$ denotes the SFT reference model. By using the unified trajectory-decision reward only for preference construction, TDJPO avoids explicit online reinforcement learning while still optimizing the policy toward outputs with higher joint planning utility.

Physical-level reward. We define a physical-level reward to assess trajectory quality in terms of accuracy and smoothness:

$$r_{\mathrm{phy}}={\lambda }_{\mathrm{traj}}{r}_{\mathrm{traj}}+{\lambda }_{\mathrm{smooth}}{r}_{\mathrm{smooth}},$$

(15)

where ${\lambda }_{\mathrm{traj}}=0.60$ and ${\lambda }_{\mathrm{smooth}}=0.25$.

Given predicted trajectory $T={\left\{p_t\right\}}_{t=1}^H$ and ground truth $T^{*}={\left\{p_t^{*}\right\}}_{t=1}^H$, we compute

$$\mathrm{ADE}={\displaystyle \frac{1}{H}}{\displaystyle \sum _{t=1}^H}\parallel p_t-p_t^{*}{\parallel }_2,\mathrm{FDE}={\parallel p_H-p_H^{*}\parallel }_2,$$

(16)

$$\mathrm{Late}={\displaystyle \frac{1}{3}}{\displaystyle \sum _{t=H-2}^H}{\parallel p_t-p_t^{*}\parallel }_2.$$

(17)

After normalization, the trajectory reward is defined as

$$r_{\mathrm{traj}}=0.4e^{-{\mathrm{ADE}}^{\prime }/2.0}+0.4e^{-{\mathrm{FDE}}^{\prime }/2.0}+0.2e^{-{\mathrm{Late}}^{\prime }/2.0}.$$

(18)

For smoothness, we compute velocity variation:

$$\Delta v={\displaystyle \frac{1}{H-2}}{\displaystyle \sum _{t=1}^{H-2}}|\parallel \Delta p_{t+1}{\parallel }_2-{\parallel \Delta p_t\parallel }_2|,$$

(19)

and define

$$r_{\mathrm{smooth}}=0.8e^{-\Delta v/0.5}+0.2(1-p_{\mathrm{osc}}).$$

(20)

Decision-level reward. We define a decision-level reward to measure semantic alignment between high-level decisions and trajectory realization:

$$r_{\mathrm{dec}}={\lambda }_{\mathrm{goal}}{r}_{\mathrm{goal}}+{\lambda }_{\mathrm{cons}}{r}_{\mathrm{act-traj}}+{\lambda }_{\mathrm{safe}}{r}_{\mathrm{safe}}-{\lambda }_{\mathrm{unsup}}{p}_{\mathrm{unsup}},$$

(21)

where ${\lambda }_{\mathrm{goal}}=0.15$, ${\lambda }_{\mathrm{cons}}=0.25$, ${\lambda }_{\mathrm{safe}}=0.10$, and ${\lambda }_{\mathrm{unsup}}=0.10$. The goal reward is defined based on the final waypoint $(x_6,y_6)$:

$$r_{\mathrm{goal}}=\left\{\begin{array}{cc}\sigma \left({\displaystyle \frac{y_6-2.0}{1.0}}\right)e^{-|x_6|/1.0},\hfill & \mathrm{FORWARD},\hfill \\ \sigma \left({\displaystyle \frac{-x_6-0.8}{1.0}}\right)\sigma \left({\displaystyle \frac{y_6-2.0}{1.0}}\right),\hfill & \mathrm{LEFT},\hfill \\ \sigma \left({\displaystyle \frac{x_6-0.8}{1.0}}\right)\sigma \left({\displaystyle \frac{y_6-2.0}{1.0}}\right),\hfill & \mathrm{RIGHT}.\hfill \end{array}\right.$$

(22)

The action–trajectory consistency is computed by segmenting the trajectory into three parts and comparing inferred actions with predicted actions:

$$r_{\mathrm{act-traj}}={\displaystyle \frac{1}{3}}{\displaystyle \sum _{t=1}^3}\left(0.5\mathbb{I}(a_t^{\mathrm{lat}}={\widehat{a}}_t^{\mathrm{lat}})+0.5\mathbb{I}(a_t^{\mathrm{lon}}={\widehat{a}}_t^{\mathrm{lon}})\right).$$

(23)

The safety reward is defined as

$$r_{\mathrm{safe}}=\min \left(1,{\displaystyle \frac{d_{\min }}{2.5}}\right),$$

(24)

and the unsupported-motion penalty is

$$p_{\mathrm{unsup}}={\tilde{p}}_{\mathrm{unsup}}·\max (0.5,u).$$

(25)

Overall reward. The final reward integrates structural validity, physical quality, and semantic consistency:

$$r={\lambda }_{\mathrm{fmt}}{r}_{\mathrm{format}}+{\lambda }_{\mathrm{phy}}{r}_{\mathrm{phy}}+{\lambda }_{\mathrm{dec}}{r}_{\mathrm{dec}}.$$

(26)

4. Experiment

4.1. Implementation Details

Our framework is built upon InternVL3.5-1B, an open-source vision-language model developed by Shanghai AI Laboratory, Shanghai, China, which serves as the backbone for multimodal perception and generation. The model is first initialized through supervised fine tuning (SFT) on structured driving data and subsequently optimized using the proposed preference learning strategy.

For training, we adopt stochastic decoding to generate multiple candidate outputs for each input sample during rollout. Unless otherwise specified, the number of sampled candidates is set to $K=6$. The preference optimization stage is implemented using Direct Preference Optimization (DPO), where preference pairs are constructed based on the unified reward.

We use the AdamW optimizer with a cosine learning rate schedule. The initial learning rate is set to $2\times {10}^{-5}$ for SFT and $1\times {10}^{-5}$ for preference optimization. Training is conducted on multiple GPUs with mixed precision to improve efficiency.

During inference, we use greedy decoding for deterministic evaluation. All experiments are conducted under the same configuration to ensure fair comparison.

Planning-Level Evaluation and Vehicle-Control Interface

Our experiments are conducted under an offline planning benchmark rather than a closed-loop vehicle-control environment. The model generates structured planning outputs, including reasoning, discrete driving actions, and a future waypoint trajectory. Following the NuScenes-TP evaluation protocol, we evaluate the predicted trajectory using planning-level metrics such as ADE, FDE, action–trajectory consistency, and safety-related indicators. Low-level vehicle control commands, including steering, throttle, and braking, are not directly generated or executed in our experiments.

In a deployed autonomous driving system, the predicted trajectory can be used as the reference path for a downstream tracking controller. Given the predicted waypoints $T={\{p_t=(x_t,y_t)\}}_{t=1}^H$, reference headings and velocities can be estimated as

$${\psi }_t=\arctan 2(y_t-y_{t-1},x_t-x_{t-1}),v_t={\displaystyle \frac{\parallel p_t-p_{t-1}{\parallel }_2}{\Delta t}}.$$

(27)

These reference states can then be tracked by standard controllers such as model predictive control, pure pursuit, PID, or LQR-based tracking. Therefore, the reported results should be interpreted as planning-level performance, while closed-loop control performance is left for future work.

4.2. Dataset and Evaluation Metrics

We evaluate our method on the NuScenes-TP benchmark, which provides a comprehensive testbed for trajectory prediction and planning under real-world driving scenarios [21]. The dataset includes diverse urban environments with complex interactions among vehicles, pedestrians, and static obstacles.

Following standard protocols, we adopt commonly used trajectory evaluation metrics, including the Average Displacement Error (ADE), Final Displacement Error (FDE), and Late-stage Displacement Error (Late). These metrics measure the geometric accuracy of predicted trajectories at different temporal stages.

In addition to trajectory-level metrics, we also evaluate semantic consistency by measuring the alignment between predicted actions and generated trajectories. This includes action consistency scores and safety-related indicators, such as minimum distance to surrounding objects.

All results are averaged over the validation set, and lower values indicate better performance for displacement errors.

4.3. Baselines

We compare our method against several representative approaches, including supervised learning baselines, trajectory-level preference optimization, RL-based planning baselines, and lightweight vision-language planners.

Supervised Fine Tuning (SFT). This baseline directly trains the model using ground-truth trajectories without any preference optimization. It serves as a fundamental reference for evaluating the effectiveness of subsequent alignment strategies.

SFT + Physical Reward. To isolate the contribution of trajectory-level optimization, we include a baseline that applies preference optimization using only physical trajectory rewards (e.g., trajectory accuracy and smoothness) without decision-level alignment. This baseline reflects trajectory-only preference learning.

PPO and GRPO. To further compare with RL-based planning and alignment methods, we include PPO and GRPO as additional baselines. For a fair comparison, both PPO and GRPO are initialized from the same SFT model and optimized using the same unified trajectory-decision reward function as TDJPO. Therefore, this comparison mainly isolates the effect of different optimization formulations rather than different reward designs.

Lightweight VLM Planners. We further compare with lightweight vision-language models that perform structured planning without preference-based alignment, including TinyLLaVA-Phi-2-SigLIP-3.1B, developed under the TinyLLaVA framework by researchers from Beihang University and Tsinghua University, Beijing, China, and DeepSeek-VL2-Tiny-1B, released by DeepSeek AI, Hangzhou, China. These models represent efficient VLM-based planners with limited capacity.

These baselines allow us to disentangle the effects of supervised learning, trajectory-only optimization, RL-based reward optimization, lightweight VLM planning, and joint trajectory-decision preference alignment.

4.4. Main Results

Table 1. Comparison with lightweight vision-language models on the NuScenes-TP dataset.

Model	Params	ADE ↓	FDE ↓	Goal ↑	Cons ↑	Unsup ↓	Lat ↓
TinyLLaVA-Phi-2-SigLIP-3.1B	3.1B	1.056	2.341	0.701	0.613	0.032	0.348
DeepSeek-VL2-Tiny-1B	1.0B	1.288	3.072	0.466	0.376	0.287	1.313
Ours	1.0B	0.945	2.130	0.774	0.765	0.026	0.485

Note: B denotes billion parameters. ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

summarizes the performance of different methods on the NuScenes-TP benchmark.

Our proposed TDJPO method consistently outperforms all baselines across trajectory accuracy metrics. Compared with SFT, TDJPO achieves significant improvements in ADE and FDE, indicating better overall trajectory quality. Compared with physical reward, which focuses only on trajectory refinement, TDJPO further reduces errors, demonstrating the benefit of incorporating decision-level signals.

In addition to geometric accuracy, TDJPO also shows clear advantages in semantic consistency. The generated trajectories exhibit better alignment with predicted actions and driving goals, resulting in more coherent and human-like behavior.

These results confirm that jointly optimizing trajectory quality and decision consistency leads to superior planning performance compared with optimizing either aspect alone.

4.5. Comparison with RL-Based Baselines

To address the comparison with RL-based planning methods, we further evaluate PPO and GRPO under the same reward function used by TDJPO. All three methods are initialized from the same SFT model and optimized with the same unified trajectory-decision reward. Thus, the comparison focuses on whether different optimization formulations can effectively leverage the same reward signal.

As shown in

Table 2. Comparison with RL-based baselines under the same trajectory-decision reward. TDJPO achieves the best overall performance.

Method	ADE ↓	FDE ↓	Goal ↑	Cons ↑	Unsup ↓	Lat ↓
PPO	1.000	2.218	0.731	0.731	0.038	0.575
GRPO	0.992	2.206	0.743	0.744	0.034	0.573
Ours (SFT + TDJPO)	0.945	2.130	0.774	0.765	0.026	0.485

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, TDJPO consistently outperforms both PPO and GRPO under the same reward function. Compared with GRPO, TDJPO reduces ADE from 0.992 to 0.945 and FDE from 2.206 to 2.130 while improving the goal score from 0.743 to 0.774 and the action–trajectory consistency from 0.744 to 0.765. Compared with PPO, TDJPO also achieves lower trajectory errors, stronger semantic alignment, and a lower unsupported penalty.

These results demonstrate that the improvement of TDJPO does not come from using an additional or stronger reward function. Instead, TDJPO more effectively converts the same trajectory-decision reward into model alignment through pairwise preference optimization. This optimization formulation is particularly suitable for structured autonomous driving planning, where the model must jointly maintain physical trajectory quality, goal consistency, and action–trajectory coherence.

4.6. Ablation Study

To better understand the contribution of each component in our framework, we conduct ablation studies by removing or modifying key elements of the proposed method.

Effect of decision-level reward. We first remove the decision-level reward and optimize the model using only physical trajectory metrics, as shown in

Table 3. Ablation study on reward design. Physical-level reward improves trajectory quality, while TDJPO further enhances both accuracy and semantic consistency.

Method	ADE ↓	FDE ↓	Goal ↑	Cons ↑	Unsup ↓	Lat ↓
SFT	1.068	2.397	0.645	0.647	0.086	0.789
SFT + Physical Reward	1.037	2.310	0.669	0.662	0.069	0.711
Ours (SFT + TDJPO)	0.945	2.130	0.774	0.765	0.026	0.485

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

. The performance drops noticeably in both trajectory accuracy and semantic consistency, indicating that decision-level alignment plays a critical role.

Overall, the ablation results validate the importance of each component in our framework.

4.7. Additional Evaluation on Waymo

To further evaluate the generalization capability of the proposed framework beyond NuScenes-TP, we conduct an additional experiment on the Waymo dataset using the same training and evaluation pipeline [22]. We evaluate the SFT baseline, the physical-reward variant, and the proposed TDJPO method on a 10,000-sample Waymo validation subset with the same trajectory-quality and semantic-alignment metrics.

As shown in

Table 4. Additional evaluation on the Waymo dataset. TDJPO consistently improves trajectory quality, semantic alignment, and behavioral regularity over the SFT baseline.

Method	ADE ↓	FDE ↓	Goal ↑	Cons ↑	Unsup ↓	Lat ↓
SFT	0.770	1.921	0.676	0.608	0.0777	0.654
SFT + Physical Reward	0.679	1.721	0.662	0.606	0.0716	0.499
Ours (SFT + TDJPO)	0.635	1.599	0.784	0.798	0.0105	0.407

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, TDJPO consistently improves over the SFT baseline on Waymo, which was released by Waymo LLC, Mountain View, CA, USA. In terms of trajectory quality, TDJPO reduces ADE from 0.770 to 0.635 and FDE from 1.921 to 1.599, showing better trajectory prediction accuracy under a different data distribution. Compared with SFT + Physical Reward, TDJPO further reduces ADE from 0.679 to 0.635 and FDE from 1.721 to 1.599, indicating that decision-level alignment provides additional benefits beyond trajectory-level optimization.

More importantly, TDJPO achieves substantial improvements in semantic alignment. The goal soft score increases from 0.676 to 0.784 compared with SFT, and the action–trajectory consistency improves from 0.608 to 0.798. In contrast, SFT + Physical Reward mainly improves geometric trajectory quality but does not improve semantic alignment, as its Goal and Cons scores are slightly lower than those of SFT. This further supports our motivation that optimizing only physical trajectory rewards is insufficient for structured driving planning, while joint trajectory-decision preference optimization can better align high-level decisions with generated motion.

In addition, TDJPO significantly reduces the unsupported behavior penalty from 0.0777 to 0.0105 and decreases the lateral magnitude from 0.654 to 0.407. These results suggest that the proposed method not only improves trajectory accuracy but also produces more coherent and behaviorally regular planning outputs. Overall, the Waymo results provide additional evidence that TDJPO is not limited to the NuScenes-TP benchmark and can generalize to another large-scale autonomous driving dataset with different data characteristics and scenario distributions.

4.8. LLM-Based Semantic Critic Evaluation

Although our decision-level reward encourages goal consistency, action–trajectory coherence, and safety-aware planning, it is still based on manually designed and rule-grounded metrics. Therefore, to further examine whether the proposed method improves language-level planning coherence beyond these rule-based indicators, we introduce an additional LLM-based semantic critic evaluation.

Specifically, we use Qwen-Flash as an external language-model judge to evaluate the generated planning responses on the full validation set of 6019 samples. For each sample, the judge is provided with the textual scene context and the model-generated planning response without access to the manually designed reward score used during training. The judge assigns scores from 1 to 5 along seven semantic dimensions: scene reasoning, motion prediction, goal consistency, action–trajectory consistency, safety awareness, reasoning faithfulness, and overall semantic quality. This evaluation is designed to assess whether the generated reasoning, discrete actions, and future trajectory form a coherent language-level planning response.

As shown in

Table 5. LLM-based semantic critic evaluation on 6019 validation samples using Qwen-Flash. All dimensions are scored on a 1–5 scale. TDJPO achieves the best scores across all semantic dimensions, indicating improved language-level planning coherence beyond rule-based evaluation metrics.

Method	Scene	Motion	Goal	Act.-Traj.	Safety	Faith.	Overall
SFT	2.9222	2.4723	4.1012	2.7500	2.0478	2.2190	2.4014
SFT + Physical Reward	2.9138	2.4431	4.0759	2.7041	2.0402	2.1906	2.3683
Ours (SFT + TDJPO)	2.9708	2.5144	4.1386	2.8089	2.1002	2.2726	2.4233

Note: Bold values indicate the best performance in each column.

, TDJPO achieves the best scores across all evaluated semantic dimensions. Compared with SFT, TDJPO improves the overall semantic score from 2.4014 to 2.4233. It also improves scene reasoning from 2.9222 to 2.9708, motion prediction from 2.4723 to 2.5144, action–trajectory consistency from 2.7500 to 2.8089, safety awareness from 2.0478 to 2.1002, and reasoning faithfulness from 2.2190 to 2.2726. These results indicate that the proposed trajectory-decision joint preference optimization not only improves trajectory-level planning metrics but also leads to more coherent language-level planning responses.

It is also worth noting that the physical-reward-only baseline slightly decreases the LLM-based semantic scores compared with SFT. For example, its overall semantic score drops from 2.4014 to 2.3683, and its action–trajectory consistency decreases from 2.7500 to 2.7041. This suggests that optimizing only physical trajectory quality does not necessarily improve semantic reasoning or decision-level coherence. In contrast, TDJPO jointly optimizes physical trajectory quality and decision-level consistency, which helps align the generated reasoning, discrete actions, and future trajectory more effectively.

Nevertheless, this evaluation should be regarded as complementary evidence rather than a replacement for human preference evaluation. The current training reward remains rule-grounded and manually specified, while the LLM-based critic is used only for post hoc evaluation. Incorporating LLM-based critics or human textual feedback into the preference construction process is an important direction for future work.

4.9. Robustness Under Adverse Weather and Illumination Conditions

To further evaluate the robustness of the proposed method beyond average-case driving conditions, we conduct an additional analysis under different visual conditions, including daytime, rainy, and nighttime scenarios. Rainy and nighttime scenes are representative challenging conditions in autonomous driving, where degraded visibility and increased perception uncertainty may make trajectory planning more difficult.

As shown in

Table 6. Performance under different visual conditions. Rainy and nighttime scenes are used to evaluate robustness under adverse weather and illumination.

Method	Scenario	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
SFT	Day	1.037	2.328	0.614	0.648	0.088	0.799
	Rain	1.044	2.275	0.736	0.627	0.078	0.754
	Night	1.341	3.101	0.736	0.674	0.083	0.775
SFT + Physical Reward	Day	1.001	2.231	0.639	0.665	0.070	0.712
	Rain	1.032	2.240	0.763	0.638	0.064	0.704
	Night	1.310	3.009	0.740	0.675	0.071	0.711
Ours (SFT + TDJPO)	Day	0.904	2.037	0.744	0.778	0.026	0.477
	Rain	0.961	2.112	0.859	0.706	0.028	0.516
	Night	1.225	2.840	0.854	0.766	0.029	0.495

Note: B denotes billion parameters. ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, TDJPO consistently outperforms the SFT baseline and the physical-reward-only variant across all three scenarios. Under rainy conditions, TDJPO reduces ADE from 1.044 to 0.961 and FDE from 2.275 to 2.112 compared with SFT while also improving goal consistency from 0.736 to 0.859. Under nighttime conditions, TDJPO achieves more pronounced improvements, reducing ADE from 1.341 to 1.225 and FDE from 3.101 to 2.840. Moreover, TDJPO substantially reduces unsupported behaviors and lateral deviation in both rainy and nighttime scenes, suggesting that the proposed trajectory-decision joint preference optimization improves not only trajectory accuracy but also behavioral stability under adverse visual conditions.

4.10. Case Study

To further analyze the behavioral differences among methods, we present five representative scenarios under diverse environmental and traffic conditions, including daytime, rainy, nighttime, a complex intersection, and a construction-site road, and we compare the qualitative performance of different approaches. These examples provide intuitive evidence that the gains of our method are not limited to lower trajectory errors but also arise from improved consistency between predicted actions and actual motion.

Scenario 1: Daytime Residential Road. As shown in Figure 2, the ego vehicle drives along a residential road with pedestrians present near the roadside, requiring stable lane following and conservative motion planning. The SFT baseline predicts repeated ACCELERATE actions and produces a trajectory with obvious lateral drift, indicating weak alignment between scene understanding and motion generation. Adding physical-level reward reduces the geometric deviation to some extent, but the predicted trajectory is still left-biased and does not fully match the desired driving behavior. In contrast, our method predicts a more appropriate MAINTAIN strategy and generates a trajectory much closer to the ground truth. This case suggests that decision-aware optimization improves not only trajectory accuracy (lower ADE/FDE) but also action–trajectory consistency and lane-level stability.

Scenario 2: Rainy Urban Road. Figure 3 shows a rainy driving scenario with reduced visibility and a visually ambiguous road surface. Such conditions make it more difficult for the model to maintain accurate longitudinal control and lateral alignment. The SFT model produces an overly aggressive plan with repeated ACCELERATE actions, leading to noticeable mismatch with the reference trajectory. The physical-level reward improves smoothness and partially suppresses the excessive forward motion, yet a clear lateral offset remains. Our method yields a more balanced behavior: it maintains forward progress while staying closer to the lane center and the ground-truth trajectory. This example illustrates that optimizing only geometric quality is insufficient in adverse conditions, whereas joint optimization helps preserve both trajectory fidelity and semantically appropriate behavior.

Scenario 3: Nighttime Driving. As shown in Figure 4, nighttime scenes are particularly challenging due to low illumination and limited visual cues. Although all three methods recognize the need to decelerate, clear differences remain in how accurately they translate this decision into motion. The SFT baseline exhibits noticeable lateral deviation, and the physical-level reward does not fully correct this bias. By comparison, our method generates a trajectory that is much closer to the ground truth in both lateral position and longitudinal evolution, producing a more stable and human-aligned response. This indicates that decision-level supervision is especially helpful when perception is uncertain, as it encourages the model to maintain coherent planning behavior rather than merely producing a plausible geometric path.

Scenario 4: Complex Intersection. As shown in Figure 5, this scenario involves a geometrically complex road region with curved lane structure, dense markings, and surrounding traffic participants, which requires the planner to preserve route intent while maintaining accurate lateral control. The SFT baseline predicts repeated MAINTAIN actions, but its trajectory exhibits a noticeable lateral bias and fails to follow the reference path closely through the local road geometry. Introducing physical-level reward slightly reduces this deviation, yet the predicted motion remains misaligned with the ground-truth trajectory and does not fully capture the required path shape. In contrast, our method produces a trajectory that better matches both the lane-level structure and the longitudinal progress of the reference motion, resulting in a more coherent mapping from high-level decision to low-level execution. This case highlights that in structurally complex scenes, joint decision-trajectory optimization is important for turning route understanding into stable and precise motion planning.

Scenario 5: Construction-Site Road. Figure 6 presents a road segment near a construction area, where roadside clutter, narrowed drivable space, and surrounding objects increase the difficulty of safe and stable planning. In this case, the SFT model predicts repeated ACCELERATE actions and generates a trajectory with obvious lateral drift, showing that it tends to produce a more aggressive and less well-centered behavior than desired. The physical-level reward alleviates part of the geometric error and makes the motion smoother, but a visible offset from the reference path still remains. Our method, by comparison, stays much closer to the ground-truth trajectory and maintains better lane-center alignment while also producing a more reasonable action sequence that balances progress with caution. This example further demonstrates that decision-aware preference optimization is particularly beneficial in cluttered or spatially constrained scenes, where geometric refinement alone is insufficient to guarantee semantically appropriate driving behavior.

Discussion. Across all five scenarios, a consistent pattern emerges. The SFT baseline can produce roughly plausible trajectories, but it often suffers from mismatches between predicted actions and actual motion, especially in challenging or structurally complex scenes. Physical-level reward improves geometric quality, yet its gains are mainly limited to trajectory refinement and do not fully resolve behavioral inconsistency. In contrast, our method achieves better alignment between high-level decisions and low-level trajectory generation, leading to more accurate, stable, and semantically coherent planning outputs. These qualitative observations are consistent with the quantitative improvements in trajectory error, goal alignment, and action–trajectory consistency reported in the main results.

4.11. Component Ablation on Decision-Level Reward

To further understand the role of each decision-level component, we conduct a leave-one-out ablation on the decision-level reward. As shown in

Table 7. Component ablation on the decision-level reward. Each row removes one decision-level term from the full TDJPO objective, while the last row shows the complete model.

Goal	Act-Traj	Safety	Unsup	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
✘	✔	✔	✔	0.957	2.145	0.739	0.744	0.038	0.501
✔	✘	✔	✔	0.965	2.160	0.732	0.697	0.043	0.515
✔	✔	✘	✔	0.969	2.186	0.777	0.749	0.027	0.520
✔	✔	✔	✘	0.959	2.169	0.763	0.756	0.038	0.529
✔	✔	✔	✔	0.945	2.130	0.774	0.765	0.026	0.485

Note: ✔ indicates that the corresponding decision-level reward term is included, while ✘ indicates that the term is removed. ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, each ablated variant disables one component from the full TDJPO objective, while the last row reports the complete model with all four components enabled.

Several observations can be drawn from the results. First, the full TDJPO model achieves the best overall performance, yielding the lowest ADE/FDE, the highest action–trajectory consistency, and the lowest unsupported penalty and lateral deviation. This shows that the four decision-level components are complementary and jointly contribute to the final planning quality.

Second, removing the action–trajectory consistency term causes the most severe degradation in semantic coherence, reducing the Consistency score from 0.765 to 0.697, while also affecting the Goal, ADE, and FDE values. This indicates that explicitly aligning discrete decisions with continuous motion is crucial for structured driving planning.

Third, removing the goal alignment term significantly weakens the model’s ability to follow high-level driving intent, leading to a clear drop in Goal score from 0.774 to 0.739. Removing the unsupported-behavior penalty increases the unsupported penalty metric from 0.026 to 0.038 and also enlarges lateral deviation, suggesting that this term is important for suppressing unreasonable or weakly supported behaviors.

Finally, removing the safety reward slightly increases the Goal score to 0.777 but degrades nearly all other metrics, especially FDE, Consistency, and lateral magnitude. This suggests that the safety term does not simply improve directional intention; it also plays a broader role in maintaining motion feasibility and behavioral stability.

Overall, these results demonstrate that the gains of TDJPO do not come from any single handcrafted reward term. Instead, goal alignment, action–trajectory consistency, safety regularization and unsupported-behavior suppression work together to improve both trajectory quality and semantic coherence.

4.12. Parameter Sensitivity Analysis

To further evaluate the robustness of the proposed reward design, we conduct parameter sensitivity analysis at two levels. We first study the coefficient of the overall decision-level reward in the unified objective, and then analyze the internal coefficients corresponding to goal alignment, action–trajectory consistency, safety regularization, and unsupported-behavior suppression. For each study, we vary one coefficient while keeping the others fixed, and we evaluate the resulting model under the same setting.

Overall, the model remains reasonably stable within a practical coefficient range, indicating that the proposed objective is not overly sensitive to precise hyperparameter tuning. At the same time, different coefficients exhibit different metric preferences, which is consistent with the intended functional roles of the corresponding reward terms.

4.12.1. Sensitivity to the Overall Decision-Level Weight

We first analyze the sensitivity of the overall decision-level weight ${\lambda }_{\mathrm{dec}}$ in the unified reward. This coefficient controls the relative importance of the decision-level reward with respect to the physical-level trajectory reward. We vary ${\lambda }_{\mathrm{dec}}$ from 0.25 to 1.50 while keeping the internal decision-level coefficients fixed and report the results in

Table 8. Sensitivity analysis of the overall decision-level weight ${\lambda }_{\mathrm{dec}}$ in the unified reward. We vary ${\lambda }_{\mathrm{dec}}$ while keeping the internal coefficients fixed.

${\mathit{\lambda }}_{\mathbf{dec}}$	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
0.25	1.032	2.310	0.713	0.714	0.055	0.692
0.50	1.026	2.297	0.712	0.715	0.054	0.680
1.00	0.945	2.130	0.774	0.765	0.026	0.485
1.50	0.931	2.098	0.768	0.762	0.030	0.458

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

.

The results show that the overall decision-level weight has a substantial impact on planning quality. When ${\lambda }_{\mathrm{dec}}$ is too small, the model performance deteriorates significantly across all metrics. For example, both ${\lambda }_{\mathrm{dec}}=0.25$ and ${\lambda }_{\mathrm{dec}}=0.50$ yield much worse ADE/FDE scores, weaker goal alignment, lower action–trajectory consistency, and larger unsupported penalty and lateral deviation than the default setting. This indicates that decision-level supervision must have sufficient weight in the unified objective; otherwise, the model cannot effectively learn semantically coherent planning behavior.

As ${\lambda }_{\mathrm{dec}}$ increases, the model performance improves substantially. In particular, ${\lambda }_{\mathrm{dec}}=1.50$ achieves the best ADE, FDE, and lateral magnitude, suggesting that a stronger decision-level signal can further improve trajectory refinement and motion stability. However, compared with ${\lambda }_{\mathrm{dec}}=1.00$, it shows slightly worse Goal, Consistency, and unsupported penalty, and it also introduces more invalid trajectory outputs during evaluation. This suggests that overly emphasizing the decision-level reward may slightly hurt the overall semantic balance and output stability.

Therefore, we adopt ${\lambda }_{\mathrm{dec}}=1.00$ as the default setting. Although ${\lambda }_{\mathrm{dec}}=1.50$ achieves the best trajectory-related metrics, ${\lambda }_{\mathrm{dec}}=1.00$ provides a better balance between trajectory accuracy, semantic coherence, behavior regularity, and output robustness. These results confirm that assigning sufficient but not excessive weight to the decision-level reward is important for achieving strong overall planning performance.

4.12.2. Sensitivity to Internal Decision-Level Coefficients

Sensitivity to the Goal-Alignment Coefficient

As shown in

Table 9. Sensitivity analysis of the goal-alignment coefficient ${\lambda }_{\mathrm{goal}}$. We vary ${\lambda }_{\mathrm{goal}}$ from 0.05 to 0.25 while keeping the other coefficients fixed.

${\mathit{\lambda }}_{\mathbf{goal}}$	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
0.05	0.982	2.206	0.771	0.743	0.028	0.542
0.10	0.998	2.256	0.762	0.757	0.037	0.619
0.15	0.945	2.130	0.774	0.765	0.026	0.485
0.20	0.955	2.151	0.776	0.758	0.022	0.488
0.25	0.937	2.113	0.778	0.745	0.021	0.460

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

,we first analyze the sensitivity of the goal-alignment coefficient ${\lambda }_{\mathrm{goal}}$ by varying it from 0.05 to 0.25 while keeping all other coefficients fixed. The results show that the model performance changes in a structured rather than erratic manner, suggesting that the goal-related reward plays a stable and interpretable role in the overall objective.

As ${\lambda }_{\mathrm{goal}}$ increases, the Goal score generally improves, indicating that a larger weight indeed strengthens the model’s ability to follow high-level driving intent. Meanwhile, several trajectory-related metrics also benefit from a sufficiently large goal coefficient. In particular, ${\lambda }_{\mathrm{goal}}=0.25$ achieves the best ADE, FDE, Goal, unsupported penalty, and lateral magnitude among the tested settings. However, its Consistency score drops to 0.745 compared with the best consistency score of 0.765 obtained at ${\lambda }_{\mathrm{goal}}=0.15$. This suggests that overly emphasizing goal alignment may slightly weaken the coherence between discrete decisions and continuous motion.

Therefore, although a larger ${\lambda }_{\mathrm{goal}}$ can further improve several trajectory- and goal-related metrics, we adopt ${\lambda }_{\mathrm{goal}}=0.15$ as the default setting because it provides the best action–trajectory consistency and a more balanced trade-off between trajectory accuracy and semantic coherence.

Sensitivity to the Action–Trajectory Consistency Coefficient

As shown in

Table 10. Sensitivity analysis of the action–trajectory consistency coefficient ${\lambda }_{\mathrm{act-traj}}$. We vary ${\lambda }_{\mathrm{act-traj}}$ from 0.15 to 0.35 while keeping the other coefficients fixed.

${\mathit{\lambda }}_{\mathbf{act-traj}}$	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
0.15	0.965	2.159	0.765	0.717	0.025	0.487
0.20	0.990	2.226	0.772	0.734	0.030	0.557
0.25	0.945	2.130	0.774	0.765	0.026	0.485
0.30	0.965	2.175	0.763	0.716	0.036	0.535
0.35	0.949	2.138	0.763	0.737	0.029	0.486

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, we next analyze the sensitivity of the action–trajectory consistency coefficient ${\lambda }_{\mathrm{act-traj}}$ by varying it from 0.15 to 0.35 while keeping all other coefficients fixed. The results show that this coefficient has a clear influence on both semantic coherence and trajectory quality.

Among the tested settings, ${\lambda }_{\mathrm{act-traj}}=0.25$ achieves the best overall performance, yielding the lowest ADE and FDE, the highest Goal score and Consistency score, and the lowest lateral magnitude. Although ${\lambda }_{\mathrm{act-traj}}=0.15$ obtains a slightly better unsupported penalty, its semantic consistency is notably worse. When the coefficient is increased beyond 0.25, the overall performance starts to decline again, especially in Goal, Consistency, and unsupported behavior.

These results suggest that the action–trajectory consistency term should be assigned a moderate weight. If the weight is too small, the model cannot fully benefit from decision–motion alignment; if it is too large, the optimization may over-constrain the structured output and hurt overall planning quality. Therefore, we adopt ${\lambda }_{\mathrm{act-traj}}=0.25$ as the default setting.

Sensitivity to the Safety Coefficient

As shown in

Table 11. Sensitivity analysis of the safety coefficient ${\lambda }_{\mathrm{safe}}$. We vary ${\lambda }_{\mathrm{safe}}$ from 0 to 0.20 while keeping the other coefficients fixed.

${\mathit{\lambda }}_{\mathbf{safe}}$	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
0.00	0.969	2.186	0.777	0.749	0.027	0.520
0.05	0.971	2.185	0.764	0.706	0.032	0.536
0.10	0.945	2.130	0.774	0.765	0.026	0.485
0.15	0.986	2.228	0.767	0.762	0.037	0.584
0.20	0.910	2.049	0.773	0.752	0.019	0.377

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, we further analyze the sensitivity of the safety coefficient ${\lambda }_{\mathrm{safe}}$ by varying it from 0 to 0.20 while keeping all other coefficients fixed. The results show that this coefficient has a strong impact on trajectory quality and behavioral regularity, but its effect is not monotonic in the low-to-medium range.

Specifically, small or moderate safety weights such as 0.05 and 0.15 do not consistently improve performance, and they even lead to clear degradation on several metrics. In contrast, when the coefficient is increased to 0.20, the model achieves the best ADE, FDE, unsupported penalty, and lateral magnitude among all tested settings while maintaining a Goal score comparable to the default setting. This suggests that sufficiently strong safety regularization can substantially improve motion quality and suppress unstable or risky behaviors.

At the same time, we observe that ${\lambda }_{\mathrm{safe}}=0.20$ yields a slightly lower Consistency score than the default setting ${\lambda }_{\mathrm{safe}}=0.10$. This indicates a mild trade-off: stronger safety supervision improves trajectory refinement and behavior regularity, but it may slightly weaken the alignment between discrete actions and continuous motion. Overall, the results suggest that the safety term becomes truly effective only when assigned sufficient weight, and ${\lambda }_{\mathrm{safe}}=0.20$ is a strong configuration for improving trajectory-level and auxiliary metrics.

Sensitivity to the Unsupported-Behavior Penalty Coefficient

As shown in

Table 12. Sensitivityanalysis of the unsupported-behavior penalty coefficient ${\lambda }_{\mathrm{unsup}}$. We vary ${\lambda }_{\mathrm{unsup}}$ from 0 to 0.20 while keeping the other coefficients fixed.

${\mathit{\lambda }}_{\mathbf{unsup}}$	ADE ↓	FDE ↓	Goal ↑	Consistency ↑	Unsup ↓	Lat ↓
0.00	0.959	2.169	0.763	0.756	0.038	0.529
0.05	0.960	2.169	0.770	0.769	0.030	0.521
0.10	0.945	2.130	0.774	0.765	0.026	0.485
0.15	0.934	2.112	0.768	0.743	0.025	0.458
0.20	0.947	2.122	0.761	0.757	0.025	0.467

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

, we finally analyze the sensitivity of the unsupported-behavior penalty coefficient ${\lambda }_{\mathrm{unsup}}$ by varying it from 0 to 0.20 while keeping all other coefficients fixed. The results show that introducing this term consistently improves behavioral regularity, but its effect also depends on the assigned weight.

Compared with removing this term entirely, using a positive coefficient substantially reduces the unsupported penalty and also improves trajectory quality and lateral stability. Among the tested settings, ${\lambda }_{\mathrm{unsup}}=0.15$ achieves the best ADE, FDE, and lateral magnitude while maintaining a low unsupported penalty. This indicates that a moderate unsupported-behavior penalty can effectively suppress unreasonable outputs and improve motion quality at the same time.

We also observe a trade-off across different settings. A relatively small value, such as ${\lambda }_{\mathrm{unsup}}=0.05$, yields the highest action–trajectory consistency, whereas a larger value, such as ${\lambda }_{\mathrm{unsup}}=0.20$, further reduces the unsupported penalty but slightly weakens Goal and overall trajectory accuracy. Therefore, we adopt ${\lambda }_{\mathrm{unsup}}=0.15$ as the default setting, since it provides the best overall balance between trajectory quality, behavioral regularity, and semantic coherence.

5. Quantization

5.1. Selective LLM Quantization Setup

Post-training quantization has been widely studied as an effective approach for reducing the memory and inference cost of large language models. Representative methods include SmoothQuant, which enables accurate W8A8 quantization by smoothing activation outliers [23], and AWQ, which protects salient weights for low-bit LLM compression and acceleration [24]. To improve deployment efficiency, we further study post-training quantization based on llm-compressor. Importantly, our setting is selective rather than full-model quantization. Only the LLM layers are quantized, while the lm_head, vision_tower, and multimodalprojector are kept in higher precision. This design is motivated by two practical considerations. First, the visual backbone in our current deployment stack is not compatible with the same quantization pipeline, making full end-to-end quantization infeasible. Second, in structured autonomous driving planning, aggressively quantizing sensitive multimodal perception modules may introduce instability in action generation and trajectory decoding. Therefore, we focus on a realistic deployment setting in which only the LLM component is quantized. Under this setup, we evaluate four representative quantization schemes, including W8A8FP8, W8A16G32, W4A16G128, and NVFP4, against the original full-precision model.

5.2. Task-Aware Evaluation for Selective Quantization

Conventional quantization studies usually report model size, raw throughput, or end-to-end latency [23,24]. However, such metrics are insufficient in our setting for two reasons. First, only the LLM layers are quantized, whereas the visual encoder and multimodal projector remain unchanged. As a result, end-to-end latency alone cannot faithfully reflect the actual acceleration on the quantized module. Second, our model performs structured autonomous driving planning, where deployment quality depends not only on speed but also on output validity and task-level planning performance. To address this issue, we evaluate each quantized model from two complementary perspectives: (1) task-level deployment utility, which is measured by Valid Quality Throughput (VQT), and (2) module-level efficiency, which is measured by local LLM efficiency metrics.

We define VQT as

$$\mathrm{VQT}={\displaystyle \frac{N·V·Q}{T_{\mathrm{total}}}},$$

(28)

where N is the number of evaluated samples, $T_{\mathrm{total}}$ is the manually recorded end-to-end wall-clock runtime over the entire evaluation set, V is the valid output rate, and Q is the quality retention ratio relative to the full-precision baseline. The valid output rate is defined as

$$V={\displaystyle \frac{N_{\mathrm{valid}}}{N}},$$

(29)

where $N_{\mathrm{valid}}$ denotes the number of samples whose structured outputs are valid; i.e., both the predicted action sequence and the predicted trajectory can be successfully parsed and evaluated.

To measure how well a quantized model preserves planning quality, we use a direction-aware quality retention ratio Q over six planning metrics. Let ${\mathcal{M}}^{+}=\{\mathrm{Goal},\mathrm{Cons}\}$ denote the metrics where higher is better, and let ${\mathcal{M}}^{-}=\{\mathrm{ADE},\mathrm{FDE},\mathrm{Unsup},\mathrm{Lat}\}$ denote the metrics where lower is better. For each metric m, its retention term is defined as

$$q_m=\left\{\begin{array}{cc}\min \left(1,{\displaystyle \frac{m_{\mathrm{q}}}{m_{\mathrm{fp}}+ϵ}}\right),\hfill & m\in {\mathcal{M}}^{+},\hfill \\ \min \left(1,{\displaystyle \frac{m_{\mathrm{fp}}}{m_{\mathrm{q}}+ϵ}}\right),\hfill & m\in {\mathcal{M}}^{-},\hfill \end{array}\right.$$

(30)

where $m_{\mathrm{fp}}$ and $m_{\mathrm{q}}$ denote the metric values of the full-precision and quantized models, respectively, and $ϵ$ is a small constant for numerical stability. The overall quality retention ratio is then computed as

$$Q={\displaystyle \frac{1}{\left|\mathcal{M}\right|}}\sum _{m\in \mathcal{M}}{q}_m,\mathcal{M}={\mathcal{M}}^{+}\cup {\mathcal{M}}^{-}.$$

(31)

In addition to VQT, we also report local LLM efficiency metrics to explicitly measure the actual acceleration achieved on the quantized module. Specifically, we use the LLM local speedup

$$S_{\mathrm{llm}}={\displaystyle \frac{T_{\mathrm{llm}}^{\mathrm{fp}}}{T_{\mathrm{llm}}^{\mathrm{q}}}},$$

(32)

the token-normalized latency

$$L_{\mathrm{tok}}={\displaystyle \frac{T_{\mathrm{llm}}}{N_{\mathrm{gen}}}},$$

(33)

the LLM token throughput

$${\mathrm{TPS}}_{\mathrm{llm}}={\displaystyle \frac{N_{\mathrm{gen}}}{T_{\mathrm{llm}}/1000}},$$

(34)

and the gain realization ratio

$$R_{\mathrm{gain}}={\displaystyle \frac{T_{\mathrm{total},\mathrm{int}}^{\mathrm{fp}}-T_{\mathrm{total},\mathrm{int}}^{\mathrm{q}}}{T_{\mathrm{llm}}^{\mathrm{fp}}-T_{\mathrm{llm}}^{\mathrm{q}}}},$$

(35)

where $T_{\mathrm{llm}}$ is the accumulated LLM generation time, $T_{\mathrm{total},\mathrm{int}}$ is the accumulated internal total inference time from per-sample logs, and $N_{\mathrm{gen}}$ is the total number of generated output tokens. These metrics complement VQT by distinguishing task-level deployment value from local module-level acceleration.

5.3. Results and Analysis

Table 13. Quantization results under a selective LLM quantization setting. Only the LLM layers are quantized, while the lm_head, vision_tower, and multimodal projector remain in higher precision.

Method	Size (GiB) ↓	Time ↓	ADE ↓	FDE ↓	Goal ↑	Cons. ↑	Unsup. ↓	Lat. ↓
Original	1.99	1:36:21	0.945	2.130	0.774	0.765	0.026	0.485
W8A8FP8	1.58	1:36:27	1.603	3.496	0.773	0.528	0.023	0.382
W8A16G32	1.63	1:38:40	0.945	2.130	0.773	0.766	0.026	0.484
W4A16G128	1.39	1:31:39	0.962	2.172	0.772	0.718	0.028	0.523
NVFP4	1.41	1:32:41	0.965	2.175	0.774	0.769	0.031	0.527

Note: ↑ indicates that higher values are better, while ↓ indicates that lower values are better. Bold values indicate the best performance in each column.

reports the standard planning metrics, Figure 7 summarizes the proposed task-aware deployment utility, and Figure 8 further analyzes the local LLM efficiency under selective quantization.

The results reveal three distinct quantization regimes. W8A16G32 behaves as a nearly lossless compression scheme. Its planning quality is almost identical to the full-precision baseline, with virtually unchanged ADE, FDE, Goal, and Consistency, leading to a quality retention ratio of $Q=0.9999$. However, it does not improve efficiency in our deployment setting. Its wall-clock runtime is slightly longer than the original model, and its local LLM efficiency is also slightly worse ($S_{\mathrm{llm}}=0.978$). As a result, its normalized VQT is below the full-precision baseline.

W8A8FP8 provides only modest local acceleration but substantially degrades planning quality. Although it achieves lower token-normalized latency and slightly higher local throughput, its ADE/FDE deteriorate severely and its action–trajectory consistency drops from 0.765 to 0.528. Consequently, its quality retention ratio decreases to $Q=0.8146$, yielding by far the worst VQT among all candidates. This indicates that overly aggressive low-precision quantization is harmful for structured autonomous driving planning.

W4A16G128 offers the best overall deployment trade-off. It achieves the smallest model size and the shortest wall-clock runtime while maintaining reasonably strong planning performance. At the module level, it also delivers the strongest local acceleration with the highest LLM local speedup ($1.118\times$), the lowest token-normalized latency, and the highest token throughput. These gains translate into the best VQT and the only normalized VQT above the full-precision baseline, making W4A16G128 the most attractive efficiency-oriented choice in our setting.

NVFP4 also achieves strong acceleration and competitive task-level utility. It provides substantial local speedup and the second-best VQT while even slightly exceeding the full-precision baseline on Goal and Consistency. However, its lower valid rate and weaker auxiliary metrics prevent it from surpassing W4A16G128 in overall deployment value.

Overall, these results show that selective LLM quantization should be evaluated from both quality and efficiency perspectives. A quantized model may preserve accuracy but fail to accelerate actual deployment (W8A16G32), or it may accelerate decoding while harming structured planning quality (W8A8FP8). In contrast, W4A16G128 achieves the best balance between compactness, inference speed, and planning quality, and it is therefore the best practical choice under our selective quantization setting.

6. Conclusions

In this paper, we proposed TDJPO, which is a trajectory-decision joint preference optimization framework for vision-language autonomous driving planning. We formulated planning as a structured autoregressive generation task, where the model jointly produces reasoning, discrete driving actions, and future trajectories. To align these structured outputs, TDJPO constructs preference pairs from stochastic rollouts using a unified reward that combines physical trajectory quality with decision-level behavioral consistency.

Experiments on NuScenes-TP demonstrate that TDJPO improves both trajectory accuracy and decision consistency compared with supervised fine tuning, trajectory-only preference optimization, and lightweight VLM baselines. Additional ablation studies, parameter sensitivity analysis, and safety-critical scenario evaluation further show that the gains arise from the joint alignment of physical feasibility, goal consistency, action–trajectory coherence, and safety-related regularization. We also investigated selective LLM quantization and showed that task-aware deployment metrics are important for evaluating the trade-off between planning quality and inference efficiency.

Despite these improvements, this paper has several limitations. The experiments are conducted in an offline planning benchmark, and the proposed method has not yet been evaluated in a closed-loop control environment. Moreover, the decision-level reward is based on interpretable rule-grounded metrics rather than human preference annotations or fully open-ended semantic critics. Future work will extend TDJPO to closed-loop simulation, integrate stronger human- or LLM-based feedback, and investigate more robust fallback mechanisms for safety-critical deployment.