Abstract
Truck platooning depends on frequent vehicle-to-vehicle communication to achieve platoon formation, which can create potentially substantial communication and computational burdens. To balance formation performance and communication efficiency, this paper proposes a Proximal Policy Optimization-based Event-Triggered Mechanism (PPO-ETM) for truck platoon formation. Under a predecessor-following topology, each follower uses locally available information and independently determines whether its state should be transmitted. PPO optimizes communication decisions by jointly considering formation errors and transmission costs, while a controller generates acceleration and steering commands. An analytically designed event-triggered filter is further incorporated to guarantee a strictly positive Minimum Inter-Event Time, thereby excluding Zeno behavior. Simulation results show that the proposed framework enables initially dispersed trucks to converge to and maintain the desired formation while avoiding redundant information transmissions. Comparative studies demonstrate that PPO-ETM achieves a favorable balance among formation accuracy, communication efficiency, and learning performance, outperforming other Reinforcement Learning methods. These results indicate that the proposed framework provides a decentralized and scalable solution for communication-constrained truck platooning.
1. Introduction
Connected and automated truck platooning has emerged as a promising technology for improving the efficiency, safety, and sustainability of road freight transportation [1,2,3,4,5,6]. Through coordinated longitudinal and lateral motion, multiple trucks can travel as an organized formation, thereby reducing aerodynamic drag, increasing road capacity, and improving their collective response to changes in traffic conditions [7]. Such coordination, however, relies on the timely exchange of vehicle states over vehicle-to-vehicle (V2V) communication networks [4,8,9]. Most existing platoon control systems employ periodic communication, under which each truck broadcasts its state at every sampling instant, regardless of whether the newly sampled information is necessary for achieving the desired formation [10]. Although straightforward to implement, this time-driven communication paradigm produces a substantial number of redundant transmissions and increases channel-access, bandwidth, and data-processing demands. These limitations become increasingly pronounced as the platoon size and operating duration increase, potentially undermining the scalability and reliability of networked platoon control. Communication-efficient platooning therefore requires a mechanism that can distinguish information updates that are valuable to the system from those that contribute little to formation regulation, so that communication is performed only when it is genuinely required.
Existing studies on truck platooning and networked control have mainly considered four related directions. The first direction uses periodic communication, in which predecessor or neighboring truck states are transmitted at every sampling instant. This strategy is simple to implement, but it may generate redundant transmissions when the formation error and the control demand are small. The second direction uses analytically designed event-triggered conditions, which can reduce communication while providing stability-related guarantees [4,5]. However, the triggering thresholds are usually designed from a prescribed model and may be conservative when the operating conditions vary. The third direction applies RL to truck control or communication scheduling. These methods provide greater adaptability, but the learned communication policy alone does not necessarily guarantee a strictly MIET. The fourth direction considers multi-agent RL for distributed platooning [6]. Such methods can handle local decision-making, but independent policy updates may introduce non-stationarity in directed predecessor-following networks. Event-Triggered Mechanisms (ETMs) provide a means of reducing this communication burden by transmitting information only when a prescribed state-dependent condition is satisfied [11]. Early static ETMs typically compared the measurement or estimation error with a fixed threshold, thereby eliminating part of the redundant communication induced by periodic sampling. Dynamic ETMs subsequently introduced auxiliary internal variables into the triggering condition, offering additional flexibility in balancing communication demand and closed-loop performance [12,13,14,15]. Despite these advances, conventional ETMs are commonly constructed within a Lyapunov-based stability framework, in which a triggering condition is derived as a sufficient condition for ensuring the non-positivity of the derivative of a selected Lyapunov function [16,17]. Although this analytical paradigm provides rigorous stability guarantees, it requires an appropriate Lyapunov function, sufficiently accurate system models, and manually tuned design parameters [18,19,20]. More importantly, because Lyapunov-based triggering conditions are generally sufficient rather than necessary, the resulting triggering boundaries may be conservative. A communication event may therefore be generated even when the newly available information provides only a marginal improvement in the actual closed-loop behavior, resulting in a higher communication frequency than is practically necessary. Reinforcement Learning (RL) offers a data-driven alternative to such predefined triggering rules [21,22]. By incorporating formation errors and communication costs into the cumulative return, an RL algorithm can learn the long-term policy of transmitting information under different operating conditions. Nevertheless, directly treating the binary actions produced by an RL policy as communication events introduces a fundamental feasibility problem [23]. The policy is optimized to maximize the expected cumulative reward rather than to regulate the temporal distribution of its triggering decisions. Consequently, the learned policy may generate locally concentrated events or arbitrarily short inter-event times, even when it achieves a low average triggering frequency. In the limiting case, these unconstrained learned decisions may give rise to Zeno behavior, and neither reward shaping nor empirical training can, by itself, establish a strictly positive Minimum Inter-Event Time (MIET) [21,22,24]. This reveals a critical gap between the two existing paradigms: Lyapunov-based ETMs provide rigorous stability and feasibility guarantees but may use communication resources conservatively, whereas RL-based triggering can adaptively learn the long-term policy of communication but does not inherently guarantee a Zeno-free event sequence. Combining the communication efficiency of learned triggering with a rigorous Zeno-free guarantee therefore constitutes the central motivation of this study [25,26].
To bridge this gap, this paper develops a Proximal Policy Optimization-based ETM (PPO-ETM) for communication-efficient truck platooning [27]. The proposed framework separates event-triggered policy from vehicle control. Rather than generating acceleration and steering commands, the PPO policy observes the locally available formation information and learns a binary strategy that determines whether the sampled information should be updated. The learning objective jointly accounts for platoon control and communication cost, enabling the policy to evaluate the long-term contribution of each information update instead of relying exclusively on a fixed and potentially conservative analytical threshold. When a communication event is triggered, the latest neighboring states are transmitted and the sampled formation information is updated; otherwise, the controller continues use the previously received information. Based on this event-triggered information, a structured formation controller generates the continuous longitudinal and lateral control inputs, thereby preserving the physical feasibility of the closed-loop truck dynamics. Crucially, the binary decisions learned by PPO are not directly implemented as unconstrained communication events. They are incorporated into the proposed ETM, which imposes an analytically designed temporal condition on the triggering instants and therefore prevents Zeno behavior that may be produced by a stand-alone RL policy. As established by the Zeno-free analysis in Section 3.2, the proposed ETM guarantees a strictly positive MIET and consequently excludes Zeno behavior. The resulting PPO-ETM thus combines the adaptability of data-driven communication scheduling with the rigorous feasibility of event-triggered control, allowing communication resources to be reduced without exposing the truck platoon to arbitrarily frequent triggering.
The present study shares several assumptions with the existing platooning literature [1,3,4]. In particular, it uses a kinematic truck model, a directed information graph, local predecessor information, bounded truck inputs, and a desired spacing policy. These assumptions are appropriate for investigating the communication–control interaction in a platoon, but they do not represent every practical traffic condition. The proposed method differs from conventional periodic and event-triggered strategies in the role assigned to RL. PPO does not directly generate acceleration or steering commands and does not directly define the final communication event. Instead, PPO produces a binary communication request according to the local truck state and formation information. This request is subsequently filtered by the ETM. The actual event occurs only when both the PPO request and ETM condition are satisfied.
The specific objective of this study is to develop and evaluate a communication-aware decision layer for truck platooning. The layer determines whether a follower should request a predecessor-state update, while the underlying constrained formation controller remains responsible for the physical motion command. The study therefore does not attempt to replace the complete vehicle control stack or claim full field validation. Instead, it focuses on the communication–control interface, where reduced V2V transmissions must be balanced against formation accuracy and control effort. Consequently, the proposed PPO-ETM framework combines a learned communication policy with an analytically triggering filter. The PPO component evaluates the long-term trade-off among formation errors, control effort, and communication requests, whereas the ETM provides a strictly positive MIET. This division of responsibilities distinguishes the proposed method from approaches that use RL as the complete controller or use a fixed event-triggering threshold without a learned communication policy.
The main contributions of this paper are summarized as follows:
- The proposed communication strategy reduces redundant V2V transmissions while maintaining the desired platoon formation performance. In contrast to conventional Lyapunov-based ETMs, whose analytically derived sufficient conditions are predefined and may produce conservative communication schedules, the proposed PPO-based strategy evaluates the long-term value of each information update by jointly accounting for formation errors and communication costs [16,17,18,19,20]. Consequently, it maintains formation performance while enabling adaptive communication scheduling, providing a more effective balance between platoon regulation and communication efficiency than fixed and manually tuned triggering rules.
- The proposed learning-based triggering framework guarantees practical feasibility by excluding Zeno behavior and ensuring that successive communication events are separated by a strictly positive MIET. Stand-alone RL triggering strategies directly execute learned binary actions and cannot prevent locally concentrated events through reward shaping alone [21,22,24]. In contrast, the proposed ETM filters the communication requests generated by PPO using an analytically designed triggering condition and theoretically guarantees a strictly positive MIET. The resulting PPO-ETM therefore retains the adaptability of data-driven communication scheduling while providing a rigorous Zeno-free guarantee for practical deployment in networked truck platoons.
- The proposed framework enables initially dispersed trucks to converge and maintain the prescribed platoon formation. Each follower uses only the state information received from its immediate predecessor, without requiring global topology information [8]. Unlike centralized or leader-dependent formation methods, the proposed approach constructs the desired relative state of each follower from its predecessor. It also allows each follower to make communication decisions locally. In this way, the formation objective propagates sequentially along the vehicle string. The resulting predecessor-following architecture reduces both information requirements and communication complexity, thereby improving the scalability and practical deployability of PPO-ETM under realistic V2V communication constraints.
2. Problem Formulation
Consider a platoon consisting of N vehicles indexed by , where vehicle 1 represents the leading vehicle, and the remaining vehicles are followers. The platoon travels along a road-aligned coordinate frame. For each vehicle , the state vector is defined as
where , , , and denote the longitudinal position, lateral displacement, velocity, and heading angle of vehicle i, respectively. The control input is given by
where and represent the acceleration and front-wheel steering angle, respectively.
The motion of each vehicle is described by the dynamical model
where denotes the wheelbase [28]. The vehicle velocity and actuation are subject to physical constraints. Let
where and are compact admissible sets. These sets encode speed limits, acceleration limits, steering limits, and actuator rate bounds. The detailed structure of the admissible sets is assumed to be known and compatible with vehicle operation.
The inter-vehicle information exchange is represented by a directed graph
where means that vehicle i can receive information from vehicle j. The neighbor set of vehicle i is defined as
The directed graph allows asymmetric information flow and does not require bidirectional communication. For each follower , let
denote its reference predecessor selected according to the platoon ordering and the available communication topology. The desired vehicle spacing is denoted by . The desired vehicle spacing between vehicle i and is
The local formation error of follower i is defined as
where
The condition represents local formation consistency, namely desired spacing, lateral alignment, velocity synchronization, and heading alignment with respect to . The leading vehicle is associated with a nominal lane reference. Its state is defined by , where denotes the desired cruising speed.
Let denote the sequence of event times associated with vehicle i. At an event-triggered time , vehicle i updates the received from vehicle . Between two consecutive event-triggered times, vehicle i retains the most recently sampled neighbor information . The formation error constructed from sampled neighbor information is denoted by
where
The information error is defined as
where is the formation error stored at the most recent event-triggered time.
The platoon coordination objective is to achieve a formation. Specifically, for every follower , the desired asymptotic relation is
The problem considered in this paper is to formulate cooperative platoon coordination for constrained vehicles under directed and event-triggered communication. The model accounts for nonlinear vehicle kinematics, directed information exchange, sampled neighbor information, and event-triggering variables. The objective is to characterize a formation in which vehicles synchronize their velocities, positions, and headings while maintaining prescribed separations.
3. Methods
The vehicle controller used in this study is a constrained predecessor-following formation controller. It consists of a longitudinal spacing/velocity controller and a preview-based lateral controller. The controller receives the local vehicle state and the latest available predecessor information. It does not receive the PPO action as a physical acceleration or steering command. Instead, the PPO action only determines whether an information update should be requested. The controller then uses either the newly received or the retained predecessor information to generate the vehicle command.
3.1. Controller
This subsection presents the vehicle controller used to achieve platoon formation under the nonlinear kinematic model in (3). The controller generates the acceleration and the front-wheel steering angle , whereas the ETM and the RL module determine when the sampled neighbor information should be updated. Therefore, the RL policy does not directly output the control input. Instead, the input is computed by a constrained formation-tracking controller based on the sampled formation error .
For each follower , the controller uses the sampled formation error , which is constructed from the most recently sampled state of the selected predecessor . The use of reflects that the controller retains the latest available predecessor information rather than requiring continuous communication between two consecutive triggering instants. The nominal acceleration command is defined as
where and denote the spacing and velocity feedback gains, respectively. This feedback structure increases the acceleration when the actual distance to the vehicle exceeds the desired spacing or when the predecessor is moving faster, and reduces the acceleration when the vehicle is too close or moving faster than vehicle .
The lateral controller is designed in a preview-based form to avoid aggressive steering actions. Instead of directly converting the instantaneous lateral displacement error into a steering command, the controller first constructs a preview heading angle. Let be the preview time and define the preview distance as
where prevents the preview distance from becoming excessively short at low speeds. The preview heading angle is given by
Combining the preview heading with the sampled heading alignment error gives the desired heading correction
The nominal steering command is then computed as
where is the heading feedback gain. This preview formulation produces smoother lateral behavior than direct lateral-error feedback and is particularly suitable for heavy-duty vehicles, for which abrupt steering actions may induce undesirable yaw-rate and lateral-position transients.
To ensure physical feasibility and operational safety, the nominal commands are projected onto admissible actuation sets and before being applied to the vehicle. The acceleration command is first saturated as
where and denote the maximum braking and driving acceleration limits, respectively. In addition, an acceleration-rate constraint is imposed to avoid unrealistic jerk and to reflect actuator dynamics
where is the admissible acceleration-rate bound. This constraint is enforced by
The steering input is constrained by both the mechanical steering limit and a speed-dependent acceleration limit. From the kinematic model, the yaw-rate dynamics satisfy (3). To keep the lateral acceleration within the admissible bound , the steering angle is restricted by
where avoids singular behavior at very low speed. Combining this speed-dependent lateral-acceleration constraint with the mechanical steering bound yields the effective steering limit
Here, denotes the mechanical steering limit, is the maximum admissible lateral acceleration, and preserves sufficient steering authority when the lateral-acceleration-induced bound becomes overly restrictive. The steering command is then saturated according to
A steering-rate constraint is further imposed to represent actuator limitations and suppress abrupt steering variations:
where is the maximum steering-rate bound. The applied steering angle is updated by
followed by a final projection onto the admissible steering interval
After state propagation, the vehicle speed is projected onto the admissible sets and . Consequently, the controller provides a physically feasible control that regulates spacing, lateral alignment, velocity synchronization, and heading consistency while respecting acceleration, steering, and speed constraints. The ETM and the RL policy determine when the sampled predecessor information used to construct is refreshed.
The reliability of the proposed controller is considered at three levels. First, the physical command is constrained before being applied to the truck. The acceleration is limited by the admissible interval and the acceleration-rate constraint in (15)–(17). The steering command is constrained by the mechanical steering bound and the lateral-acceleration limitation in (18). Therefore, a communication decision cannot directly produce an inadmissible acceleration or steering command. Second, when no communication event occurs, the follower does not use an arbitrary or missing state. It retains the most recently received predecessor information. This implementation is consistent with the event-triggered communication architecture, but it also introduces information age. The information age should be monitored in practical implementation because excessive age may degrade formation performance. Third, a practical implementation should include a timeout and safety-supervisory layer. If the age of the retained predecessor information exceeds a prescribed threshold, the truck should switch from the nominal communication-aware controller to a safe controller. The safety supervisor should also override the learned communication decision when the measured spacing approaches a safety threshold or when an emergency braking condition is detected.
Remark 1.
From an electronics perspective, the controller is part of a networked embedded control loop. The state-estimation module, V2V transceiver, PPO actor, ETM, and low-level actuator interface operate as interconnected electronic modules. The communication decision affects the freshness of the received state, the retained information affects the controller input, and the vehicle response changes the next observation received by the PPO actor. Therefore, the proposed method concerns not only vehicle dynamics but also the scheduling of information exchange in an embedded cyber-physical system.
3.2. Event-Triggered Mechanism
A dynamic ETM is introduced to regulate information updates and prevent Zeno behavior. For each follower i, an auxiliary variable is governed by
where , , and are design parameters. The denotes a step function, with for and for . Compared with a static threshold, introduces a time-varying margin that adapts the communication frequency to the evolution of the formation and information errors.
The triggering function is defined as
where is a design parameter. The strictly positive term is particularly important near the desired formation, where both and may become small. It prevents the triggering threshold from vanishing and provides a positive lower bound on the inter-event time, as established below. Accordingly, the next event instant is determined by
At , the predecessor information is refreshed, and the sampled formation error and are updated. Between two consecutive events, the latest transmitted information is retained.
Since a triggering policy learned through RL does not inherently guarantee a positive inter-event time, it may produce excessively dense triggering decisions and potentially induce Zeno behavior. The ETM is therefore introduced as a safeguard to exclude such triggering sequences by ensuring a strictly positive lower bound on the inter-event time.
Proof.
Immediately after an information update, the sampled and current information coincide, yielding
At the subsequent event instant, the triggering condition and the non-negativity of imply
Therefore,
Between two consecutive event instants, the sampled information is held constant, whereas the current information evolves with the vehicle dynamics. Therefore,
Integrating (29) over the interval , the evolution of the information error can be expressed as
This relation shows that, during each inter-event interval, the information error is entirely induced by the evolution of . There exists a finite positive constant M, independent of the event index, such that the accumulated variation in over the considered interval is bounded. Applying the triangle inequality then gives
Combining this threshold condition with (28) yields
It follows that a non-zero amount of time is necessarily required for the information error, initially reset to zero, to reach the triggering threshold. Thus,
Consequently, the proposed ETM admits a positive MIET, and Zeno behavior is excluded. □
3.3. PPO-Based Event-Triggering Policy
To reduce unnecessary communication while maintaining the desired platoon performance, the event-triggering decision is formulated as a MDP and learned using PPO. In contrast to a fixed threshold-based rule, the learned policy determines whether communication is required according to the current vehicle state and the available information.
At time t, vehicle i constructs the local observation
where is the physical state of the vehicle i, denotes the current formation-related information, and represents the information retained from the latest transmission. Hence, the observation simultaneously characterizes the vehicle motion and the discrepancy between the current and previously transmitted information.
Based on , the actor produces a probability distribution over the binary action space
where denotes the actor parameters. The action represents a communication request, whereas indicates that the policy does not request an information update. The PPO action is evaluated at each policy sampling instant but does not, by itself, constitute a communication event. The actual communication decision is defined as
Thus, PPO determines whether communication is requested according to the current operating condition, whereas the ETM determines whether the requested transmission is admissible. By guaranteeing a strictly positive lower bound on the inter-event times, the ETM prevents the PPO policy from generating Zeno behavior under this triggering logic.
When , the vehicle i receives the current information from its reference predecessor and refreshes the retained formation information . The corresponding update rule is
If , the current time is recorded as the actual triggering instant,
Otherwise, the retained information remains unchanged, and no new element is added to the event-time sequence . Therefore, t represents a general policy-evaluation time, whereas is reserved exclusively for an actual communication event.
The reward is designed to account for formation performance, control effort, and communication demand. A general stage reward for vehicle i is defined as
where are the formation-error weights, are the control-effort weights, and is the communication-request weight. In particular, the communication penalty is imposed on the PPO action rather than the actual triggering decision . Hence, every transmission request generated by the PPO policy incurs a communication cost, even when its execution is subsequently inhibited by the ETM. Maximizing the expected cumulative reward therefore enables the policy to balance formation accuracy, control smoothness, and communication economy.
During training, the actor is updated using the PPO clipped surrogate objective
where
is the probability ratio between the updated and behavior policies, is the estimated advantage, and is the clipping parameter. The critic, parameterized by , estimates the state value and is trained by minimizing the value loss
where denotes the sampled return. By limiting the policy probability ratio to a neighborhood of unity, the clipped objective prevents excessively large policy updates and improves training stability.
The actor and critic are trained using trajectories generated by the platoon system. During actual operation, only the actor is required. At each policy sampling instant, it maps directly to the binary communication request , after which the ETM evaluates and determines the actual communication decision . The vehicle control input remains generated by the constrained formation controller and is not directly produced by the RL policy.
The proposed framework is organized as a layered vehicle-side architecture consisting of state estimation, local observation construction, communication decision-making, event-triggered filtering, and constrained motion control. At each control cycle, vehicle i obtains its local state from onboard sensors or a state-estimation module. The local formation error is then constructed from the vehicle state and the latest available information from the reference predecessor. The PPO actor receives the local observation and outputs a binary communication request . This request is not directly interpreted as a transmitted packet. Instead, it is evaluated by the dynamic ETM. An actual transmission occurs only when and the ETM condition is satisfied. If no transmission occurs, the follower retains the most recently received predecessor information. After the communication information is updated or retained, the constrained formation controller computes the longitudinal acceleration and steering commands. Therefore, the learned policy is responsible for communication scheduling, whereas the model-based controller is responsible for generating physically admissible vehicle inputs. This separation allows the safety-related input constrains to be enforced independently of the policy output. During online execution, the critic network and the policy-update procedure are not required. Only the trained actor, the ETM, the state-estimation module, and the constrained controller are executed on the vehicle-side computing unit. This implementation structure reduces the online computational requirement compared with performing policy optimization onboard.
Algorithm 1 summarizes the execution of the proposed PPO-ETM framework. The PPO actor generates only a communication request , whereas an actual communication event occurs only when the request satisfies the ETM condition. If no event is triggered, the latest valid predecessor information is retained. The constrained formation controller, rather than the RL policy, generates the acceleration and steering commands.
| Algorithm 1 PPO-ETM Communication and Platoon Control |
|
4. Experiments
This section evaluates the proposed PPO-based ETM from four perspectives. First, the proposed PPO-ETM strategy is compared with full triggering and PPO triggering to investigate its control-communication tradeoff. Second, PPO is compared with several representative RL algorithms in terms of execution time and triggering frequency. Finally, the training behavior of IPPO is analyzed to explain the use of a shared PPO policy instead of independent policies for individual vehicles.
The experiments are conducted using the vehicle platoon model, constrained controller, dynamic ETM, and PPO communication policy introduced in the preceding sections. All experiments use the same vehicle parameters, initial conditions, sampling period, and reference trajectory to ensure a fair comparison. The platoon consists of a leader and four followers. The desired inter-vehicle spacing is set to , and the reference velocity is . Based on the steady-state cornering relation [29], the maximum front-wheel steering angle is set to to limit the lateral acceleration during high-speed platoon operation. The principal simulation, controller, ETM, and PPO parameters are summarized in Table 1. The parameters are kept unchanged in the comparisons unless a different setting is explicitly stated.
Table 1.
Simulation and algorithm parameters.
The parameters in Table 1 were determined according to three principles: physical feasibility, control-performance requirements, and learning-performance requirements. The vehicle and formation parameters were first selected from the operating scenario and the kinematic truck model. The desired spacing and reference velocity define the nominal platooning task. The wheelbase L, acceleration bounds, steering bounds, and actuator-rate bounds were selected to represent physically admissible truck motion. In particular, the steering bound was calculated from the lateral-acceleration constraint using the steady-state cornering relation described in [29]. The controller gains and ETM parameters were then selected to maintain a feasible closed-loop response while avoiding unnecessarily frequent information updates. The parameters , , , and determine the evolution of the dynamic triggering threshold. The positive constant was selected to maintain a non-zero triggering margin near the desired formation and is required by the Zeno-free triggering analysis. The PPO learning rates, discount factor, and network dimensions were selected using preliminary training runs. The final values were selected according to validation performance measured by formation error, control effort, triggering frequency, and cumulative reward. The test trajectories were not used to tune the PPO hyperparameters.
The information topology used in the experiments is illustrated in Figure 1. The leader is denoted by vehicle 1, whereas vehicles 2∼5 are followers. A directed edge indicates that vehicle i uses the latest transmitted information from vehicle j. The topology is fixed throughout the simulation, while the availability of the information associated with each edge is determined by the adopted communication strategy.
Figure 1.
Communication topology.
4.1. Comparison of Triggering Strategies
This subsection examines the role of the two constituent components of the proposed communication strategy. The comparison with full triggering evaluates whether PPO-ETM can preserve the platoon behavior under substantially fewer information updates. The comparison with PPO triggering reveals how the analytical ETM filters the communication requests generated by the learned policy.
Remark 2.
To examine the generalization of the learned communication policy, the simulation conditions were divided into training and test scenarios. The training scenarios were used to update the actor and critic parameters. The test scenarios were reserved for the final evaluation and were not used for parameter tuning. Although RL does not suffer from over-fitting in the same way as supervised learning, policy over-specialization to the training scenarios may occur. To examine this issue, we evaluated the learned policy on independent testing instances that were not used during training. The consistent performance between training and testing scenarios indicates that the proposed RL model does not suffer from significant overfitting.
4.1.1. Full Triggering Versus PPO-ETM
To evaluate whether communication reduction affects the platoon control performance, the proposed PPO-ETM strategy is compared with the full-triggering strategy. Under full triggering, predecessor information is updated at every sampling instant, corresponding to a communication frequency of . In contrast, PPO-ETM executes a transmission only when the PPO policy requests an information update and the request satisfies the ETM condition.
Figure 2 compares the trajectories obtained under the two communication strategies. The solid curves represent the trajectories generated by PPO-ETM, whereas the dashed curves represent those generated by full triggering. The star and circle markers indicate the initial and final positions of the trucks, respectively. Despite their substantially different information-update frequencies, the two strategies produce closely comparable trajectories throughout the platoon formation process. In particular, only small differences can be observed for Trucks 2∼5. The trucks start with different positions and therefore exhibit distinct responses. Trucks 2 and 3 undergo the largest positive and negative lateral corrections, respectively, whereas Trucks 4 and 5 require smaller lateral adjustments than Trucks 2 and 3. Under both communication strategies, the lateral displacements converge smoothly toward the reference lane center, . Moreover, PPO-ETM introduces neither sustained lateral oscillations nor an appreciable lateral offset.
Figure 2.
Truck trajectories under full triggering and PPO–ETM. Solid curves denote the PPO–ETM trajectories, whereas dashed curves denote the full-triggering trajectories. The star markers indicate the initial positions of individual trucks, while the circular markers indicate their corresponding final positions.
Notably, PPO-ETM achieves trajectory responses comparable to those of full triggering while operating at a communication frequency of only in Figure 3, defined as the ratio of the number of executed communication events to the total number of possible transmissions. Although small differences between the solid and dashed curves arise during the transient stage, these differences gradually diminish as the trucks converge to the desired platoon formation without causing sustained lateral oscillations or appreciable steady-state offsets. Therefore, the proposed strategy substantially reduces the communication demand while preserving formation performance close to that of full triggering, demonstrating an effective tradeoff between platoon control performance and communication efficiency.
Figure 3.
The faded markers represent the communication requests generated by PPO, whereas the opaque markers represent the events executed under PPO–ETM. PPO–ETM reduces the triggering frequency from to while preserving comparable platoon trajectories.
4.1.2. PPO Triggering Versus PPO-ETM
This experiment further investigates the role of the ETM by comparing stand-alone PPO triggering with the proposed PPO-ETM strategy. In Figure 3, the faded markers represent the event-triggering instants by stand-alone PPO, whereas the opaque markers denote the event-triggering instants under PPO-ETM. The vertical axis identifies the four follower trucks, namely Trucks 2∼5.
The stand-alone PPO policy generates a large number of densely clustered event-triggers, particularly during the initial platoon-formation. For several followers, the event-triggers are nearly continuous over short time intervals. This phenomenon occurs because the learned PPO policy optimizes the cumulative reward but does not inherently impose a strictly positive MIET. Consequently, PPO alone may produce unsafe Zeno behavior.
By contrast, the opaque PPO-ETM event-triggers are clearly separated and much sparser over the entire simulation horizon. A PPO event-trigger is executed only when it also satisfies the triggering condition (36). The ETM, therefore, filters the densely clustered and state-wise unnecessary event-triggers produced by PPO. More importantly, the ETM guarantees a strictly positive MIET, as established in the theoretical analysis. Hence, the proposed PPO-ETM strategy excludes Zeno behavior rather than merely reducing the number of event-triggers.
The triggering frequency of the stand-alone PPO is , whereas that of PPO-ETM is only . Thus, the ETM eliminates more than of the communication requests that would otherwise be executed by the stand-alone PPO policy. The remaining events are distributed over the complete simulation horizon, indicating that communication is retained when required by the state rather than being entirely suppressed.
Figure 4 compares the corresponding platoon trajectories. The dashed curves represent stand-alone PPO, while the solid curves represent PPO-ETM. Despite the substantial reduction in triggering frequency, the solid and dashed trajectories remain closely aligned for all followers. Small differences are visible mainly during the initial transient when the trucks correct their lateral offsets and establish the platoon. The PPO trajectories are also close to those under full triggering because PPO generates high-frequency communication requests during this initial transient, resulting in an information-update pattern similar to full triggering. These deviations subsequently decay, and both PPO and PPO-ETM guide the trucks toward the same longitudinal path with vanishing lateral displacement.
Figure 4.
The dashed and solid curves denote the trajectories under PPO and PPO–ETM, respectively. The star symbols denote the starting locations of the trucks, whereas the circular markers represent their corresponding ending locations.
The comparison of trajectories confirms that the lower communication frequency of PPO-ETM is not achieved by sacrificing the essential formation behavior. Instead, PPO-ETM retains trajectory performance comparable to stand-alone PPO while substantially reducing the communication burden and enforcing the theoretically established Zeno-free. Therefore, the ETM serves as both a communication filter and a safeguard for the learned triggering policy.
4.1.3. Sensitivity Analysis of the Communication Weight
To further investigate the influence of the communication term in the reward function. a sensitivity analysis is conducted by varying while keeping the remaining reward weights and the ETM parameters unchanged. The PPO request frequency and the actual event-triggering frequency are reported separately because the PPO action only represents a communication request, whereas an transmission occurs only when the request also satisfies the ETM condition in (36). The communication-request weight was varied over , while all other reward weights, PPO hyperparameters, ETM parameters, vehicle parameters, and initial conditions were kept unchanged.
Figure 5 illustrates the sensitivity of the communication behavior to the communication-request weight . The upper figure presents the frequency of PPO communication requests , whereas the lower figure shows the frequency of actual communication events after filtering by the ETM. As increases from 0 to , the PPO request frequency generally decreases from to approximately 14.3%∼15.7%, indicating that a larger communication penalty discourages the policy from requesting information updates. However, the actual event-triggering frequency remains much lower, decreasing from to , because not every PPO request satisfies the ETM condition. The non-monotonic variations observed at and suggest that the learned policy and the ETM interact in a nonlinear manner. Therefore, increasing the communication penalty does not necessarily produce a strictly monotonic reduction in actual transmissions. These results demonstrate that provides an effective tuning parameter for balancing communication demand and information-update availability, while excessively penalizing communication may cause the policy to request fewer updates and potentially increase the age of the retained predecessor information. Consequently, the communication weight should be selected jointly with formation accuracy and control performance rather than by minimizing the triggering frequency alone.
Figure 5.
Sensitivity of PPO communication requests and actual ETM-triggering events to the communication-request weight . The upper panel shows the PPO request frequency , while the lower panel shows the actual event-triggering frequency . The difference between the two frequencies demonstrates the filtering effect of the ETM.
When , the policy has no explicit penalty for communication requests, resulting in the highest request frequency of . Increasing the weight to and reduces the request frequency to and , respectively. The request frequency reaches its lowest value of at , but it slightly increases at larger values. This slight increase may be caused by the change in the state distribution of the learned policy and the resulting interaction between communication requests and formation regulation. In contrast, the actual event-triggering frequency remains within a narrow range of 1.22%∼1.53%, confirming that the ETM substantially filters the PPO requests and maintains sparse communication. Therefore, the communication-request weight primarily affects the learned decision tendency, while the ETM determines whether a requested update is admissible as an actual communication event.
4.2. Selection of PPO
The runtime and communication results are summarized in Figure 6 and Figure 7, respectively. As shown in Figure 6, PPO requires to complete the experiment. Although its runtime is not the lowest among the evaluated algorithms, the difference from the fastest method, REINFORCE, is limited to . Moreover, PPO remains faster than A2C, IPPO [30], and SAC [31], whose runtimes are , , and , respectively. Therefore, PPO does not impose the minimum computational cost, but its execution time remains acceptable for the task.
Figure 6.
Runtime comparison of the evaluated RL algorithms.
Figure 7.
Event-triggering frequencies of the evaluated RL algorithms.
A substantially different trend is observed for communication efficiency. As shown in Figure 7, PPO achieves the lowest event-triggering frequency, namely . The frequencies of DQN [32], DDQN [33], SAC, A2C, IPPO, and REINFORCE are , , , , , and , respectively. Thus, PPO reduces the communication frequency by relative to DQN and by relative to REINFORCE. Compared with IPPO, which has the second-highest triggering frequency among the compared methods, PPO achieves a reduction of .
These results indicate that the computational cost and communication cost do not necessarily favor the same learning algorithm. Selecting REINFORCE solely according to runtime would reduce the execution time by compared with PPO, but this advantage is accompanied by a threefold higher triggering frequency versus . In a truck–platoon communication system, such an increase directly results in more channel access requests and greater wireless communication overhead over prolonged operation. Hence, the small runtime advantage of REINFORCE is less significant than the communication savings achieved by PPO.
PPO therefore provides the most favorable system-level tradeoff among the evaluated methods. It incurs a moderate computational cost while substantially reducing the communication burden. This property is particularly important for the proposed architecture, in which the learning policy is used to determine whether communication is needed rather than to generate control commands at every sampling instant. On this basis, PPO is selected as the learning component of the event-triggered communication mechanism.
Remark 3.
The current study has two important scope limitations. First, the numerical experiments consider a platoon of five trucks, including one leader and four followers. This configuration is sufficient to examine the interaction between local communication decisions and formation control, but it does not fully represent large commercial platoons. As the number of vehicles increases, communication-channel contention, computational load, information age, and error propagation may become more significant. In particular, the performance of a distant follower may depend on a sequence of event-triggered information updates along the directed graph. Second, the experiments uses a fixed predecessor-following topology. This topology is useful for isolating the communication–control problem and is consistent with local V2V information exchange, but real platoons may use bidirectional, two-predecessor, leader-assisted, or dynamically reconfigured topologies. Vehicle insertion, vehicle departure, link failure, and topology switching are not considered in the present version. Therefore, the decentralization demonstrated in this paper should not be interpreted as a proof of scalability to arbitrary fleet sizes and communication graphs. The present results establish the behavior of the proposed PPO-ETM framework under the considered five-truck predecessor-following scenario. Systematic evaluation with larger fleets, alternative graph structures, communication congestion, and dynamic topology changes is required before practical deployment.
4.3. Discussion on IPPO
Figure 8 and Figure 9 present the platoon trajectories and event-triggering instants, respectively, obtained at an intermediate stage of IPPO training. A pronounced difference is observed between the policies learned by Trucks 4 and 5. Truck 5 has already acquired a policy that generates a smooth lateral maneuver toward the target lane. By contrast, Truck 4 has not yet learned an appropriate communication strategy, and its resulting longitudinal motion remains markedly inconsistent with the desired platoon configuration. Consequently, the downstream part of the platoon fails to attain the formation established by Trucks 1∼3.
Figure 8.
Platoon trajectories at an intermediate stage of IPPO training. Stars mark the initial truck positions, while circles denote the final positions.
Figure 9.
Event-triggering instants at an intermediate stage of IPPO training.
This result reveals a topology-induced training difficulty of IPPO. According to the topology, Truck 5 relies on information from Truck 4. The observation received by Truck 5 is therefore affected not only by its own policy and state, but also by the evolving policy of Truck 4. Because the policy of Truck 4 is still being updated and has not yet converged, the state-action distribution encountered by Truck 5 changes during training. Thus, even though Truck 5 has individually learned a seemingly suitable policy, subsequent updates of Truck 4 alter its upstream information, thereby perturbing the training target of Truck 5. The learned policy of Truck 5 must consequently adapt repeatedly to a moving upstream policy.
Consider the predecessor-following chain
The local observation of Truck 5 can be written abstractly as
where and denote the communication policies of Truck 4 and Truck 5. Because Truck 4 determines when its predecessor information is refreshed, the distribution of depends on the evolving policy of Truck 4. During independent training, is updated simultaneously with . Therefore, from the perspective of Truck 5, the transition distribution is not stationary. Since changes during training, the target distribution used by Truck 5 also changes. This is the moving-target effect observed in the IPPO experiment.
Accordingly, IPPO would require additional training iterations for the upstream policy to stabilize and for the downstream policy to readapt to the resulting state distribution. The required training effort is therefore greater than that of PPO in the present task. Additional iterations may alleviate this effect, but they also increase the training cost and do not remove the underlying instability in the training environment caused by simultaneous independent updates. Given that PPO attains the required platoon formation with a lower triggering frequency and without this inter-policy interference, IPPO is not selected for the proposed communication architecture.
5. Conclusions
This paper developed a PPO-based event-triggered communication framework for truck platoon formation under limited V2V communication. In the proposed framework, each follower uses only the state information of its immediate predecessor and independently determines whether an information update is required. PPO evaluates communication requests by jointly considering formation performance and communication cost, while the analytically designed ETM filters the learned triggering actions. This combination preserves the adaptability of data-driven scheduling and provides a strictly positive MIET, thereby excluding Zeno behavior. Simulation results demonstrated that the proposed PPO-ETM enables initially dispersed trucks to form and maintain the prescribed platoon while reducing unnecessary information transmissions. Comparisons with alternative RL methods further showed that PPO provides a favorable balance among formation accuracy, communication efficiency, and computational cost. The results also revealed that IPPO suffers from topology-induced training instability because downstream policies depend on continuously evolving upstream policies.
The proposed PPO-ETM framework also provides several managerial implications for connected truck–platoon operations. First, communication should be treated as an operational resource rather than an unlimited service. By transmitting predecessor information only when it is useful for the closed-loop platoon task, the proposed strategy can reduce channel occupancy and data-processing demand. This is particularly relevant when multiple platoons share the same wireless infrastructure. Second, the communication-cost weight in the reward function provides an operational tuning mechanism. A larger communication penalty may reduce the transmission frequency, but excessive penalization can increase formation errors or information age. Fleet operators should therefore select this weight according to the required safety margin, available communication capacity, and desired control accuracy rather than minimizing transmissions alone. Finally, communication reduction should not be evaluated independently of safety and service quality. Practical deployment should monitor formation error, minimum spacing, information age, packet-loss rate, triggering frequency, and emergency-controller activations. These indicators can be used by fleet managers to determine whether the learned communication policy remains within an acceptable operating envelope. The present study has not yet been validated on a physical truck, hardware-in-the-loop platform, or public-road test. Accordingly, the reported results should be interpreted as simulation-based evidence rather than a complete demonstration of field deployment. The implementation-oriented architecture clarifies how the PPO actor, ETM, V2V module, and constrained controller can be integrated, but hardware-level issues such as processor scheduling, sensor noise, time synchronization, packet encoding, and real communication-channel contention require further validation.
Overall, the proposed framework provides a decentralized communication-scheduling architecture for the considered predecessor-following truck platoon without requiring global topology information. The present results should be interpreted within the scope of the five-vehicle simulation study and the assumed communication topology. The framework has not yet been validated through hardware-in-the-loop or public-road experiments. Future work will therefore focus on hardware-oriented implementation, communication delays and packet losses, model uncertainties, heterogeneous vehicle dynamics, larger platoons, and dynamically changing communication topologies.
Author Contributions
Conceptualization, Y.W., X.W. and S.G.; methodology, Y.W., X.W. and S.G.; validation, S.G.; writing—review and editing, Y.W., X.W. and S.G. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported in part by the Natural Science Research Project of Jiangsu Higher Education Institutions under Grant 25KJB120003; and in part by the National Natural Science Foundation of China under Grant 62603335.
Data Availability Statement
The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author
Conflicts of Interest
Author Yuanming Wang is employed by Shanxi Digital Government Construction and Operation Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Abbreviations
The following abbreviations are used in this manuscript:
| PPO | Proximal Policy Optimization |
| ETM | Event-Triggered Mechanism |
| RL | Reinforcement Learning |
| MIET | Minimum Inter-Event Time |
References
- Botelho, T.C.; Duarte, S.P.; Ferreira, M.C.; Ferreira, S.; Lobo, A. Simulator and On-Road Testing of Truck Platooning: A Systematic Review. Eur. Transp. Res. Rev. 2025, 17, 4. [Google Scholar] [CrossRef] [Scilit]
- Song, K.; Xiao, B.; Shi, L. Parameter Estimation of the Distributed Drive Mining Dump Truck Based on SH-AUKF. Electronics 2026, 15, 2113. [Google Scholar] [CrossRef] [Scilit]
- Olivari, E.; Carboni, A.; Caballini, C.; Pasquale, C.; Dalla Chiara, B.; Sacone, S. Planning and Control Strategies for Truck Platooning: A Benefit-Driven Literature Review. Future Transp. 2025, 5, 187. [Google Scholar] [CrossRef] [Scilit]
- Zou, H.; Ye, H.; Luo, W.; Zhou, X.; Wen, J. Event-Triggered and Adaptive ADMM-Based Distributed Model Predictive Control for Vehicle Platoon. Vehicles 2025, 7, 115. [Google Scholar] [CrossRef] [Scilit]
- Zhou, C.; Gong, J.; Ling, Q.; Liang, J. Event-Triggered Cruise Control of Connected Automated Vehicle Platoon Subject to Input Limitations. Machines 2024, 12, 866. [Google Scholar] [CrossRef] [Scilit]
- Narayanasamy, I.; Rajamanickam, V. A Cascaded Multi-Agent Reinforcement Learning-Based Resource Allocation for Cellular-V2X Vehicular Platooning Networks. Sensors 2024, 24, 5658. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Guo, S.; Meng, X. Optimal Resequencing of Connected and Autonomous Electric Vehicles in Battery SOC-Aware Platooning. IEEE Trans. Transp. Electrif. 2025, 11, 9298–9305. [Google Scholar] [CrossRef] [Scilit]
- Peng, C.; Guo, S.; Liu, M.; Xiao, L. Congestion-Aware Platoon Re-sequencing Optimization for Electric Vehicles Using Deep Reinforcement Learning. Neurocomputing 2026, 676, 133040. [Google Scholar] [CrossRef] [Scilit]
- Zheng, B.; Guo, S. Dynamic Platoon Re-Sequencing for Electric Vehicles Based on Bootstrapped DQN. Electronics 2025, 14, 4417. [Google Scholar] [CrossRef] [Scilit]
- Guo, S.; Meng, X.; Farasat, M. Energy Efficient and Battery SOC-aware Coordinated Control of Connected and Autonomous Electric Vehicles. In Proceedings of the 2022 American Control Conference (ACC), Atlanta, GA, USA, 8–10 June 2022; pp. 4707–4712. [Google Scholar] [CrossRef] [Scilit]
- Zhang, X.M.; Han, Q.L.; Ge, X.; Ding, D.; Ning, B.; Zhang, B.L. An Overview of Recent Advances in Event-Triggered Control. Sci. China Inf. Sci. 2025, 68, 161201. [Google Scholar] [CrossRef] [Scilit]
- Wu, J.; Lu, M.; Deng, F.; Chen, J. Robust Output Regulation of Linear Uncertain Systems by Dynamic Event-Triggered Output Feedback Control. IEEE Trans. Cybern. 2023, 53, 7333–7341. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ge, X.; Han, Q.L.; Ding, L.; Wang, Y.L.; Zhang, X.M. Dynamic Event-Triggered Distributed Coordination Control and its Applications: A Survey of Trends and Techniques. IEEE Trans. Syst. Man. Cybern. Syst. 2020, 50, 3112–3125. [Google Scholar] [CrossRef] [Scilit]
- Wang, Y.; Zhu, F. Distributed Dynamic Event-Triggered Control for Multi-Agent Systems with Quantization Communication. IEEE Trans. Circuits Syst. II Express Briefs 2024, 71, 2054–2058. [Google Scholar] [CrossRef] [Scilit]
- Ruan, X.; Feng, J.; Xu, C.; Wang, J. Observer-Based Dynamic Event-Triggered Strategies for Leader-Following Consensus of Multi-Agent Systems with Disturbances. IEEE Trans. Netw. Sci. Eng. 2020, 7, 3148–3158. [Google Scholar] [CrossRef] [Scilit]
- Guo, S.; Meng, X. Fully Distributed Control of Multi-agent Networks with Edge-Based Event-Triggered Communication. IEEE Trans. Autom. Control 2023, 68, 5630–5637. [Google Scholar] [CrossRef] [Scilit]
- Cheng, B.; Lv, Y.; Li, Z.; Duan, Z. Discrete Communication and Control Updating in Adaptive Dynamic Event-Triggered Consensus. IEEE Trans. Autom. Control 2024, 69, 347–354. [Google Scholar] [CrossRef] [Scilit]
- Li, X.; Peng, D.; Cao, J. Lyapunov Stability for Impulsive Systems via Event-Triggered Impulsive Control. IEEE Trans. Autom. Control 2020, 65, 4908–4913. [Google Scholar] [CrossRef] [Scilit]
- Nakayama, S.; Kobayashi, K.; Yamashita, Y. Lyapunov-Based Approach to Event-Triggered Control with Self-Triggered Sampling. Adv. Robot. 2024, 38, 603–609. [Google Scholar] [CrossRef] [Scilit]
- Xu, Z.; Ji, R.; Hu, Q.; Ge, S.S.; Li, D. Dual-Link Coded Event-Triggered Control for Nonlinear Multiagent Systems. IEEE Trans. Cybern. 2026, 56, 379–392. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Yu, F.; Yue, Y.; Lu, Q.; Zhang, B. A New Framework for Event-Triggered Consensus Control with Reinforcement Learning. In Proceedings of the 2025 37th Chinese Control and Decision Conference (CCDC), Xiamen, China, 16–19 May 2025; pp. 4218–4223. [Google Scholar] [CrossRef] [Scilit]
- Chen, J.; Meng, X.; Li, Z. Reinforcement Learning-based Event-Triggered Model Predictive Control for Autonomous Vehicle Path Following. In Proceedings of the 2022 American Control Conference (ACC), Atlanta, GA, USA, 8–10 June 2022; pp. 3342–3347. [Google Scholar] [CrossRef] [Scilit]
- Yu, H.; Dou, L.; Zhang, X.; Li, J.; Zong, Q. Safe Reinforcement Learning: Optimal Formation Control with Collision Avoidance of Multiple Satellite Systems. IEEE Trans. Cybern. 2025, 55, 447–459. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Baumann, D.; Zhu, J.J.; Martius, G.; Trimpe, S. Deep Reinforcement Learning for Event-Triggered Control. In Proceedings of the 2018 IEEE Conference on Decision and Control (CDC), Miami, FL, USA, 17–19 December 2018; pp. 943–950. [Google Scholar]
- De Persis, C.; Postoyan, R.; Tesi, P. Event-Triggered Control From Data. IEEE Trans. Autom. Control 2024, 69, 3780–3795. [Google Scholar] [CrossRef] [Scilit]
- Xu, T.; Sun, Z.; Wen, G.; Duan, Z. Data-Driven Dynamic Event-Triggered Control. IEEE Trans. Autom. Control 2024, 69, 8804–8811. [Google Scholar] [CrossRef] [Scilit]
- Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal Policy Optimization Algorithms. arXiv 2017, arXiv:1707.06347. [Google Scholar]
- Bhaskar, R.; Wahi, P.; Potluri, R. Platooning of Position-Controlled Vehicles on a Curved Path under Communication Losses and Delays. SAE Int. J. Connect. Autom. Veh. 2024, 7, 435–445. [Google Scholar] [CrossRef] [Scilit]
- Rajamani, R. Vehicle Dynamics and Control, 2nd ed.; Springer: New York, NY, USA, 2012. [Google Scholar] [CrossRef] [Scilit]
- de Witt, C.S.; Gupta, T.; Makoviichuk, D.; Makoviychuk, V.; Torr, P.H.S.; Sun, M.; Whiteson, S. Is Independent Learning all You Need in the StarCraft Multi-Agent Challenge? arXiv 2020, arXiv:2011.09533. [Google Scholar]
- Haarnoja, T.; Zhou, A.; Hartikainen, K.; Tucker, G.; Ha, S.; Tan, J.; Kumar, V.; Zhu, H.; Gupta, A.; Abbeel, P.; et al. Soft Actor-Critic Algorithms and Applications. arXiv 2019, arXiv:1812.05905. [Google Scholar]
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; Riedmiller, M. Playing Atari with Deep Reinforcement Learning. arXiv 2013, arXiv:1312.5602. [Google Scholar]
- van Hasselt, H.; Guez, A.; Silver, D. Deep Reinforcement Learning with Double Q-learning. arXiv 2015, arXiv:1509.06461. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.








