Control and Communication Co-Optimization Method with Handshake Frequency Hopping for Multi-AGVs

Abstract

In dynamic, high-interference industrial and logistics environments, multi-AGV cooperative tasks are often affected by communication delays and data loss, leading to information staleness and reduced control accuracy. Traditional handshake frequency hopping communication strategies introduce additional overhead in high-load environments, and channel selection strategies struggle to adapt to dynamic changes. To address challenges related to communication delay, task coordination, and real-time information exchange, we propose a control and communication co-optimization method based on a nonlinear Age of Information (AoI) penalty and an adaptive handshake frequency hopping mechanism. The method constructs a coupled control-communication model, designs an adaptive handshake period and multi-channel frequency hopping strategy to reduce channel conflicts, and introduces a nonlinear AoI penalty function that prioritizes the update of critical timely information, improving communication success rates and path control accuracy. Furthermore, by integrating the differential dynamics model, state estimation under communication delay and control error modeling, we propose a cooperative optimization algorithm for perception control and communication based on nonlinear AoI optimization (PPO-CCBNA). The algorithm achieves efficient solution based on approximate policy optimization (PPO). Simulation results demonstrate that PPO-CCBNA significantly outperforms benchmark algorithms in communication success rates, control stability, and energy efficiency, validating its effectiveness and feasibility in complex multi-AGV cooperative tasks.

1. Introduction

In modern industrial and intelligent logistics systems, multi-autonomous guided vehicles (AGVs) rely on stable wireless communication networks to enable dynamic environmental sensing, real-time information sharing, and precise collaborative control, which are foundational to ensuring production efficiency, system safety, and reliable operation. These capabilities are crucial for optimizing collaborative tasks, such as material handling, autonomous transport, and system-wide synchronization. However, in high-interference, dynamically changing environments, wireless communication links often suffer from signal attenuation, occlusion, and multipath effects. These environmental factors lead to communication delays, data loss, and information staleness, which severely degrade control accuracy, coordination efficiency, and the overall stability of collaborative operations [1,2]. As the number of devices increases and tasks grow more complex, these challenges become more pronounced, requiring more robust solutions to maintain system performance.

To enhance communication reliability and mitigate channel interference, handshake-based frequency hopping communication has become a widely adopted technique in modern wireless systems [3]. It disperses interference and reduces the probability of signal collision by dynamically switching between multiple preset communication channels, thereby maintaining a relatively stable low-latency communication connection in high-noise or signal-fading environments. However, this mechanism faces multiple challenges in practical deployment: frequent channel switching itself introduces additional communication overhead; the randomness or simple strategy of channel selection increases the uncertainty and complexity of the system state; especially in the scenarios with dense equipment and high communication load, its interference avoidance capability decreases, and it is difficult to optimize the end-to-end latency and the cooperative performance, so that the validity of the information faces a severe test [4,5,6].

How to improve the efficiency of handshake frequency hopping and guarantee the timeliness of information transmission is the key to break through the above problems and realize efficient collaboration. Traditional Age of Information (AoI)-based optimization methods reflect the timeliness of information in a system by measuring the time interval between its generation and its acceptance, but it is difficult to accurately portray the nonlinear negative impact of delay on system performance. To this end, this paper proposes a control and communication co-optimization framework based on nonlinear AoI optimization, taking the multi-autonomous guided vehicle (AGV) co-optimization scenario as an example. Specifically, this paper establishes a joint state-space model for control and communication coupling by coupling the differential dynamics model, the state estimation and communication hysteresis model, and the control error model, and designs the adaptive handshake period and the multichannel frequency hopping communication mechanism to enhance the system’s anti-interference ability and communication efficiency in the complex environment. And in order to accurately portray the nonlinear negative impact of delay on system performance, a nonlinear AoI metric mechanism is introduced to more accurately portray the nonlinear impact of communication delay on system performance. Therefore, this paper establishes a communication performance optimization problem based on perceptual control. Finally, to solve the problem, this paper proposes a cooperative optimization algorithm for perception control and communication based on nonlinear AoI optimization (PPO-CCBNA), which is used to solve the above problem and to achieve precise trajectory control perception and communication optimization of industrial control systems in dynamic environments.

The main contributions of this work are summarized below:

(1)

Integration of system models and communication mechanism: this work integrates the differential dynamics model, state estimation and communication hysteresis model, and control error model of AGVs to accurately capture the coupling between system motion control and communication performance. Based on this integration, an adaptive handshake period and multi-channel frequency hopping mechanism are developed to optimize communication resource allocation, enhance anti-jamming capability, and improve communication efficiency while being aware of control stability requirements.

(2)

Nonlinear AoI optimization for real-time control: to improve the real-time responsiveness and control accuracy of the system, this paper introduces a nonlinear AoI optimization mechanism. By incorporating nonlinear AoI into the optimization objective and designing a corresponding nonlinear AoI penalty function, the paper effectively models the nonlinear impact of delay on system performance. This approach encourages the agent to prioritize information updates in high-delay environments, thus mitigating the negative effects of information lag on control accuracy and overall collaboration.

(3)

POMDP-based algorithm for perceptual control and communication optimization: to address the non-convex minimization problem, this paper formulates it as a partially observable Markov decision process (POMDP) and proposes a cooperative optimization algorithm for perception control and communication based on nonlinear AoI optimization (PPO-CCBNA). Comparative experiments with a benchmark algorithm validate the superiority of the proposed approach in terms of control stability, communication efficiency, and energy efficiency, particularly in multi-agent collaborative tasks, where it demonstrates significant advantages.

The work that follows is organized as follows. Section 2 briefly describes the related work. Section 3 gives the system model and presents the joint optimization problem for control and communication performance. Section 4 gives the transformation procedure for the MDP problem. Section 5 presents the proposed PPO-CCBNA algorithm. Section 6 verifies the accuracy and good performance of the proposed algorithm through extensive comparative experiments. Finally, Section 7 summarizes the whole work.

2. Related Work

In this section, reviews and critically analyzes existing research related to task and resource scheduling in MEC for IIoT, emphasizing three key areas: co-design methods in IIoT, task and resource scheduling research in MEC for IIoT, and AoI in time-sensitive IIoT. The text highlights current achievements and identifies unresolved challenges, including complex task dependencies, delay propagation effects, and the accurate portrayal of information freshness. It also summarizes research gaps to motivate proposed solutions.

2.1. Wireless Network Transmission Technology

Frequency hopping communication technology is widely used in complex radio environments to avoid channel interference and improve the stability and reliability of communication. Frequency hopping communications are capable of dynamically switching between multiple channels, maintaining low latency and high efficiency by reducing interference and signal collisions. However, frequent channel switching may bring additional communication overheads and computational burdens, especially in multi-agent collaborative tasks, where uncertainty in channel selection may increase the complexity of the system and affect the effectiveness of the collaboration. Yang et al. [7] proposed an efficient cooperative-free channel rendezvous algorithm for heterogeneous cognitive radio networks. Lou et al. [8] systematically modeled and optimized AoI for the first time in a multichannel, multihop wireless network. Wang et al. [9] proposed a hopping time estimation method for the case of missing observations. Han et al. [10] proposed a novel SIC-enhanced multihop state updating mechanism. Wang et al. [11] proposed a very resource efficient OTA aggregation mechanism.

2.2. Age of Information Optimization

AoI is a key metric to measure the timeliness of information update in wireless communication, which is widely used in wireless networks, industrial control systems, and multi-agent collaborative tasks. Many existing studies use linear AoI optimization methods to reduce the impact of information lag on system performance. Zhang et al. [12] addressed the problem of timeliness analysis and optimization in multi-source state update system for IoT, and proposed a method to assess the system timeliness through the default probability of AoI and PAoI. Long et al. [13] proposed AoI-STO algorithm by optimizing the trajectory, sensing, and forwarding strategies of the UAVs. Zhang et al. [14] proposed an algorithm to minimize the energy consumption of AoI and UAVs through joint optimization of the data collection time, UAV trajectory, and time slot duration to minimize the energy consumption of AoI and UAV. Huang et al. [15] proposed a wireless information acquisition scheduling framework for AoI guarantees in response to the difficulty of satisfying strict AoI constraints with unknown channel reliability. Shi et al. [16] introduced AoI into ADS task scheduling for the first time and demonstrated that optimizing AoI can optimize RT and throughput at the same time. Lu et al. [17] proposed a joint design architecture of control and transmission based on full loop AoI. Liao et al. [18] proposed a joint sensing-communication-control optimization framework based on ultra-low AoI.

2.3. Co-Optimization of Wireless Network and Control Systems for Robot Clusters

With the development of industrial intelligence, collaborative optimization is increasingly used in wireless network systems for robot clusters. Most of the existing collaborative optimization methods focus on task allocation, path planning, and collaborative control. Wang et al. [19] designed a QoS-based co-design method for sensing, communication and control for UAV positioning system. Qiao et al. [20] designed a wireless communication and control co-optimization method based on resource allocation, communication scheduling, and control strategies in cloud-controlled AGV systems. Ma et al. [21] proposed a co-design method based on beamforming technique and control optimization for the communication interference and energy consumption problems in multi-loop wireless control systems. Girgis et al. [22] proposed a co-design method based on beamforming technique and control optimization for the cooperative control problems of multiple systems in wireless hybrid logic dynamic system species. Zhao et al. [23] proposed a communication and control co-design methodology to address the need for real-time control in future wireless networks. Han et al. [24] proposed a wireless communication and control co-design framework based on finite time wireless system identification. Xu et al. [25] developed a nonsingular predefined-time adaptive dynamic surface control method for quantized nonlinear systems. Sui et al. [26] proposed a finite-time adaptive fuzzy event-triggered consensus control scheme for high-order MIMO nonlinear multi-agent systems.

In summary, although some research progress has been made on the existing frequency hopping communication techniques, information timeliness optimization and cooperative optimization methods for wireless networks, there are still obvious deficiencies in dealing with communication delays, information lags and efficient cooperation. Existing frequency hopping mechanisms have the problem of computational burden caused by frequent channel switching in practical applications, the AoI optimization method fails to fully consider the nonlinear effect of communication delay, and the co-design scheme is difficult to balance the control accuracy and communication efficiency in the dynamic interference environment. In order to solve these problems, this paper designs a control and communication synergy method by combining nonlinear AoI optimization and adaptive handshaking mechanism so as to guarantee the real-time and stability requirements of the system.

3. System Modeling and Problem Formulation

3.1. System Model

In this paper, we consider an intelligent industrial system consisting of m AGVs as shown in Figure 1. All AGVs are distributed in a two-dimensional plane and are responsible for tasks such as cooperative handling and path tracking. In the process of task execution, AGVs interact with each other through frequency-hopping device-to-device (D2D) communication for status and task information to ensure cooperative operation and information synchronization among AGVs in the system. The communication system adopts a multi-channel frequency hopping mechanism, which can effectively improve the overall anti-interference capability of the system and enhance the utilization efficiency of resources, thus ensuring the efficient and stable operation of AGV in complex environments [27,28].

Figure 1. Multi-AGVs Intelligent Industrial System.

In time slot t, let $p_i\left(t\right)=\left(x_i\left(t\right),y_i\left(t\right)\right)$ denote the two-dimensional position of the i-AGV in time slot t, its position in the two-dimensional plane. Let ${\theta }_i\left(t\right)$ denote the facing angle of the AGV, $v_i\left(t\right)$ and ${\omega }_i\left(t\right)$ are denoted as the linear and angular velocities of the AGV, respectively. On this basis, the evolution of the dynamics equation of the AGV can be expressed as follows:

$$\left\{\begin{array}{cc}\hfill x_i(t+1)& =x_i\left(t\right)+v_i\left(t\right)cos{\theta }_i\left(t\right)\iota \hfill \\ \hfill y_i(t+1)& =y_i\left(t\right)+v_i\left(t\right)sin{\theta }_i\left(t\right)\iota \hfill \\ \hfill {\theta }_i(t+1)& ={\theta }_i\left(t\right)+{\omega }_i\left(t\right)\iota ,\hfill \end{array}\right.$$

(1)

where $\iota$ is the time step.

3.2. Communication Model

Let the set of available channels of the system be $\mathcal{F}=\{1,2,\cdots ,F\}$. The channels selected by the i-AGV and the j-AGV at the moment of time are denoted as $f_i\left(t\right)$ and $f_j\left(t\right)$, respectively, and only when then $f_i\left(t\right)=f_j\left(t\right)$, the i-AGV and the j-AGV can interact with each other in terms of information. Accordingly, the direction angle of joint AGV propagation can be established as a basis for judging the frequency hopping of inter-AGV handshake as [29,30] follows:

$${\delta }_{i,j}^{(f,t)}=1\left[\begin{array}{c}{f}_i\left(t\right)=f_j\left(t\right)\hfill \\ cos\left({\theta }_i\left(t\right)-{\varphi }_{i\to j}\left(t\right)\right)\ge cos(\Omega {\theta }_{max})\hfill \end{array}\right],$$

(2)

where, ${\varphi }_{i\to j}\left(t\right)$ denotes the direction angle from i-AGV to j-AGV, and $\Omega {\theta }_{max}$ is the directional judgment threshold. Only when $f_i\left(t\right)=f_j\left(t\right)$ and $cos\left({\theta }_i\left(t\right)-{\varphi }_{i\to j}\left(t\right)\right)\ge cos\left(\Omega {\theta }_{max}\right)$, do we have ${\delta }_{i,j}^{(f,t)}=1$, indicating that i-AGV and j-AGV successfully establish a handshake at time t.

Due to the small amount of data per communication packet, strict transmission delay and BER requirements in industrial control tasks, the traditional Shannon channel capacity, which assumes an infinitely long packet and an infinitesimal BER, is not applicable to the actual environment of this system. In order to truly reflect the communication performance under finite packet length and actual BER, this paper adopts the finite block length transmission model [31] in order to calculate the actual transmission rate of the i-AGV and the j-AGV at time slot t and channel f:

$$\begin{array}{c}\hfill R_{i,j}^{(f,t)}={\delta }_{i,j}^{(f,t)}·\left[B{log}_2\left(1+{\displaystyle \frac{p_{i,j}^{(f,t)}{g}_{i,j}^{(f,t)}}{{\displaystyle \sum _{k\ne i,f^{\prime }\in \mathcal{F}}}{p}_{k,j}^{(f^{\prime },t)}{g}_{k,j}^{(f^{\prime },t)}+{\sigma }^2}}\right)+\sqrt{{\displaystyle \frac{V_{i,j}^{(f,t)}}{n_f}}}{Q}^{-1}\left(ϵ\right)\right],\end{array}$$

(3)

where B is the channel bandwidth, $p_{i,j}^{(f,t)}$ is the transmit power, $g_{i,j}^{(f,t)}$ is the channel gain, ${\sigma }^2$ is the noise power, $n_f$ is the block length, $ϵ$ is the block BER, Q is the inverse of the Gaussian function and $V_{i,j}^{(f,t)}$ is the channel dispersion can be expressed as

$$V_{i,j}^{(f,t)}=1-{\displaystyle \frac{1}{{\left(1+{\displaystyle \frac{p_{i,j}^{(f,t)}{g}_{i,j}^{(f,t)}}{{\displaystyle \sum _{\begin{array}{c}k\ne i,f^{\prime }\in \mathcal{F}\end{array}}}{p}_{k,j}^{(f^{\prime },t)}{g}_{k,j}^{(f^{\prime },t)}+{\sigma }^2}}\right)}^2}}.$$

(4)

For the i-AGV and the j-AGV, the information transmission delay is calculated as

$$T_{i,j}^{(f,t)}={\delta }_{i,j}^{(f,t)}{\displaystyle \frac{L_{i,j}^{(f,t)}}{R_{i,j}^{(f,t)}}}+\varsigma ,$$

(5)

where $L_{i,j}^{(f,t)}$ is the packet size and $\varsigma >0$ is a very small positive number used to ensure that the denominator is always non-zero, and is introduced to enhance the robustness of the numerical calculations and maintain dimensional consistency.

The corresponding communication energy consumption can be calculated as

$$E_{i,j}^{(f,t)}=p_{i,j}^{(f,t)}{T}_{i,j}^{(f,t)}.$$

(6)

However, in high-density communication scenarios where the number of AGVs is much larger than the number of channels, it can lead to communication packet loss and message aging problems due to channel conflicts. For this reason, this paper introduces a time-division multiplexing mechanism as a complementary strategy in the system communication framework. When the system detects that channel resources are tight or conflicts are frequent, it switches to the time-division multiplexing mode and allocates communication time slots to different AGV communication pairs on the time axis, thus avoiding conflicts in the time domain and guaranteeing the timeliness and reliable transmission of mission-critical information.

3.3. Age of Information

Age of Information (AoI), as an important indicator of information timeliness in a system, has been widely used in various scenarios that require real-time data interaction, and is especially suitable for wireless communication and control systems that have high requirements for information timeliness [32,33]. It is used to characterize the time interval that elapses between the time a message is generated and the time it is received, thus reflecting the freshness of the information currently in use. Specifically, for a task, assuming its data sampling time is ${\tau }_t$, the Age of Information at the current moment t can be defined as

$${\Delta }_t=t-{\tau }_t.$$

(7)

The timeliness of the information is crucial for the co-optimization of trajectory control and information communication. It directly affects the coordination and cooperation among AGV clusters, and the effect of information staleness on control accuracy and system stability will show a trend of nonlinear exacerbation as the communication delay increases. Specifically, when communication delays are long, inaccurate information may lead to erroneous decisions, which may trigger path deviations, improper task execution, or even system failures. As a result, traditional AoI, while reflecting the staleness of the information, is unable to accurately portray the negative impact of latency on system performance in the presence of high latency.

In order to more accurately reflect the negative effects of delay, a nonlinear AoI is introduced. As shown in Figure 2, in high-latency scenarios, nonlinear AoI can more effectively direct the system to prioritize states that are affected by greater latency by imposing greater penalties on older information. This nonlinear penalty mechanism enables the system to identify and prioritize the correction of situations with severe information lag in a timely manner, avoiding AGV path deviation or communication failure due to inaccurate or outdated information. Specifically, nonlinear AoI improves the sensitivity of the system to communication delays by more strongly penalizing those states with higher information ages, thus effectively improving control accuracy and system stability. In systems with stringent requirements for high-precision control and efficient collaboration, nonlinear AoI is better adapted to these needs, ensuring that the system can make more accurate and coordinated decisions in the face of delays [34].

Figure 2. Comparison between Nonlinear AoI Function and Linear AoI Function.

In addition, the traditional definition of AoI only measures the time interval between when a message is generated from sampling and when it is received, and does not take into account the time overhead required for communication connection establishment (i.e., networking). However, in the system studied in this paper, the communication pairs between AGVs are not predefined, but the communication targets are dynamically decided after the generation of information packets, and the networking pairing is accomplished through the frequency-hopping handshake mechanism. Therefore, the whole process from information generation, communication pair determination, networking establishment, to successful data transmission is covered in the AoI considered in this paper.

3.4. Control Model

Since there is a communication delay for information synchronization between AGVs, we perform the computation of state estimation by introducing AoI, and the state estimation delay can be expressed as

$$\tau ={\Delta }_t·\iota ,$$

(8)

where ${\Delta }_t$ is the age of the information.

The current state estimate of the i-AGV is denoted as

$$\left\{\begin{array}{cc}\hfill {\widehat{x}}_i\left(t\right)& =x_i(t-\tau )+v_i(t-\tau )cos{\theta }_i(t-\tau )\tau ,\hfill \\ \hfill {\widehat{y}}_i\left(t\right)& =y_i(t-\tau )+v_i(t-\tau )sin{\theta }_i(t-\tau )\tau .\hfill \end{array}\right.$$

(9)

Define the trajectory error as

$${\tilde{\epsilon }}_i\left(t\right)=\left[\begin{array}{c}{x}_r^i\left(t\right)-{\widehat{x}}_i\left(t\right)\\ y_r^i\left(t\right)-{\widehat{y}}_i\left(t\right)\end{array}\right],$$

(10)

where $x_r^i\left(t\right)$ and $y_r^i\left(t\right)$ are reference trajectories.

Therefore, this design control law is

$$\left\{\begin{array}{cc}\hfill v_i\left(t\right)& =k_v·∥{\tilde{\epsilon }}_i\left(t\right)∥\hfill \\ \hfill {\omega }_i\left(t\right)& =k_{\omega }·\left(atan2(\tilde{y},\tilde{x})-{\theta }_i\left(t\right)\right),\hfill \end{array}\right.$$

(11)

where $k_v$ and $k_{\omega }$ are proportional gains.

To evaluate the control performance, the control stability function is introduced as

$${\kappa }_i\left(t\right)={\displaystyle \frac{1}{\alpha ·{∥{\tilde{\epsilon }}_i\left(t\right)∥}^2+\beta \left(v_i^2\left(t\right)+{\omega }_i^2\left(t\right)\right)+\chi }},$$

(12)

where $\alpha$ and $\beta$ are weighting factors and $\chi >0$ is a very small positive number to ensure that the denominator is bounded.

In order to improve the overall performance of the system, an adaptive handshake frequency adjustment function is introduced:

$$\begin{array}{c}\hfill {\eta }_i\left(t\right)={\left({\omega }_1∥v_i\left(t\right)∥+{\omega }_2∥x_i\left(t\right)-x_i(t-1)∥+{\omega }_{3}p\left({\Delta }_t\right)+{\omega }_4\left({\displaystyle \frac{1}{{\kappa }_i\left(t\right)}}\right)\right)}^{-1},\end{array}$$

(13)

where ${\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4$ are weighting factors.

Therefore, we can modify the handshake frequency hopping criterion to

$${\delta }_{i,j}^{(f,t)}=1\left[\begin{array}{c}{f}_i\left(t\right)=f_j\left(t\right)\hfill \\ c_i\left(t\right)\ge {\eta }_i\left(t\right)\hfill \\ cos\left({\theta }_i\left(t\right)-{\varphi }_{i\to j}\left(t\right)\right)\ge cos\left(\Omega {\theta }_{max}\right)\hfill \end{array}\right],$$

(14)

where $c_i\left(t\right)$ is the handshake frequency beat counter.

3.5. Description of the Problem

The focus of this paper is to minimize the system energy consumption, nonlinear AoI and maximize the system control stability under the constraints imposed on the system by considering the communication model and the control model. Therefore the following problem is constructed:

$$\mathrm{P}:min\sum _t\left[{\lambda }_1\sum _{i,j}{E}_{i,j}^{(f,t)}+{\lambda }_2\sum _{i,j}p\left({\Delta }_{i,j}^t\right)-{\lambda }_3\sum _i{\kappa }_i\left(t\right)\right]$$

(15)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& \mathrm{C}1:0\le p_{i,j}^{(f,t)}\le p_{max},\hfill \\ \hfill & \mathrm{C}2:{\eta }_i\left(t\right)\le {\eta }_{max},\hfill \\ \hfill & \mathrm{C}3:x_{min}\le x_i\left(t\right)\le x_{max},\hfill \\ \hfill & \mathrm{C}4:y_{min}\le y_i\left(t\right)\le y_{max},\hfill \\ \hfill & \mathrm{C}5:0\le v_i\left(t\right)\le v_{max},\hfill \\ \hfill & \mathrm{C}6:|{\omega }_i\left(t\right)|\le {\omega }_{max},\hfill \\ \hfill & \mathrm{C}7:0\le t\le t_{max},\hfill \end{array}$$

(16)

where ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ are the corresponding weight coefficients, all belonging to the range $[0,1]$, and ${\lambda }_1+{\lambda }_2+{\lambda }_3=1$. And constraint C1 is the maximum transmit power constraint of the AGV. C2 is the maximum adaptive handshake frequency regulation function constraint to ensure that it can be efficiently regulated within the constraint range. C3 and C4 are the positional boundary constraints of the AGV, and C5 and C6 are the velocity constraints of the AGV to ensure that it can perform its motions within feasible ranges and velocity requirements. C7 is the maximum handshake pairing period constraint.

4. Problem Formalization and Transformation

Since Problem $\mathcal{P}$ is a mixed integer nonlinear programming problem that is NP-hard. Therefore, it is necessary to transform the collaborative optimization into a POMDP problem and solve it using multi-agent deep reinforcement learning. Therefore, it is possible to construct an integrated decision model consisting of elements such as $\{S,A,Z,O,P,R\}$. Where $\mathcal{S}$ denotes the state space, $\mathcal{A}$ denotes the action space, $\mathcal{Ƶ}$ is the state transfer probability matrix, $\mathcal{O}$ is the observation space, $\mathcal{P}$ is the observation probability, and $\mathcal{R}$ is the reward function.

4.1. State Space $\mathcal{S}$

At each decision moment t the state space consists of AGV position coordinates, AGV facing angle, AGV velocity, channel state, and AoI, which is defined as

$$\begin{array}{cc}\hfill \mathcal{S}\left(t\right)& ={\left\{{\mathcal{S}}_i\left(t\right)\right\}}_{i=1}^M\hfill \\ \hfill & =\left\{\left\{x_1\left(t\right),y_1\left(t\right),{\theta }_1\left(t\right),v_1\left(t\right),{\omega }_1\left(t\right),f_1\left(t\right),{\Delta }_1\left(t\right)\right\},\dots \right.\hfill \\ \hfill & \left.\left\{x_m\left(t\right),y_m\left(t\right),{\theta }_m\left(t\right),v_m\left(t\right),{\omega }_m\left(t\right),f_m\left(t\right),{\Delta }_m\left(t\right)\right\}\right\},\hfill \end{array}$$

(17)

where ${\mathcal{S}}_i\left(t\right)$ is the state space of the i-AGV.

4.2. Action Space $\mathcal{A}$

At each decision moment t the state space consists of channel number selection and transmit power, defined as

$$\begin{array}{cc}\hfill \mathcal{A}\left(t\right)& ={\left\{{\mathcal{A}}_i\left(t\right)\right\}}_{i=1}^M=\left\{\left\{f_1\left(t\right),p_1\left(t\right)\right\},\dots ,\left\{f_m\left(t\right),p_m\left(t\right)\right\}\right\},\hfill \end{array}$$

(18)

where ${\mathcal{A}}_i\left(t\right)$ is the action space of the i-AGV.

4.3. State Transition Probability $\mathcal{Ƶ}$

At each decision period t, the state transfer probability ${\mathcal{Ƶ}}_i\left(t\right)$ describes the probability that ${\mathcal{S}}_i\left(t\right)$ transfers to ${\mathcal{S}}_i(t+1)$ when action ${\mathcal{A}}_i\left(t\right)$ is performed, and is denoted by ${\mathcal{Ƶ}}_i\left(t\right)\left({\mathcal{S}}_i(t+1);{\mathcal{S}}_i\left(t\right),{\mathcal{A}}_i\left(t\right)\right)$.

4.4. Observation Space $\mathcal{O}$

The observation environment is set as a partially observable environment, i.e., each agent can only access part of the state information, so the observation information is defined as

$$\begin{array}{cc}\hfill \mathcal{O}\left(t\right)& ={\left\{{\mathcal{O}}_i\left(t\right)\right\}}_{i=1}^M=\left\{\left\{{\tilde{x}}_1\left(t\right),{\tilde{y}}_1\left(t\right),g_1\left(t\right),{\Delta }_1\left(t\right)\right\},\dots ,\right.\left.\left\{{\tilde{x}}_m\left(t\right),{\tilde{y}}_m\left(t\right),g_m\left(t\right),{\Delta }_m\left(t\right)\right\}\right\},\hfill \end{array}$$

(19)

where ${\tilde{x}}_m\left(t\right)$ and ${\tilde{y}}_m\left(t\right)$ are position information with noise interference and $g_m\left(t\right)$ is the channel gain of the current communication link.

4.5. Observation Probability $\mathcal{P}$

Observation probability is the probability that agent observes a particular observation given the system state and action, defined as follows:

$${\mathcal{P}}_i\left(t\right)=Pr\left({\mathcal{O}}_i\left(t\right)\mid {\mathcal{S}}_i\left(t\right),{\mathcal{A}}_i(t-1)\right).$$

(20)

4.6. Reward $\mathcal{R}$

Based on the objective function given and considering matching failure, load balancing, and channel conflicts, the reward function is defined as

$$\begin{array}{cc}\hfill {\mathcal{R}}_i\left(t\right)=& {\lambda }_3{\kappa }_i\left(t\right)-{\lambda }_1{E}_{i,j}^{(f,t)}-{\lambda }_{2}p\left({\Delta }_{i,j}^t\right)-P_{\mathrm{fail},i}\left(t\right)-P_{\mathrm{balance},i}\left(t\right)-P_{\mathrm{collision},i}\left(t\right),\hfill \end{array}$$

(21)

where $P_{\mathrm{fail},i}\left(t\right)$ is the matching failure penalty, $P_{\mathrm{balance},i}\left(t\right)$ is the load imbalance penalty, and $P_{\mathrm{collision},i}\left(t\right)$ is the channel conflict penalty.

5. Collaborative Multi-Agent Control and Communication Algorithm Based on Nonlinear AoI Optimization

On the basis of the POMDP model for co-optimization of AGV network control and communication established above, in order to effectively solve the problems of state estimation delay, channel dynamic matching, and co-optimization between communication energy consumption and AoI for multi-AGV systems in industrial scenarios, this paper proposes a cooperative optimization algorithm for perception control and communication based on nonlinear AoI optimization (PPO-CCBNA), which is based on proximal policy optimization (PPO). The overall structure of the algorithm is shown in Figure 3. The algorithm adopts the framework of centralized training and distributed execution, fully combines the advantages of multi-intelligence body reinforcement learning and proximal policy optimization techniques, and utilizes the Actor–Critic architecture to achieve efficient optimization of policies. Specifically, each agent outputs its actions through a shared Actor network, which makes decisions on communication channel selection and transmit power control, respectively, while the Critic network is responsible for evaluating the value of the state in which the action is located, providing generalized advantage estimates (GAE) to enhance the stability of the training. In addition, the PPO-Clip technique is used to constrain the policy update amplitude, and gradient clipping is used to prevent gradient explosion, which guarantees the high efficiency and stability of the training process. At the same time, the algorithm introduces a strategy entropy term, which encourages diversification of actions, effectively prevents the strategy from falling into local optimality, and significantly improves the overall control accuracy, communication efficiency and stability.

Figure 3. Structure of PPO-CCBNA.

5.1. Algorithm Design

This algorithmic framework trains two networks for each agent: the Actor and the Critic. The Critic network learns to map the global state to its corresponding value function, providing evaluation for the current situation. Meanwhile, the Actor network is responsible for learning a probability distribution that maps local observations to possible actions, thereby enabling action selection through sampling. By adopting this design, the framework supports centralized training and distributed execution, allowing multiple agents to make both coordinated and autonomous decisions. The policy optimization is achieved by gathering trajectory data and utilizing it to iteratively update the network parameters. The detailed steps of the algorithm are illustrated in Algorithm 1.

Algorithm 1 PPO-CCBNA-Based Data Collection and Training

Input: Initialize $x_i\left(t\right),y_i\left(t\right),{\theta }_i\left(t\right),v_i\left(t\right),{\omega }_i\left(t\right),f_i\left(t\right)$,             ${\Delta }_i\left(t\right)$ of the system parameters; Output: ${\pi }_{\theta }^{\ast },V_{\varphi }^{\ast },f_i\left(t\right),p_i\left(t\right)$; 1: Initialize the parameter $\theta$ for policy model ${\pi }_{\theta }$ and the parameter $\varphi$ for value function $V_{\varphi }$ 2: for episode = 1, 2, ⋯ , E do 3:   set data buffer $D=\left\{\right\}$ 4:      for $w=1,2,\cdots ,W$ do 5:         Set empty list $\tau =\left\{\right\}$ 6:            for $t=1,2,\cdots ,T$ do 7:                  for agent $i=1,2,\cdots ,M$ do 8:                    $P_t^{\left(i\right)}={\pi }_{\theta }^{\left(i\right)}(a_t^{\left(i\right)}\mid o_t^{\left(i\right)})$ 9:                    $a_t^{\left(i\right)}\sim P_t^{\left(i\right)}$ 10:                  $V_t^{\left(i\right)}=V_{\varphi }\left(s_t^{\left(i\right)}\right)$ 11:                end for 12:              Execute action $a\left(t\right)\to R\left(t\right),s(t+1),o(t+1)$ 13:              $\tau +=\left\{s\right(t),o(t),a(t),R(t),s(t+1),o(t+1\left)\right\}$ 14:            end for 15:         Compute reward $\mathcal{R}$ on $\tau$ 16:         Compute advantage estimate $\widehat{A}$ via GAE on $\tau$ 17:         Compute ${\tilde{V}}_{\varphi }\left(s_t^{\left(i\right)}\right)$ with value-norm on $\tau$ 18:         Split trajectory $\tau$ into chunks of length $\mathrm{L}$ 19:            for l = 1, 2, ⋯, $\frac{T}{L}$ do 20:              $D=D\cup (\tau [l:l+T],\mathcal{R}[l:l+L],\widehat{A}[l:l+L])$ 21:            end for 22:      end for 23:      for mini-batch r = 1, 2, ⋯, R do 24:         $b\leftarrow$ random mini-batch from D with all agent data 25:            for each data chunk c in the mini-batch b do 26:              Update V 27:            end for 28:      end for 29:   Adam update $\theta$ on $\mathcal{L}\left(\theta \right)$ with data b 30:   Adam update $\varphi$ on $\mathcal{L}\left(\varphi \right)$ with data b 31: end for

Under the reinforcement learning framework, the agent continuously interacts with the environment to learn an optimal policy that maximizes cumulative rewards. To enhance the accuracy of Advantage function estimation and stabilize the policy optimization process, Generalized Advantage Estimation is adopted. The GAE is defined as follows:

$${\widehat{A}}_i\left(t\right)=\sum _{k=0}^{T-t-1}{\left(\gamma \lambda \right)}^k{\xi }_i(t+k),$$

(22)

where $\lambda$ is denoted as the attenuation factor of the GAE and ${\xi }_i\left(t\right)$ is the time error denoted as

$${\delta }_i\left(t\right)=R_i\left(t\right)+\gamma V_{\varphi }\left(s_i(t+1)\right)-V_{\varphi }\left(s_i\left(t\right)\right),$$

(23)

where $R_i\left(t\right)$ localizes the reward value, and $V_{\varphi }\left(s_i\left(t\right)\right)$ and $V_{\varphi }\left(s_i(t+1)\right)$ are the value functions of the current state $s_i\left(t\right)$ and the next moment state $s_i(t+1)$, respectively. Therefore, the objective value function can be estimated by generalized advantage estimation can be expressed as

$${\widehat{V}}_{\varphi }\left(s_i\left(t\right)\right)=V_{\varphi }\left(s_i\left(t\right)\right)+{\widehat{A}}_i\left(t\right).$$

(24)

In order to further reduce the estimation error in a multi-agent setting, the dominance estimation is extended to the entire population of agent by introducing a value function normalization technique. The value function ${\tilde{V}}_{\varphi }\left(s_i\left(t\right)\right)$ after the normalization process is

$${\tilde{V}}_{\varphi }\left(s_i\left(t\right)\right)={\displaystyle \frac{{\widehat{V}}_{\varphi }\left(s_i\left(t\right)\right)-\widehat{\mu }}{\widehat{\sigma }}},$$

(25)

$$\widehat{\mu }={\displaystyle \frac{1}{MT}}\sum _{i=1}^M\sum _{t=1}^T{\widehat{V}}_{\varphi }\left(s_i\left(t\right)\right),$$

(26)

$$\widehat{v}={\displaystyle \frac{1}{MT}}\sum _{i=1}^M\sum _{t=1}^T{\left({\widehat{V}}_{\varphi }\left(s_i\left(t\right)\right)\right)}^2,$$

(27)

where $\widehat{\sigma }=\sqrt{\widehat{v}-{\widehat{\mu }}^2}$ standard deviation, M denotes the number of agent, and T denotes the trajectory collection time. Accordingly, it can be obtained that the objective function of Actor network can be expressed as

$$\begin{array}{cc}\hfill \mathcal{L}\left(\theta \right)=& {\displaystyle \frac{1}{WMT}}\sum _{w=1}^W\sum _{i=1}^M\sum _{t=1}^T\hfill \\ \hfill & min\left(r_i\left(t\right){\widehat{A}}_i\left(t\right),\mathrm{clip}\left(r_i\left(t\right),1-\epsilon ,1+\epsilon \right){\widehat{A}}_i\left(t\right)\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{WMT}}\sum _{w=1}^W\sum _{i=1}^M\sum _{t=1}^TϵH\left({\pi }_{\theta }\right),\hfill \end{array}$$

where W denotes the batch size, $\epsilon$ is the coefficient of the clipping function, $ϵ$ is the coefficient of the strategy entropy regularization term, and $r_i\left(t\right)$ is the update strategy ratio, calculated as

$$r_{w,k}^{\left(i\right)}={\displaystyle \frac{{\pi }_{\theta }\left(a_i\left(t\right)\mid o_i\left(t\right)\right)}{{\pi }_{{\theta }_{\mathrm{old}}}\left(a_i\left(t\right)\mid o_i\left(t\right)\right),}}$$

(28)

where $H\left({\pi }_{\theta }\right)$ is the strategy entropy function, which can be expressed as

$$H\left({\pi }_{\theta }\right)={\mathbb{E}}_{a\sim {\pi }_{\theta }(·\mid o_i\left(t\right))}\left[-log{\pi }_{\theta }(a\mid o_i\left(t\right))\right].$$

(29)

Similarly, the objective function of the Critic network can be expressed as

$$\mathcal{L}\left(\varphi \right)={\displaystyle \frac{1}{WMT}}\sum _{w=1}^W\sum _{i=1}^M\sum _{t=1}^T{\left(V_{\varphi }\left(s_i\left(t\right)\right)-{\tilde{V}}_{\varphi }\left(s_i\left(t\right)\right)\right)}^2.$$

(30)

5.2. Algorithm Analysis

In the algorithm, the overall complexity of the algorithm needs to be calculated based on the complexity of each agent, since all agent are trained in parallel. Actor network consists of one input layer, one output layer and three hidden layers where the number of neurons in each layer is $d_{ai}$, $d_{ao}$ and $d_{ah}$ respectively. Therefore, the complexity of the Actor network can be calculated as ${\mathcal{O}}_{Actor}=\left(d_{ai}{d}_{ah}+2d_{ah}{d}_{ah}+d_{ah}{d}_{ao}\right)$. The same Critic network consists of one input layer, one output layer and three hidden layers, where the number of neurons in each layer is $d_{ci}$, $d_{co}$ and $d_{ch}$. Therefore, its computational complexity is ${\mathcal{O}}_{Critic}=\left(d_{ci}{d}_{ch}+2d_{ch}{d}_{ch}+d_{ch}\right)$. Let the training E set, each set contains T step and $d_{ah}=d_{ch}=d_h$. Therefore, the overall complexity of the algorithm can be obtained as $\mathcal{O}={\mathcal{O}}_{Actor}+{\mathcal{O}}_{Critic}=\left(ET{d}_h\left(d_{ai}+d{n}_h+d_{ao}+d_{ci}+1\right)\right)$. Therefore, it can be shown that the complexity of the algorithm is polynomial, indicating that it is executable.

6. Experimental Results

In this section, in order to evaluate the performance of the PPO-CCBNA algorithm, we build a simulation platform for a comprehensive evaluation.

6.1. Experimental Setup

(1) Experimental environment and parameter settings: For the PPO-CCBNA algorithm proposed in this paper, the specific parameters are set as follows: the number of neurons in the hidden layer of the Actor network and the Critic network are both set to 256, and the discount factor is 0.95. The learning rate of the two networks is uniformly set to 0.0005, and the cropping ratio in the PPO algorithm is set to 0.2 in order to improve the stability of the training and to prevent the strategy from updating too large. And in order to simulate the dynamic interference in industrial and logistic environments, a Rayleigh fading model is used to simulate the multipath propagation characteristics in such environments. Additionally, a random interference factor is introduced to simulate unpredictable disturbances in the environment, adding variability to the communication channel. This factor models interference that could arise from external sources or changing environmental conditions, which impacts the overall system performance. The hardware platform used for the experiments is Intel i9-14900ks processor with NVIDIA RTX 4080 SUPER 16 GB graphics card (manufactured by ASUS in Taiwan, China), and the software environment is Python 3.8.20 and PyTorch 2.1.2 + CUDA 12.1. The weight coefficients for the optimization problem are set as follows: ${\lambda }_1=0.3$ (weight for energy consumption), ${\lambda }_2=0.2$ (weight for nonlinear AoI penalty), and ${\lambda }_3=0.5$ (weight for system stability). These coefficients are chosen to balance the trade-off between energy efficiency, information freshness, and system stability, with a higher emphasis placed on stability in the system. Other key parameters are set in Table 1 [35,36].

Table 1. System Parameters.

Parameter	Value
Number of AGVs	10
Number of Channels	9
$P_{max}$	200 mW
${\sigma }^2$	${10}^{-11}$ mW
B	20 MHz
Path Loss Exponent	3
L	800 Byte
$n_f$	128
$\epsilon$	${10}^{-4}$

(2) Algorithms for comparison: In order to consider the fairness of the experiment, the following benchmark algorithm was set up for the control test.

• PPO-CCBNA: Joint optimization algorithm for handshake communication based on MAPPO algorithm with nonlinear AoI perception.
• TD3-CCBNA: Joint optimization algorithm for handshake communication based on MATD3 algorithm with nonlinear AoI sensing.
• DDPG-CCBNA: Joint optimization algorithm for handshake communication based on MADDPG algorithm with nonlinear AoI perception.
• SAC-CCBNA: Joint optimization algorithm for handshake communication based on ISAC algorithm with nonlinear AoI perception.
• PPO-CCBA: Joint optimization algorithm for handshake communication based on MAPPO algorithm with AoI awareness.

6.2. Algorithmic Performance Analysis

Figure 4 demonstrates the training reward variation curve, and the proposed PPO-CCBNA algorithm significantly outperforms the other algorithms in terms of reward values. Its reward curve is smooth and less fluctuating, showing high stability during strategy updates. Compared with the TD3-CCBNA, SAC-CCBNA and DDPG-CCBNA algorithms, the PPO-CCBNA algorithm is able to achieve higher convergence reward values in a shorter time, which suggests that the PPO-CCBNA algorithm possesses a more efficient ability to explore and exploit the optimal control and communication performance. By introducing a nonlinear AoI-aware mechanism, PPO-CCBNA is able to optimize the communication strategy between intelligences more efficiently, further enhancing the overall performance. Therefore, PPO-CCBNA demonstrates significant superiority over other algorithms.

Figure 4. Reward curves based on different strategies and algorithms.

Table 2 compares the algorithm training, loading, and execution delays of PPO-CCBNA, TD3-CCBNA, DDPG-CCBNA, and SAC-CCBNA under different agent counts. While PPO-CCBNA does not have the lowest training and loading delays compared to the other algorithms, these higher offline delays do not affect its online execution performance. The execution delay of PPO-CCBNA remains low and competitive, making it suitable for real-time applications. As the number of agents increases, the delays for all algorithms grow, but PPO-CCBNA continues to deliver efficient performance. These results suggest that PPO-CCBNA offers a good balance between training efficiency and low-latency execution, particularly when the number of agents scales up.

Table 2. Comparisons for Algorithmic Delay.

Agent	Algorithm	Training Delay (s)	Loading Delay (s)	Execution Delay (s)
10	PPO-CCBNA	1069.14	0.0852	0.0021
	TD3-CCBNA	6844.50	0.1309	0.0030
	DDPG-CCBNA	736.03	0.1052	0.0025
	SAC-CCBNA	1249.37	0.0209	0.0025
20	PPO-CCBNA	1947.82	0.0981	0.0024
	TD3-CCBNA	10,844.50	0.1505	0.0034
	DDPG-CCBNA	1252.52	0.1217	0.0028
	SAC-CCBNA	2471.03	0.0234	0.0028

Given that the industrial environment in which the AGV system is located has obvious dynamic change characteristics, and the state of each intelligent body, the mission objectives and the communication requirements will change in each round of mission execution, this paper adopts the information statistics results in terms of each round as the performance evaluation index. Based on the variation of energy consumption of different algorithms as shown in Figure 5, the energy consumption level of PPO-CCBNA algorithm is significantly lower than other algorithms, which is mainly due to the fact that the PPO-CCBNA algorithm adopts a more efficient mechanism of policy updating and exploring, which is combined with the clipping method to stabilize the optimization policy. Through this mechanism, PPO-CCBNA is able to effectively avoid redundant actions and unnecessary wastage of communication power, thus significantly reducing the overall energy consumption while ensuring performance.

Figure 5. Energy consumption under different strategies.

Figure 6 demonstrates the control stability variation curves based on different algorithms, and the PPO-CCBNA algorithm exhibits higher stability. This is mainly due to the efficient exploration of the state space and the effective estimation of the global value function of the PPO-CCBNA algorithm. Through this approach, PPO-CCBNA is able to predict the system behavior more accurately and control it effectively, thus avoiding drastic fluctuations in the strategy and ensuring the smoothness and stability of the control process.

Figure 6. Control stability under different strategies.

Figure 7a and Figure 7b show the nonlinear AoI penalty and AoI change curves based on different algorithms, respectively, and the PPO-CCBNA algorithm performs well in reducing AoI. Its excellent performance is mainly due to the introduction of the nonlinear AoI-aware mechanism, which imposes stronger penalties on higher AoI values, thus motivating the intelligences to be more active in data communication tasks and update information in a timely manner. Through this mechanism, PPO-CCBNA is able to effectively reduce the overall AoI value, ensuring the freshness of information and communication efficiency.

Figure 7. Nonlinear AoI Penalty and AoI Change. (a) Nonlinear AoI penalty as a function of system variables. (b) AoI dynamics across different scheduling strategies.

Figure 8a presents the handshake success index across various algorithms, while Figure 8b displays the channel conflict index. The handshake success index measures the proportion of successful handshake matches to the total number of attempted matches, whereas the channel conflict index quantifies how often agents select the same communication channel.The PPO-CCBNA algorithm achieves a notably higher handshake success rate compared to other methods. This improvement is mainly due to PPO-CCBNA’s enhanced ability to learn effective communication pairing strategies, particularly in making more precise handshake-related decisions. By integrating a reward signal for successful handshakes during policy optimization, PPO-CCBNA incentivizes agents to optimize their communication behaviors, leading to much greater pairing efficiency. Additionally, PPO-CCBNA demonstrates the lowest channel collision rate among all tested algorithms. This advantage arises from its superior capability to manage spectrum resource allocation among multiple agents through efficient action mapping and policy gradient updates. By fostering collaborative communication strategies and more accurate action selection, PPO-CCBNA significantly reduces communication interference and channel conflicts, resulting in enhanced overall communication efficiency.

Figure 8. Comparison of Handshake Success Rate and Conflict Rate. (a) The handshake success rate under different strategies. (b) Conflict rate dynamics reflecting scheduling congestion and coordination.

Figure 9 illustrates the impact of the number of AGVs on the average reward. As the number of AGVs increases, the average reward gradually decreases, indicating that higher AGV density leads to increased communication interference and scheduling complexity. Among all algorithms, the PPO–CCBNA model achieves the highest average reward and shows better stability compared with TD3–CCBNA, SAC–CCBNA, and DDPG–CCBNA.

Figure 9. Average rewards for different numbers of AGVs.

Figure 10a–d illustrate the differences in performance between nonlinear AoI-based optimization and linear AoI-based optimization. The results show that the PPO-CCBNA algorithm, which adopts a nonlinear AoI optimization approach, delivers superior outcomes compared to the linear AoI-based PPO-CCBA algorithm across several core indicators, including handshake success rate, channel collision rate, message freshness, and nonlinear AoI penalty. This performance boost is mainly attributed to the nonlinear AoI mechanism’s ability to impose stronger penalties on high AoI values, which in turn encourages agents to make more proactive and effective communication decisions. As a result, the PPO-CCBNA approach not only achieves higher handshake success rates and reduces channel collisions, but also significantly enhances the freshness of transmitted information throughout the system. Overall, these improvements mean that the proposed method offers clear advantages in terms of information transmission timeliness, stability, and the joint optimization of control and communication processes. In summary, the approach presented in this paper stands out in several key areas—including AoI management, handshake efficiency, channel resource allocation, and control stability—enabling rapid establishment of communication links, reducing overall communication delay, and enhancing the system’s real-time control capabilities.

Figure 10. Comparison across four performance indicators. (a) Handshake success rate under various strategies. (b) Conflict rate variation reflecting message collisions. (c) AoI dynamics across time. (d) Nonlinear AoI penalty considering delay impact. All metrics provide insight into control efficiency and information freshness.

Figure 11 demonstrates the performance comparison of three PPO-based algorithms under different strategies: the PPO-CCBNA (time-division multiplexing + adaptive frequency hopping), the TDM-PPO-CCBNA (time-division multiplexing only) and the FF-PPO-CCBNA (fixed frequency). The results show that PPO-CCBNA achieves the highest handshake success rate and the lowest channel collision rate, proving the effectiveness of combining time-division multiplexing with adaptive frequency hopping. In contrast, TDM-PPO-CCBNA and FF-PPO-CCBNA present higher collision rates with lower success rates, with FF-PPO-CCBNA performing the worst due to the lack of both time-division multiplexing and frequency hopping mechanisms. This highlights the significant advantages of adaptive frequency hopping technology in improving communication efficiency and reducing interference.

Figure 11. Performance evaluation of algorithms based on different complementary strategies. (a) Handshake Success Rate. (b) Conflict Rate.

Figure 12 compares the performance of the proposed PPO-CCBNA algorithm with two heuristic approaches: Greedy and RR+AoI-Trigger (Round Robin with AoI Trigger). The results demonstrate the effectiveness of the proposed learning-based approach in optimizing handshake success and reducing channel collisions compared to simpler, non-learning baselines.

Figure 12. Performance evaluation with different algorithms. (a) Handshake Success Rate. (b) Conflict Rate.

7. Conclusions

To tackle the challenges of communication delays, information lags, and efficient collaboration in industrial intelligent systems, this paper proposes a cooperative optimization algorithm for perception control and communication based on nonlinear AoI optimization (PPO-CCBNA). This algorithm effectively mitigates the issue of information lag by introducing a nonlinear AoI penalty mechanism, which reduces the negative impact of communication delays on both control accuracy and system stability. Additionally, by integrating a differential dynamics model, state estimation and communication lag models, and a control error model, the algorithm incorporates an adaptive handshake period and a multi-channel frequency hopping communication strategy. These innovations optimize communication efficiency, enhance the system’s resistance to interference, and improve overall communication performance. Experimental results show that the PPO-CCBNA algorithm significantly outperforms benchmark algorithms in terms of control stability, communication efficiency, and energy efficiency. Future research will focus on further improving the algorithm’s robustness and adaptability to better handle complex real-world scenarios, ultimately enhancing the cooperative efficiency and control precision of multi-agent systems.