Proximal Policy Optimization in 5G, B5G, and 6G Communication Systems: A Systematic Review

Abstract

Fifth-generation (5G), Beyond 5G (B5G), and sixth-generation (6G) wireless networks, along with the Internet of Things (IoT), are core communication infrastructure in smart cities. Their increased deployments create high-dimensional optimization and resource management challenges. Consequently, researchers have increasingly explored the use of Artificial Intelligence (AI) models for optimizing networks. The Proximal Policy Optimization (PPO) is one such algorithm that optimizes networks. This Systematic Literature Review (SLR) follows the PRISMA 2020 protocol to review 76 studies published between 2023 and 2026 to synthesize recent PPO-based approaches to optimize communication systems. This study examines key PPO variants in major communication domains. It outlines the primary obstacles to real-world deployment and provides a cross-domain classification. According to this study, PPO provides continuous action spaces with good training stability for AI models. Its stable policy-learning capabilities make it suitable for next-generation communication systems. However, sim-to-real transfer, reward design, and multi-agent scalability are a few key challenges encountered. Future directions emphasize robust, deployable PPO frameworks for 6G, IoT, and internet architecture.

1. Introduction

The rapid rollout of 5G networks is expected to more than double global mobile data traffic, reaching over 300 exabytes per month by 2030 [1]. At the same time, 5G networks will carry more than 80% of all mobile traffic, creating unprecedented demand for intelligent, adaptive communication management. Since real-time data transfer is crucial, technologies such as Artificial Intelligence (AI) and the Internet of Things (IoT) play a vital role [2]. IoT connectivity is a cornerstone of smart city infrastructure. Sustainable management of spectrum and energy resources directly affects urban efficiency and resilience. AI methods such as machine learning (ML) and deep learning (DL) [3] are increasingly used in wireless networks [4] and for spectrum management [5]. This digital transformation of network management can reduce energy consumption and infrastructure costs [6,7,8]. Recent PPO-based optimization studies reported measurable energy-efficiency improvements, including gains of 15.8% over DDPG and 34.2% over fractional programming baselines in RIS-assisted MU-MISO systems [9]. Specifically, reinforcement learning offers a dynamic framework for these methods to adapt to shifting network environments in real time [10]. Despite the growing use of deep reinforcement learning (DRL) in communication systems, existing review studies primarily examine either general DRL methods or isolated communication tasks such as routing, beamforming, or resource allocation. No recent review systematically synthesizes how Proximal Policy Optimization (PPO) supports energy-efficient and communication optimization across multiple domains in 5G, IoT, smart-city, and B5G/6G contexts.

To address the challenges of AI implementation in communication systems, this review focuses on Proximal Policy Optimization (PPO). PPO has emerged as a popular deep reinforcement learning algorithm used for high-dimensional, continuous-control optimization in modern communication networks. PPO is particularly relevant because it combines training stability with efficient handling of continuous decision variables such as transmit power, beamforming weights, routing priorities, and computation offloading. [11]. Its training stability often outweighs sample efficiency when compared to other algorithms [12,13]. Further, PPO has a remarkable ability to handle continuous action spaces inherent to wireless network problems, making it one of the most suitable approaches for many communication optimization problems [14].

The review integrates perspectives from various fields. The review examines PPO applications across wireless communications, edge computing, sensing, and control systems. Several recent reviews have examined reinforcement learning in wireless communication systems, DRL for resource allocation [15], or AI-enabled 6G networking [16]. However, these reviews either (i) treat PPO only briefly among many DRL algorithms, (ii) focus on a single application domain such as RIS, MEC, or routing, or (iii) do not compare PPO variants across multiple communication domains. In contrast, this review specifically synthesizes PPO variants across multiple communication domains.

The review covers communication, sensing, computation, and control domains within 5G, B5G, and 6G systems. Global 6G subscriptions are forecasted to reach 180 million by 2031 [17]. The first commercial deployments are expected around 2030 in countries such as the United States, China, Japan, South Korea, GCC nations, and India [18]. 6G is expected to support data rates up to 95 Gbit/s and eventually approach 1 Tbit/s with microsecond-level latency [19,20]. Applications of urban communication systems involving the deployment of IoT and edge devices are expected to be among the fastest-growing segments of the 6G ecosystem [17,21]. In particular, algorithms such as PPO can benefit in various areas in the communication networks domain.

Table 1. Comparison of Existing Review Articles and the Present Study.

Review	PPO Specific	Multi-Domain Coverage	Sustainability Analysis	PPO Variant Comparison	6G/B5G Focus	Cross-Domain Taxonomy	Key Limitation
Cheng et al. (2023)—DRL for Wireless Resource Allocation [15]	✗	Partial	✗	✗	Partial	✗	Primarily focused on resource allocation and broad DRL methods.
Jiao et al. (2024)—AI-Enabled 6G Networking Review [16]	✗	✓	Partial	✗	✓	✗	PPO was discussed only briefly among multiple AI techniques
Musaddiq et al. (2023)—RL-Based Routing and Spectrum Management Survey [22]	✗	✗	✗	✗	Partial	✗	Limited to routing and spectrum-access applications
Puspitasari and Lee (2023)—DRL for RIS and Beamforming Optimization Review [23]	✗	✗	Partial	✗	✓	✗	Domain-specific focus on RIS and beamforming
Ismail et al. (2025)-—–MEC and Computation Offloading Using DRL Survey [24]	✗	✗	✗	✗	Partial	✗	Focused exclusively on MEC and offloading systems
Hady et al. (2025)—Multi-Agent Reinforcement Learning for Wireless Networks Review [25]	Partial	Partial	✗	Partial	✓	✗	Limited comparison of PPO variants across domains
Amodu et al. (2025)—RL-Based UAV Communication Systems Survey [26]	✗	✗	Partial	✗	Partial	✗	Restricted to UAV-assisted communication networks
PPO in Intelligent Communication Systems (This Review)	✓	✓	✓	✓	✓	✓	Provides integrated cross-domain synthesis of PPO architectures and deployment challenges

demonstrates that existing review studies primarily focus either on broad, deep reinforcement learning techniques or on isolated communication domains, such as resource allocation, RIS optimization, MEC, routing, or UAV-assisted communication systems. In contrast, the present review specifically synthesizes PPO and its major variants across multiple communication domains while integrating sustainability considerations, 6G/B5G perspectives, and cross-domain system-level analysis within a unified taxonomy.

This review treats PPO applications as a single unified framework for intelligent communication systems and not as separate optimization problems. These systems operate across different 5G, B5G, and 6G environments and require collaboration across multiple layers spanning multiple domains. The analysis outlines persistent architectural trends within these systems, specifically in adaptive resource management, policy learning, multi-agent coordination, and sustainability-aware network orchestration. These trends frequently appear across several communication domains such as beamforming, MEC, satellite systems, and V2X networks. Identifying them as interconnected systems rather than isolated, domain-specific solutions enables systems developers to conduct a more in-depth systems-level study. This approach reveals deployment issues and optimization trade-offs that limit real-time deployment.

Novel Contributions of This Review

Although RL in wireless communications has attracted considerable scholarly interest, most current studies have focused on general categories of deep RL algorithms. In particular, the use of PPO for communication system optimization is understudied. This review goes beyond a mere cataloging of PPO applications. It presents a cross-domain taxonomy that links the communication task, the PPO variant used, the implemented reward structure, the targeted sustainability parameter, and the encountered deployment challenge. This analysis juxtaposes PPO, MAPPO, IPPO, DPPO, hybrid PPO-DQN, and PPO-Transformer designs across nine communication domains. Challenges such as the sim-to-real transfer gap, the underreported energy cost of training large PPO models, and the multi-agent scalability ceiling in ultra-dense 6G environments are explicitly identified and positioned as frontier problems that the next wave of research must confront.

2. Technical Background of PPO for Communication Systems

PPO provides a robust framework for autonomous decision-making in highly dynamic environments with continuous action spaces (e.g., power levels, beamforming vectors). PPO offers several advantages over many alternative deep reinforcement learning methods. For example, Trust Region Policy Optimization (TRPO) requires solving a constrained optimization problem via conjugate gradients. PPO achieves similar stability guarantees using only first-order optimization and is far simpler to implement. It provides stability in training, sample efficiency, and native support for continuous action spaces. Further, it can handle high-dimensional state spaces, which are common in dynamic network environments. Owing to these reasons, PPO is used as a foundational DRL algorithm for communication networks [27].

2.1. Applications of PPO in Communication Systems

Communication systems present unique challenges such as rapidly changing channel conditions, interference, and stringent quality of service requirements.

Table 2. Benefits of using PPO in Communications.

Feature	Benefit in Communications
Clipped Objective Function	Prevents large, destructive policy updates, ensuring stable learning even in volatile wireless environments [12,13].
Continuous Action Spaces	Unlike DQN, PPO can directly optimize continuous variables, such as transmit power and phase shifts [11,12].
Sample Efficiency	Allows for multiple epochs of updates on the same batch of data, reducing the need for massive real-time datasets [12,13].
Ease of Tuning	Generally, requires less hyperparameter tuning than algorithms like DDPG or SAC, making it more practical for deployment [13].
Energy Efficiency	PPO-based optimization improved energy efficiency by 15.8% over DDPG in RIS-assisted MU-MISO systems [9].

presents the key communication features and how PPO is particularly well-suited for these tasks.

Several highly cited communication-specific papers (e.g., the PPO-ISAC-THz paper with 84 citations) confirm that PPO has emerged as a widely adopted approach for complex mixed-integer and continuous optimization problems across next-generation wireless systems [28]. PPO applications span diverse fields, including satellite power management, network slicing, and drone trajectory planning. This widespread adoption demonstrates the algorithm’s flexibility as a primary framework for developing intelligent communication systems. Figure 1 illustrates the system-level operational framework of PPO in intelligent communication systems. The framework highlights how PPO-based agents continuously interact with dynamic communication environments by collecting real-time states, optimizing policies, taking adaptive control actions, and improving performance with feedback, thereby supporting adaptive resource allocation and energy-aware communication optimization in 5G/B5G/6G infrastructures.

2.2. Alternatives to PPO

Several alternative deep reinforcement learning algorithms were developed for use in communication networks [3]. Alongside PPO, other popular ones include Soft Actor-Critic (SAC), Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Advantage Actor-Critic (A3C), and Trust Region Policy Optimization (TRPO) [24]. This review specifically focuses on PPO because of its widespread adoption in recent 5G, B5G, and 6G communication studies. SAC generally provides higher sample efficiency through off-policy learning and entropy-based exploration, while TD3 and DDPG can achieve strong performance in continuous-control environments. However, these methods often require more careful hyperparameter tuning. Further, they exhibit greater training instability in highly dynamic wireless environments. PPO offers a practical balance between stability, implementation simplicity, computational efficiency, and continuous-action optimization. These features explain its growing popularity across communication domains, as discussed in Section 4. Therefore, this review focuses on PPO and its major variants (MAPPO, IPPO, DPPO, PPO-Transformer, and hybrid PPO architectures). We do acknowledge the importance of alternative DRL approaches as complementary research directions.

Table 3. Comparison of Proximal Policy Optimization (PPO) with Alternative Deep Reinforcement Learning Algorithms for Communication Systems.

Algorithm	Key Strength	Limitation	Typical Communication Use
SAC	High sample efficiency, entropy-driven exploration, and strong performance in continuous control	More hyperparameter tuning, higher implementation complexity	Power control, beamforming, dynamic spectrum management, UAV communications
TD3	Mitigates overestimation bias, improved stability over DDPG	Sensitive to parameter selection, increased architectural complexity	Resource allocation, RIS optimization, continuous control problems
DDPG	Efficient continuous-action optimization, lower interaction cost	Training instability, exploration challenges, prone to local optima	Power allocation, computation offloading, and beamforming
PPO	Stable training, easy implementation, strong continuous-action support, robust convergence	Lower sample efficiency than off-policy methods	Resource allocation, beamforming, network slicing, MEC, V2X, satellite communications
TRPO	Strong theoretical guarantees and monotonic policy improvement	Computationally expensive, difficult implementation due to constrained optimization	Early wireless resource management and continuous-control optimization
A3C	Parallel learning, reduced training time, simple architecture	Less stable than PPO, lower sample efficiency	Routing, congestion control, adaptive network management

gives a brief comparison of various algorithms.

3. Materials and Methods

The review methodology reports the databases, search strings, inclusion criteria, and PRISMA screening stages to ensure reproducibility.

3.1. Review Design and Research Questions

The study uses the Systematic Literature Review (SLR) approach, following the PRISMA 2020 guidelines [29]. It comprises the standard PRISMA checklist for core reporting requirements. The checklist is made available as a supplement to this document. The review protocol was not prospectively registered.

Objective of the Review: The main objective of this review is to synthesize recent applications of the Proximal Policy Optimization (PPO) algorithm across modern communication systems and networks. A secondary objective is to identify key limitations and research directions that will help shape the future research requirements for building next-generation sustainable networks.

Research Questions (RQs): Based on the research objectives, three research questions are framed to identify the current state of PPO applications in communication systems and to identify emerging future directions.

RQ1: What are the major application domains and recurring design patterns of PPO-based approaches in modern communication systems?

RQ2: What key limitations and emerging future research directions are shaping the development of sustainable PPO-based communication frameworks for 5G, B5G, and 6G networks?

RQ3: How do PPO-based communication frameworks contribute to sustainable digital infrastructure, smart cities, and IoT system design?

3.2. Search Strategy

Search Databases Used: Scopus (n = 108) and Web of Science (n = 475) are the primary databases used for the literature review search. Web of Science yielded a larger number of records because the Topic search covered multiple indexed collections and broader metadata fields than the Scopus search. Google Scholar (n = 57) is used to selectively identify recent high-quality publications from reputed publishers that are not yet indexed in the primary databases. Greater emphasis was placed on studies published through IEEE Xplore, Springer, MDPI, Wiley, Association for Computing Machinery (ACM), and ScienceDirect. To ensure consistency across databases, the same search keywords and Boolean operators were used throughout the literature search. In Scopus, the search was performed using the fields “Article Title, Abstract, and Keywords”. In Web of Science, the search was conducted using “Title, Abstract, and Author Keywords”. Google Scholar was also used as a supplementary search source to identify potentially relevant studies that may not have been indexed in the primary databases. Because Google Scholar does not offer the same structured field-selection capabilities as Scopus and Web of Science, searches were conducted using the same keyword combinations, and results were manually screened for relevance. Records retrieved from all databases were subsequently screened for duplicates and assessed against predefined inclusion and exclusion criteria.

Search Keywords and Strings: The searches were performed using the following Boolean keyword combinations:

(“Proximal Policy Optimization” OR “PPO”) AND (“wireless communication” OR “communication systems” OR “5G” OR “B5G” OR “6G”)

(“MAPPO” OR “IPPO”) AND (“resource allocation” OR “routing” OR “dynamic spectrum access” OR “beamforming”)

(“PPO”) AND (“RIS” OR “MEC” OR “ISAC” OR “UAV” OR “V2X” OR “satellite communications” OR “edge computing”)

(“deep reinforcement learning”) AND (“communication networks”) AND (“PPO”)

(“PPO”) AND (“network slicing” OR “computation offloading” OR “cognitive radio”)

(“PPO” OR “deep reinforcement learning”) AND (“energy efficiency” OR “smart city” OR “IoT” OR “sustainable network” OR “green communication”)

Search Timeframe & Search Date: The review considered studies published between 2023 and 2026 to capture the most recent developments in PPO-based communication systems. This timeframe is essential for a sustainability-focused review, given that many “green” technologies, such as RIS and ISAC, only reached maturity in these years. The literature search was conducted on 22 June 2026.

3.3. Study Selection Criteria

Inclusion Criteria: Selected studies (especially those retrieved from Google Scholar) that explicitly focused on PPO algorithms and PPO variants in the communications domain are included to supplement the previously obtained search results from the primary databases. We prioritized peer-reviewed journal articles and conference proceedings published between 2023 and 2026. Priority was given to publications that demonstrated empirical performance benchmarks, novel architectural designs, or significant conceptual contributions relevant to PPO-enabled communication systems.

Exclusion Criteria: Duplicate records common across multiple database searches are removed prior to screening. Similarly, publications that are not predominantly related to communications domains or that lack DRL methodologies are excluded. Works lacking sufficient methodological detail or technical detail regarding algorithmic implementation, or not providing sufficient empirical analysis or simulation result analysis, are omitted entirely. The exclusion criteria helped remove many publications and ensured the study focused on high-quality works only.

Language Restrictions: To ensure consistent interpretation of technical concepts, only publications written in the English language were included. The restriction helped maintain uniformity during the data extraction and synthesis phases of the review.

Document Types Considered: The search included archival journal articles, conference proceedings, and early-access articles from major databases. Book chapters are considered, but with lower priority, only to support theoretical or foundational aspects. Editorials and unpublished, non-peer-reviewed literature (such as preprints on arXiv and TechRxiv) are excluded to maintain high scientific rigor.

3.4. Screening Process (PRISMA Core)

Initial Retrieval Count: The initial search strategy yielded 640 records across the selected digital libraries and adequately covered recent developments in PPO.

Duplicate Removal Process: Spreadsheet software (Microsoft Excel (v2605, Microsoft 365)) helped quickly identify and remove 180 duplicate entries from the combined search results. Though tedious, manual verification is then conducted to ensure that no unique studies are accidentally discarded during cleaning.

Title and Abstract Screening: Reviewers screened the titles and abstracts individually to assess relevance to our research questions. Cohen’s kappa coefficient helped in statistically determining inter-rater reliability when agreement is reached. The title and abstract screening stage yielded a kappa value of 0.82, indicating strong agreement. Disagreements that arise are resolved through discussion and, where necessary, consultation with an additional reviewer. As many as 300 papers were excluded for lacking clear connections to communication systems or for failing to address optimization tasks at this stage.

Full-Text Eligibility Assessment: The remaining 160 articles underwent a rigorous full-text assessment to verify compliance with all inclusion criteria. Interrater agreement for the full-text eligibility stage was also evaluated using Cohen’s kappa, yielding a value of 0.86. The remaining disagreements were resolved through consensus. Then, 84 were excluded for non-compliance with the inclusion criteria. We evaluated methodological soundness and clarity of reported experimental results during this detailed examination phase. Of the 84 excluded studies, 38 were outside the (strict) scope of PPO-enabled communication systems, 26 lacked sufficient methodological detail, and 20 did not provide adequate empirical evaluation.

Final Study Count: The final selection yielded 76 qualified studies for the comprehensive qualitative synthesis analysis.

3.5. PRISMA Flow Diagram

The PRISMA flow diagram in Figure 2 visually explains the progression of the review from identification through screening to eligibility and inclusion. It follows the PRISMA 2020 study selection workflow. It presents the identification, screening, eligibility, and inclusion stages in a customized visual format that improves readability while preserving all required PRISMA reporting elements. It brings transparency into the study selection process and the reasons for excluding specific records. The initial retrieval count consisted of 108 records from Scopus and 475 records from Web of Science. A supplementary Google Scholar search yielded an additional 57 records. The combined set resulted in 640 records. Entries that are duplicates in the databases are identified using Microsoft Excel and removed. It eliminated 180 records, leaving 460 unique studies for screening. Title and abstract screening then excluded 300 papers because they lacked a clear connection to communication systems, PPO, or deep reinforcement learning. Consequently, 160 articles remained for full-text eligibility assessment. During this stage, 84 studies were excluded. Specifically, 38 studies were outside the review scope, 26 lacked sufficient methodological detail, and 20 did not provide adequate empirical evaluation. The final curated dataset consisted of 76 peer-reviewed studies used for qualitative synthesis and thematic analysis.

3.6. Data Extraction and Analysis

Data Extraction Criteria: The review synthesizes performance trends and identifies recurring design patterns in PPO-based communication frameworks. From each study, specific algorithmic details, such as the PPO variant and the associated hyperparameters, are collected. Data extraction was conducted independently by the reviewers using a structured extraction framework. Disagreements, if any, between the authors were resolved through discussion and consensus. Structured data extraction is used to collect data uniformly across studies. Appendix A provides the complete list of studies in which PPO, MAPPO, IPPO, DPPO, or another named PPO variant was demonstrably implemented or analyzed as the primary or comparator algorithm. This is the qualitative-synthesis set used for the quantitative and architectural comparisons in Section 4 and Section 5. A separate set of contextual references, including survey and review articles on broader DRL/MARL studies that do not isolate PPO as the evaluated algorithm, and perspective papers on sustainability, governance, or deployment, is cited throughout the narrative text. These are added to motivate the choice of DRL/PPO and to help position it against alternative RL approaches. It also supports claims about system architecture and deployment context. These contextual references are listed in the main reference list. However, they are excluded from Appendix A, since they do not strictly meet the PPO-implementation criterion used for the structured comparison.

Classification Framework/Taxonomy: The selected studies are divided into major application domains in contemporary wireless communication systems. The main groups are resource allocation, intelligent routing, beamforming, and multi-access edge computing.

Quality Assessment and Risk of Bias: The included studies were qualitatively assessed based on methodological clarity, reproducibility, empirical evaluation, and completeness of the reported experimental settings. Because the reviewed studies primarily consisted of engineering simulation studies rather than clinical trials, formal medical risk-of-bias instruments were not directly applicable. Potential reporting bias was minimized through multi-database searching and predefined inclusion criteria.

Certainty/Confidence Statement: Due to substantial heterogeneity across datasets, simulation environments, evaluation metrics, and communication scenarios, a formal certainty-of-evidence assessment was not performed.

Synthesis Approach: The retained studies are synthesized using thematic analysis. The thematic coding process used four predefined categories derived from the research questions:

(i)

Communication application domain;

(ii)

PPO variant and architectural design;

(iii)

Primary system benefit;

(iv)

Recurring technical limitation or future research direction.

Additional subthemes were identified inductively during full-text review. In this, publications are grouped by communication domain, PPO variant, sustainability contribution, and recurring technical challenges. The insights are derived through comparative analysis of empirical performance benchmarks across the selected studies. The review further identified recurring design patterns, including hybrid PPO architectures and centralized training with decentralized execution. Studies were coded independently into these categories, after which recurrent patterns were grouped into the thematic domains presented in Section 4.

Data Availability Statement: No new datasets were generated during the current study. The analyzed studies are available through the cited references.

4. Results and Discussion

Studies confirmed the feasibility of Multi-agent PPO (MAPPO) in real-world 5G systems, proving its effectiveness when combined with domain-aware observation design and sustainable network management goals [30,31]. Spectrum scarcity is worsening due to surging wireless traffic. In the upcoming years, it is anticipated that the total amount of mobile data traffic worldwide, excluding Fixed Wireless Access (FWA), will increase dramatically. It is projected to increase by approximately 2.3 times, reaching nearly 280 exabytes per month by 2030 [32]. This rapid growth is driven by rising demand for data-intensive applications such as video streaming, IoT services, and immersive communication technologies. Hence, there is an urgent need to address Dynamic Spectrum Access (DSA) in Cognitive Radio Networks (CRNs) to enhance spectrum efficiency. Most prior DSA approaches assume homogeneous Secondary Users (SUs). Deep reinforcement learning can help in jointly optimizing sensing and access policies under partial observations [33]. These approaches learn coordinated strategies without requiring prior knowledge of channel state statistics [34].

Fully decentralized multi-agent reinforcement learning frameworks effectively capture the distributed nature of participating stations [35]. One-bit gradient quantization reduces communication costs by more than an order of magnitude. These methods achieve performance comparable to centralized training frameworks while preserving complete decentralization. Further, prior DSA approaches ignored real-world heterogeneity. A Multi-agent PPO (MAPPO) can handle agent heterogeneity without parameter sharing. Further improvements made to the MAPPO helped achieve higher average reward (throughput) and provide efficient, stable, and fair spectrum access [36].

Researchers compared various PPO variants for use in the communications domain. For example, in a study, Independent PPO (IPPO), MAPPO, and a MAPPO + Transformer variant (called DET-FAIR-WINGS) are compared [30]. The study demonstrated that MAPPO-based approaches outperform IPPO in terms of convergence speed, packet loss, and latency. Several studies went beyond routing and spectrum access. PPO-based methods are applied to congestion control mechanisms such as Explicit Congestion Notification (ECN) [37]. ECN is a mechanism in computer networking that allows routers or switches to signal impending congestion to end hosts without dropping packets. The idea is to improve network performance by enabling early congestion signaling. Doing this can reduce the need for packet loss as a congestion-control signal, thereby helping in dynamic data-center communication environments. The existing static ECN thresholds are ineffective and inadequate for handling many-to-one (in-cast) traffic and heterogeneous flow types (mice vs. elephant flows). Combining offline and online training to adapt is found to benefit in dynamic network conditions. This was made possible with a multi-agent Independent Proximal Policy Optimization (IPPO) algorithm [37].

Studies extended PPO beyond its conventional use by embedding spatiotemporal reasoning directly into the policy network. This enables more intelligent, adaptive, and QoS-aware routing decisions in highly dynamic tactical networks [27]. Furthermore, such enhancements enable autonomous learning to adapt to dynamic traffic conditions. It also supports finer-grained routing control, thereby effectively reducing congestion. PPO optimizes probabilistic routing decisions across multiple nodes, thereby improving the routing approach used in communication devices. For example, it can integrate well with the actor-critic architecture, using Graph Attention Networks (GAT) to handle the complexities of network topologies [38]. It can have autonomous learning capabilities to adapt to dynamic traffic conditions and support finer-grained routing control. Eventually, it can effectively reduce congestion and resource wastage [39].

Recent work extends PPO to large-scale, high-mobility vehicular networks [40,41]. These environments require joint optimization of sensing, communication, and computing under dynamic conditions—similarly, cross-layer multi-agent PPO frameworks used dynamic Time Division Duplexing for enhanced resource allocation [42]. Centralized training with distributed execution optimizes uplink and downlink resource allocation while incorporating conditional activation. Simulation results demonstrate an 18% increase in throughput and a 16% reduction in interference.

Recent studies extend this approach by integrating sensing and communication within a unified learning framework [43]. Energy-efficient DRL-based beamforming has been explored for V2X scenarios under dynamic and uncertain channel conditions. PPO-driven optimization significantly reduces the average AoI compared to existing baseline approaches, thereby effectively balancing sustainable energy harvesting with data freshness [44,45].

Similarly, challenging dynamic environments are found in maritime wireless networks [46]. Devices on navy ships, in shipping, and on drones keep moving, and their channels change quickly [33,47]. These platforms also operate under strict power and bandwidth constraints. Therefore, traditional methods for assigning these resources will not be effective in such unpredictable environments. Adaptive algorithms adjust to dynamic channel conditions to improve reliability and throughput across mobile nodes. These characteristics make maritime networks a natural application domain for adaptive RL algorithms such as PPO [48]. Adaptive PPO-based resource allocation can reduce unnecessary power transmission and improve bandwidth utilization in maritime communication systems [49]. Meta-learning techniques can improve PPO adaptation in dynamic sea conditions. Domain knowledge can also help PPO perform better with limited data.

Table 4. Taxonomy of PPO-Based Applications in Intelligent Communication Systems.

Communication Domain	Typical PPO Variant	Key Mechanism/Outcome
Resource Allocation	PPO, MAPPO, PPO-Transformer	Reduces unnecessary power transmission and enables greener network operation [35,50].
Beamforming & RIS	MAPPO, IPPO	Hardware-efficient signal enhancement lowers overall energy requirements [9,28,51].
MEC & Offloading	PPO, DPPO	Balances local energy use with transmission latency for improved trade-offs [52,53,54].
Vehicular Edge Computing & Task Offloading	PPO, MAPPO	Joint optimization of computation offloading, resource allocation, latency reduction, and edge-resource utilization
Integrated Sensing (ISAC)	PPO, MAPPO, PPO-DQN	Reduces the need for dedicated sensing hardware in smart city infrastructure [55,56,57].
UAV Communications	PPO, Dual-Clip MAPPO	Energy management extends flight time and balances secrecy with efficiency [51,58].
Network Slicing	PPO, MAPPO	Enables adaptive multi-service resource orchestration [59,60,61].
Satellite Communications	PPO, PPO-DQN Hybrid	Reduces the energy cost of providing connectivity to remote regions [62,63,64].
V2X & Cognitive Radio	PPO, MAPPO	Reduces idle time, device battery drain, and overall transportation congestion [40,65,66].

presents a taxonomy of PPO applications in communication systems based on the communication domain, PPO variant, and the corresponding optimization mechanism or outcome. Additional discussions regarding optimization objectives, sustainability considerations, and deployment challenges are provided in the subsequent sections.

4.1. Resource Allocation and Management

Resource allocation remains a central optimization challenge in next-generation networks [8]. Dynamic resource allocation is perhaps the most prominent use case with several active application areas in wireless systems. Addressing scalability challenges in dense 5G network planning, recent investigations introduce a hierarchical multi-agent PPO framework integrated with representation learning [67]. This architecture effectively decouples base station deployment from antenna parameter tuning while leveraging UNet mechanisms to process large-scale spatial information. Empirical evaluations demonstrate significant gains in coverage rates and average throughput compared to conventional multi-agent baselines.

Several studies reported PPO outperforming heuristic algorithms across a range of user loads and channel conditions when jointly optimizing transmit power, bandwidth, and scheduling in heterogeneous networks (HetNets) [50,68]. In Open Radio Access Networks (O-RAN), PPO (on-policy) has been compared with ACER (off-policy) for QoS-sensitive resource allocation, with PPO demonstrating a favorable balance between energy consumption and user latency [69]. O-RAN uses a disaggregated architecture. In this setup, different xApps manage various RAN functions, such as power control and resource block allocation. These xApps need to coordinate their decisions without causing interference. A Digital Twin-enabled HAPPO framework successfully synchronizes them without requiring direct communication between the xApps [70]. The concepts can be extended this to energy sustainability. For example, a hierarchical rApp-xApp PPO framework has demonstrated that jointly optimizing radio unit activation schedules and user association weights [71]. Such system substantially reduces O-RAN operational power consumption making it ideal for 6G deployments. In adversarial scenarios, researchers proposed a new jamming estimator that uses deep hierarchical PPO and is integrated into the MEC-O-RAN framework [72]. This estimator models the ON-OFF jammer’s behavior as a partially observable Markov process. It predicts which time slots will be jammed. This allows a Transformer-based task scheduler to work only on safe slots. As a result, the cumulative prediction error is only 13 time slots per 100, compared to 48 for standard DDQN. It also reduces task execution time from 1260 s to 316 s for 50 active users. By using self-attention to map user sequences to resource sequences, a Transformer-based PPO framework in multiuser cellular systems provides greater spectrum efficiency and user fairness [73]. In addition to these approaches, current research has investigated hybrid reinforcement learning strategies for handling multi-objective optimization issues. A hybrid reinforcement learning framework that combines PPO and deep Q-networks has been presented to increase exploration efficiency and policy stability in dynamic control contexts [74]. These techniques allow simultaneous optimization of conflicting goals, such as energy sustainability, system stability, and performance efficiency, by using multi-objective reward structures [75]. These results demonstrate the growing importance of incentive design and hybrid learning architectures for solving complex, practical optimization problems in control and communication systems.

Effective coordination between dispersed stations is enabled by quantized gradient updates that do not require exchanging local data [35]. Independent PPO successfully operates within both decentralized training and decentralized execution paradigms. Communication overhead is reduced substantially during agent training while maintaining competitive throughput levels. In UAV base station networks, a hierarchical PPO architecture decomposes a complex multi-channel allocation problem from exponential to linear complexity, achieving superior throughput and convergence.

Adaptive Bitrate (ABR) Video Streaming is a key example of QoE-aware resource management [76]. Providing high Quality of Experience (QoE) for video streaming over dynamic 5G networks is a challenging sequential decision problem well-suited to PPO. PPO-ABR is a system that addresses the lag between behavior and target policies in Asynchronous Advantage Actor-Critic (A3C) methods by using PPO’s on-policy multi-agent DRL framework [77]. A related approach combines behavior cloning with PPO to accelerate convergence while maintaining stable bitrate selection under fluctuating network conditions. The method demonstrates improved QoE compared to benchmark schemes by using PPO’s controlled policy update mechanism to adapt reliably to 5G network dynamics [78]. By maximizing sample efficiency through clipped probability ratios across multiple epochs of minibatch updates, PPO-ABR outperforms state-of-the-art ABR methods across different QoE metrics on real network traces.

4.2. Adaptive Sensing and Beamforming Systems

Reconfigurable Intelligent Surfaces (RIS) technology reduces active transmission power requirements, making it a hardware-efficient and energy-sustainable approach to signal enhancement in dense urban environments. Hence, RIS hardware-efficient signal enhancement becomes a sustainable alternative to power-hungry active transmitters. PPO can continuously refine beamforming policies as channel conditions evolve [79]. The clipped-policy update mechanism helps prevent unstable beamforming decisions during training [80,81]. So, PPO is attractive for real-time beam management in highly dynamic wireless networks. Recent research explored PPO for RIS, in which the agent optimizes the surface’s reflection coefficients to enhance signal quality. Further advancements have extended metasurface technology to stacked intelligent surfaces, enabling the effective use of enhanced PPO algorithms [51]. This approach effectively manages secure mobile communications. It helps in jointly optimizing power and phase shifts in dynamic environments. These methods achieve significant improvements in achievable secrecy rates compared to conventional DRL baselines.

PPO has become a leading solver for non-convex beamforming and RIS phase-shift optimization problems [51]. In an IRS-aided Integrated Sensing and Communications (ISAC) system in the THz band, a primal-dual PPO framework jointly optimizes continuous transmit beamforming (actor) and discrete phase-shift design (critic), with a distributed PPO (DPPO) variant for multiuser MIMO scenarios [82]. For RIS-assisted SWIPT with Rate Splitting Multiple Access (RSMA), a PPO-based approach simultaneously optimizes beamforming vectors, power splitting ratios, and discrete phase shifts. Simultaneous Wireless Information and Power Transfer (SWIPT) enables devices to receive both energy and data from the same wireless signal. PPO-optimization helped achieve energy efficiency close to that of successive convex approximation (SCA) methods while requiring substantially less runtime in time-varying channels [28]. Similarly, in an RIS-assisted MU-MISO system, the PPO algorithm outperforms fractional programming (FP) by 34.2% and DDPG by 15.8% in energy efficiency. PPO also proves effective for statistical CSI-based beamforming, significantly reducing channel estimation overhead compared to instantaneous CSI methods [9]. Such reductions in estimation overhead and transmission power directly lower the carbon footprint of dense urban network deployments.

4.3. Communication–Computation Co-Design

For IoT and vehicular networks, PPO can help decide whether a task should be processed locally or offloaded to an edge server [83]. MEC enables devices to offload compute-intensive tasks to nearby edge servers, recognizing that computation offloading is a central resource-scheduling problem [24]. Similarly, blockchain-enabled digital twin architectures have been explored to improve interoperability, trust management, and system coordination in distributed IoT and edge-computing environments [84]. Similar integrations are possible in edge intelligence and digital twin frameworks for IoV systems [85]. In such systems, RL enables reliable, energy-efficient coordination between communication and computing resources. Within these sustainable computing frameworks, PPO-based multi-agent reinforcement learning effectively solves resource-allocation problems while balancing delay, energy constraints, and energy-aware operations. These choices emphasize PPO’s role in coordinating communication channels and allocating processing resources to achieve the best possible system performance. Recent work extends this paradigm by simultaneously optimizing inference execution and communication decisions in dispersed edge settings. To jointly optimize DNN partitioning, uplink channel allocation, and transmission power regulation under shared wireless restrictions, a MAPPO framework has been suggested [52]. By adopting centralized training with decentralized execution and a multi-head critic design, the approach improves coordination among edge devices while maintaining scalability in large hybrid action spaces [86]. Experimental results demonstrate improved trade-offs between latency and energy. It highlights the effectiveness of PPO in tightly coupled communication–computation optimization scenarios within IoT edge systems [39,53].

The system adjusts to the current network load and server availability while balancing the trade-off between transmission latency and local energy consumption. Task deadlines, time-varying channels, and heterogeneous devices are common in MEC environments. It makes computation-offloading decisions challenging optimization problems [24]. Further, these situations are dynamic and stochastic, which makes PPO a perfect fit [87,88]. The problem is modeled as an MDP by a Device-to-Device (D2D)-assisted compute offloading approach based on PPO. Compared with three benchmarked techniques for 5G and beyond networks, it achieves superior latency and energy convergence by concurrently optimizing offloading decisions and resource allocation.

Task offloading has emerged as a critical optimization problem in vehicular edge computing (VEC) environments, and PPO has been increasingly used for this application [89]. In these, the vehicles, based on the computational overhead, dynamically decide on which of the following the tasks should be:

• Executed locally;
• Offloaded to nearby edge servers through Vehicle-to-Infrastructure (V2I) communication;
• Delegated to neighboring vehicles through Vehicle-to-Vehicle (V2V) communication.

Vehicular networks are highly dynamic. Added to their mobility, there will be fluctuating wireless channel conditions and strict latency requirements. These situations make task-offloading decisions particularly challenging. Recent studies have demonstrated the effectiveness of PPO-based RL frameworks for addressing these challenges. Researchers proposed a PPO-based joint task-offloading and resource-allocation framework for vehicular edge computing [90]. The framework jointly optimizes communication and computational resources across V2I and V2V links, improving service latency and resource utilization. Similarly, PPO-based DRL approaches are used to deal with computation offloading in vehicular communications [54]. It demonstrated that adaptive policy learning can significantly improve task completion efficiency under dynamic traffic and network conditions. These studies further confirm PPO’s suitability for communication–computation co-design problems involving continuous decision spaces, real-time adaptation requirements, and heterogeneous edge resources.

4.4. Integrated Sensing and Communication (ISAC)

In conventional communication networks, sensing (radar, localization, imaging) and communication (data transfer) are treated as separate processes that use different frequency bands. ISAC sees 6G waveforms as intrinsically containing both information bits and environmental-scattering information, thereby unifying both roles. [55]. ISAC is therefore a crucial component of 6G, as it reduces material resource consumption by eliminating the need for dedicated sensing hardware. In this case, each broadcast signal serves as both a radar probe and a data carrier. Due to growing spectrum scarcity, increased device connectivity, and its integration into an AI-ready network architecture, 6G prioritizes ISAC over 5G. [91,92]. AI-native 6G systems are expected to integrate sensing and computing capabilities directly into the network architecture, making ISAC a natural design paradigm. Such systems follow a layered approach [93], reduce the need for separate sensing hardware and dedicated spectrum resources, and thereby lower overall material use [56]. Improved spectral efficiency may reduce infrastructure expansion requirements and associated deployment overhead in dense urban networks [8,94].

PPO supports the joint integration of sensing, computation, and control [95]. PPO-based approaches manage shared resources while preserving both sensing accuracy and communication reliability [96]. PPO is particularly suitable for ISAC because it can jointly optimize sensing and communication objectives within a unified policy framework. Its continuous-action capability enables adaptive adjustment of sensing and transmission parameters in dynamic environments. This flexibility helps maintain a balance between sensing accuracy and communication quality. This integration has also been applied to multi-agent formation control, where joint sensing-communication optimization reduces synchronization errors and closed-loop latency. Studies showed that such integration enables more reliable coordination in dynamic environments. Multiple automated guided vehicles performing cooperative tasks under base-station supervision [97] is an example of this. This model extends beyond conventional communication boundaries to represent integrated engineered systems.

4.5. UAV-Assisted Communications

UAVs and drones are part of the geospatial technology stack for surveillance, disaster response, and last-mile connectivity in urban and smart cities [98,99]. PPO-based energy management extends the operational endurance of UAVs. Researchers have extensively applied PPO to UAV communication systems, especially in areas that require real-time joint trajectory, power, and resource decisions [100]. PPO allows UAVs to learn long-term control strategies without requiring explicit mathematical models of the environment. The algorithm can simultaneously consider mobility, energy consumption, and communication quality in decision-making [101]. Such capabilities are valuable for autonomous UAV operations in rapidly changing network conditions. PPO-based DRL algorithm jointly controls 3D UAV positions, transmit power, and energy harvesting [102]. For secure UAV networks with eavesdropping, they can maximize a utility function that balances secrecy rate, energy efficiency, and interference. Such UAVs are commonly deployed in surveillance and defense applications. Similar enhanced policy optimization frameworks address physical layer security challenges in mobile communication systems under eavesdropping threats [51]. To capture both long-term mobility patterns and short-term channel fading, these algorithms use bidirectional long- and short-term memory methods. As a result, they maintain robustness against residual hardware defects and channel uncertainty while achieving improved secrecy-rate performance. Compared to fixed-deployment techniques, PPO achieves greater overall communication rates in STAR-RIS-assisted mobile wireless networks by concurrently optimizing hybrid beamforming and dynamically deploying STAR-RIS. A PPO-based sum rate maximization scheme for STAR-RIS-assisted vehicular networks underlaying UAV, achieving 22.05%, 35.12%, and 48.9% higher sum rate compared to DDPG, Zero forcing, and random schemes [103]. More coverage than traditional RIS is possible with Simultaneous Transmitting and Reflecting Reconfigurable Intelligent Surfaces (STAR-RIS), which can both broadcast and reflect signals.

Compared with off-policy RL techniques, the MAPPO strategy, which combines centralized training with decentralized execution, reduces the global Age-of-Updates (AoU) for UAV networks gathering time-sensitive IoT data by at least a factor of 1/2. MAPPO uses rate-splitting multiple access to maximize energy efficiency in multi-UAV wireless networks [58]. This approach jointly manages trajectory and beamforming, outperforming conventional multiple access schemes in complex environments. These results show that PPO effectively handles coupled mobility and communication constraints. Complementing physical-layer optimizations, a logical network architecture requires flexible slicing mechanisms to effectively support diverse service requirements. In satellite-assisted UAV data-collection scenarios, a compound-action PPO variant has been used to jointly optimize UAV trajectory, sensor-node scheduling, and the choice between direct and satellite-relayed transmission, minimizing information staleness while accounting for energy and communication costs [104].

4.6. Network Slicing, Service Function Chaining, and Orchestration

Network slicing in 5G and B5G involves real-time or autonomous resource management to handle heterogeneous services [105]. Dynamic network slicing allows efficient sharing of physical infrastructure across multiple services [59]. It partitions a common network into multiple logical slices tailored to different service demands. This enables flexible resource allocation across core, radio access, and end-to-end network segments while supporting diverse vertical applications, heterogeneous user requirements, and tenant-based service models. As a result, existing infrastructure can be used without compromising network service quality. In network slicing, PPO dynamically partitions physical network resources into virtual slices tailored to specific service requirements such as eMBB and URLLC. PPO can be used for up/downlink time slots allocation in 5/6G Dynamic TDD networks, dynamically assigning resources based on changing user requests and QoS requirements [106]. The co-existence of eMBB and URLLC traffic within shared physical infrastructure creates a particularly acute scheduling conflict. This is because URLLC packets must preempt already-scheduled eMBB resources, and they do so within mini-slots. To address this, an RIS-assisted hybrid slicing framework has been proposed. This framework uses PPO to choose between two actions for each incoming URLLC packet: NOMA-based superposition or puncturing [107]. Compared to standalone NOMA and puncturing schemes, this hybrid approach achieves significant gains. It improves the URLLC packet admission rate by 5.31%. It also increases the eMBB sum rate by 144% over optimization-based baselines. Additionally, it reduces execution time by 78.3%. Complementing this, traffic-aware slice management has also been demonstrated in B5G O-RAN. This approach couples two components: a lightweight SVM-based KPI classifier and a PPO agent [108]. The PPO agent is used for slice-aware physical resource block allocation within the Near-RT RIC. Compared to static schedulers, this method achieves up to 25% higher eMBB throughput. It also achieves up to 28% lower URLLC latency. In addition, the system includes a silence-aware reclamation mechanism. This mechanism recovers more than 90% of idle PRBs when demand fluctuates. Enhanced Mobile Broadband (eMBB) supports high-data-rate services such as video streaming. In contrast, Ultra-Reliable Low-Latency Communication (URLLC) supports mission-critical applications that require very low latency and high reliability.

To achieve the broad goals, PPO-based frameworks have been developed for end-to-end network slicing in multi-layer MEC environments, especially for Computation Offloading and Resource Allocation [109,110]. Both independent and collaborative agents within these frameworks learn to maximize resource efficiency while satisfying per-slice QoS requirements [111]. Such deployments significantly outperform the standard greedy algorithms. Network slicing improves infrastructure utilization and resource efficiency [60].

For Service Function Chain (SFC) deployment, PPO effectively balances node deployment cost and end-to-end service latency while ensuring quality-of-service requirements [112,113]. To further enhance performance, complementary studies have integrated particle swarm optimization with DRL. They achieved faster convergence and lower deployment costs under strict resource constraints. Building on these advances, PPO was successfully applied to multi-objective network slice provisioning. In these, PPO approximates a solution to an online Multi-Objective Integer Programming problem. In the context of Industrial 5G, such AI-driven orchestration is becoming increasingly critical. It enables sustainable, adaptive network management [61]. This approach aligns with network slicing paradigms in which AI-driven coordination enables autonomous resource allocation, admission control, and scheduling across shared industrial infrastructure. Such intelligence is increasingly necessary in softwarized 5G, 6G, and beyond networks, where SDN, NFV, and edge computing support multiple industrial services with diverse, stringent quality-of-service requirements [61]. These solutions yield lower SLA violation rates and network operational costs than state-of-the-art solvers. Service Level Agreements (SLAs) define the minimum performance targets that network operators must meet, including latency, reliability, and throughput.

4.7. Satellite Communications

Efficient satellite resource allocation extends satellite operational life [62,114,115]. Studies evaluated PPO-based DRL architecture for joint power and bandwidth allocation in a 23-beam GEO multibeam satellite [116]. The work demonstrated superior performance, comparable to optimized genetic-algorithm approaches, while enabling substantially faster decision-making, suitable for real-time resource allocation. High-throughput satellites (HTS) with flexible multibeam architectures require dynamic resource allocation that is impractical with manual tuning [63]. Ultra-dense LEO deployment significantly introduces new challenges to connection continuity and network stability [64,117]. Addressing these challenges requires jointly optimizing multiple interdependent control variables rather than treating them in isolation. A dual-agent DRL framework was proposed that jointly optimizes satellite handover decisions and transmission power allocation in LEO networks [118]. By combining DQN for discrete handover control and PPO for continuous power allocation, the approach effectively handles mixed action spaces and improves coordination between coupled decision variables. Simulation results show notable gains in throughput and user service completion rates. They showcase the advantage of hybrid PPO-based designs over single-agent optimization strategies [119].

To address these problems, multi-agent RL frameworks for efficient handover management in integrated satellite-terrestrial networks are studied [120]. Under shared network resource constraints, a MAPPO-based handover strategy enables coordinated decision-making among several users. Combining centralized training with distributed execution reduces unnecessary handovers and failure rates. At the same time, they bring long-term benefits in dynamic LEO scenarios. Simulation results showed better QoS, with reduced drop effects and more stable mobility management, compared to conventional and single-agent learning approaches. Decoupling decision-making into temporal and spatial subproblems reduces the number of handovers and boosts success rates [64]. Such approaches show their effectiveness in highly dynamic LEO satellite environments with frequent handovers. The clipping mechanism in PPO prevents destabilizing policy updates that could cause catastrophic resource misallocation during demand peaks. Unmet System Demand (USD) denotes the portion of requested communication capacity that the satellite system cannot satisfy. Evaluations on a 23-beam GEO multibeam satellite show that a PPO-based policy achieves comparable USD performance to an optimized genetic algorithm while running roughly 1500 times faster, reaching a decision in about 0.08 s versus roughly 2 min for the genetic-algorithm baseline [116]—making DRL operationally feasible. Hierarchical RL frameworks dissociate handover timing and target selection for dense satellite constellations [64]. Temporal agents utilize PPO algorithms to determine handover timing by predicting future satellite load. Spatial agents effectively employ deep Q-networks to select target satellites based on utility functions. Apart from handover and power management, PPO variants have also been applied to joint beam hopping and resource allocation in multi-LEO constellations, where hierarchical and multi-agent designs coordinate beam scheduling, bandwidth, and power across satellites to reduce load imbalance and inter-beam interference [121,122].

4.8. Vehicle-to-Everything (V2X) and Cognitive Radio

V2X communication optimization directly supports sustainable transportation, making it a key component of smart city planning [123,124]. V2X enables platooning and collision avoidance, enhances cooperative perception among connected vehicles, reduces their idle time, and lowers vehicular emissions [40]. It uses anti-jamming technology in dense urban traffic to improve traffic flow. It integrates well with roadside edge servers and blends with smart-city traffic management. V2X communications with energy harvesting can use a PPO-based multi-agent DRL framework [65]. Such systems use power splitting to divide harvested energy between information decoding and energy storage. Such implementations outperform traditional DRL resource allocation approaches in both effectiveness and robustness. This approach shows system-level innovation by jointly managing energy harvesting, communication decoding, and storage control.

PPO and its multi-agent variants are applied to cognitive radio and anti-jamming communication scenarios. In these, the agents learn adaptive spectrum access, power control, and interference mitigation strategies in dynamic adversarial environments. Recent studies have shown that MAPPO-based approaches are effective for cooperative jamming, cognitive-radio security, and intelligent anti-jamming decision-making in wireless communication systems [125,126,127]. Based on a single-agent PPO, MAPPO uses the KL-divergence and provides more stable data transmission than DQN in such complex vehicle environments [66]. The approach also supports computation offloading in vehicular edge networks, where it simultaneously manages task dependencies and stringent deadlines. Intelligent offloading decisions enabled by PPO reduce device battery drain and server energy usage. To achieve this, the algorithm uses surrogate clipping and generalized advantage estimation [12]. This helps achieve stable policy updates even in dynamic environments. This yields significant reductions in total delay and dropped-task ratios compared to DQN and related methods. These implementations demonstrate that PPO’s stability and continuous control capabilities are effective across diverse network environments. Concurrently, complementary studies investigated opportunistic routing in urban environments to address mobility uncertainty [128]. By integrating convolutional neural networks with Q-learning, these approaches predict node transitions and optimize data-routing decisions across diverse urban scenarios [129].

4.9. Common Trends in PPO-Based Communication Systems

Across the reviewed studies, several common patterns emerge. Most PPO-based communication-system models formulate the state space using network-level observations. They include channel quality indicators, signal-to-interference-plus-noise ratio (SINR), queue lengths, traffic loads, user mobility information, resource-utilization levels, and energy-related metrics. Similarly, the action space is commonly defined through resource-management decisions. These decisions are mostly related to power-control adjustments, spectrum allocation, beam-selection strategies, routing decisions, computation offloading choices, and network slicing configurations.

Another notable trend is the design of reward functions that jointly optimize multiple performance objectives. The analysis and results focused on throughput maximization, latency reduction, energy efficiency, quality-of-service (QoS) enhancement, and resource-utilization optimization. It reflects the inherently multi-objective nature of communication-network optimization problems.

Most studies rely on simulation-based evaluation using custom network simulators or established communication frameworks. This discusses the practical challenges of deploying and testing reinforcement-learning algorithms in operational communication infrastructures. The reported results consistently demonstrate improvements in network efficiency, reduced latency, optimized throughput, and improved energy management compared with conventional baseline methods. However, evidence of large-scale real-world deployment remains limited. Consequently, bridging the gap between simulation-based validation and practical deployment represents an important research direction for PPO-enabled communication systems.

4.10. Comparative Performance

Apart from raw performance measurements, sustainability metrics (particularly energy efficiency) are increasingly used as primary benchmarks in PPO-based communication studies. This shows a growing awareness of the environmental impact of intelligent network operation. As

Table 5. Comparative Analysis of PPO-Based Studies across Communication System Applications.

Comparison	Communication Domain	Metric/Outcome	Representative Result
PPO vs. DQN	Vehicular edge computing	Total delay reduction	PPO reduced total delay by 13.85% compared with DQN under dynamic vehicle speeds and task deadlines [54].
PPO vs. DDQN	Vehicular edge computing	Total delay reduction	PPO reduced total delay by 11.24% compared with DDQN [54].
PPO vs. DDPG	RIS-assisted MU-MISO beamforming	Energy efficiency	PPO improved energy efficiency by 15.8% relative to DDPG [9].
PPO vs. Fractional Programming (FP)	RIS-assisted MU-MISO beamforming	Energy efficiency	PPO improved energy efficiency by 34.2% relative to FP [9].
PPO vs. Heuristic Algorithms	Heterogeneous networks (HetNets)	Throughput and user adaptation	PPO outperformed heuristic resource-allocation methods under varying user loads and channel conditions.
PPO vs. Greedy Algorithms	Network slicing and MEC	Resource efficiency/QoS	PPO-based slicing achieved better resource utilization and QoS satisfaction than greedy approaches [111].
PPO vs. Genetic Algorithm	GEO satellite power allocation	Runtime/unmet system demand	PPO achieved comparable unmet-demand performance while operating faster than an optimized genetic algorithm [116].
PPO vs. ACER	O-RAN resource allocation	Energy–latency trade-off	PPO achieved a more favorable balance between energy consumption and user latency than ACER.
MAPPO vs. IPPO	5G resource allocation/routing	Convergence speed, packet loss, and latency	MAPPO converged faster and achieved lower latency and packet loss than IPPO [30].

Note: Cross-study comparison should be interpreted cautiously because reviewed studies used different simulation environments, datasets, traffic models, network assumptions, and evaluation metrics.

shows, empirical studies consistently show that PPO offers an optimal balance between performance and stability, giving it an edge [13].

Several reviewed studies reported measurable sustainability-related improvements associated with PPO-based optimization. Representative findings include a 15.8% increase in energy efficiency in RIS-assisted MU-MISO beamforming systems, a 34.2% improvement over fractional programming baselines, and substantial reductions in delay and interference in vehicular and satellite communication environments. However, most studies evaluate operational efficiency rather than full lifecycle sustainability metrics. Metrics such as AI training energy consumption, inference cost, embodied carbon impact, and end-to-end deployment footprint remain underreported across the literature.

Despite these advantages, PPO typically requires greater computational resources and longer training times than lightweight heuristic methods. In some communication scenarios with limited training data or highly discrete action spaces, alternative DRL approaches may remain preferable.

Many reviewed studies reported that PPO outperforms value-based methods in continuous-control and high-dimensional optimization settings. This advantage was especially evident in problems involving continuous control variables, such as transmit power, beamforming, and computation offloading. Several studies observed improved performance relative to heuristic and optimization-based approaches under dynamic communication conditions. Energy-sensitive beamforming and satellite resource-allocation scenarios reported the largest gains. Compared with other DRL methods, PPO’s main strengths were greater training stability and faster convergence in volatile network conditions.

However, PPO performance is highly dependent on the communication scenario, reward formulation, state representation, and hyperparameter calibration. Several reviewed studies used different simulation settings, traffic models, and evaluation metrics. This limits a direct cross-study comparison. PPO does not universally outperform alternative DRL methods in all conditions. For example, in highly discrete decision environments or sample-limited settings, value-based approaches such as DQN or DDQN may remain competitive. Similarly, PPO performance can degrade in certain situations. For example, in poorly designed reward structures, unstable channel observations, or insufficient exploration. In communication problems dominated by discrete decision spaces, such as channel selection or binary routing actions, value-based approaches like DQN and DDQN may offer lower computational complexity and simpler policy learning. Similarly, in sample-limited environments where large-scale policy training is impractical, off-policy methods may achieve faster learning efficiency. Lightweight heuristic or rule-based algorithms may also remain preferable in latency-sensitive edge deployments with limited computational resources. Therefore, the reported advantages are best interpreted within the context of the specific communication environment and evaluation framework used in each study.

4.11. Synthesis of Research Questions

The synthesis of literature provides clear and direct answers to the specific research questions formulated earlier in Section 3.1 of this review paper. Looking at the first question, PPO appears widely applied in resource allocation and beamforming across many different network types. Recurring design patterns for multi-agent coordination and hybrid architectures for energy-aware optimization in complex communication systems are documented in the literature. For the second question, future work faces challenges related to scalability and transitioning from simulation to real networks. Answers to the third question show PPO contributes directly to sustainable digital infrastructure through energy-efficient smart-city services and IoT. IoT coordination and intelligent communication management are key areas where these algorithms effectively support greener infrastructure goals in modern cities. Finally, this study identified several unresolved challenges (Section 5.1) and future directions (Section 6) for PPO-enabled communication systems that researchers should address soon.

5. Challenges and Limitations

5.1. Challenges with PPO Frameworks

PPO-based communication frameworks face substantial deployment challenges despite encouraging empirical findings. At an algorithmic level, it has limitations related to the static entropy coefficient and the hard-clipping mechanism, which require solutions [12]. The sim-to-real gap, the difficulty of reward structuring, multi-agent scalability, and the energy consumption and carbon footprint of AI-driven network operations are some of the major issues noted in the study. The majority of reviewed research does not assess PPO frameworks in practical smart-city or large-scale IoT scenarios; instead, they employ general wireless simulation setups. Therefore, there is a sim-to-real gap in the literature that has not been fully addressed, which limits the applicability of its conclusions to sustainable urban infrastructure environments.

Real-world channel dynamics, hardware impairments, and environmental interference make it difficult to replicate observations in synthetic testbeds faithfully. This gap undermines the practical transferability of simulation-trained models to operational wireless systems. Poorly calibrated models waste energy through unnecessary retransmissions and suboptimal power settings, making sim-to-real transfer a priority for both performance and efficiency.

The energy cost of AI training is another key challenge [130]. Training deep PPO models requires significant computational resources. Studies have noted that large-scale AI model training can require substantial GPU utilization and energy consumption, resulting in measurable carbon-equivalent emissions during training and deployment [130]. However, the carbon footprint and lifecycle energy cost of PPO-based reinforcement training remain underreported across communication-system studies [131]. Future evaluations should report standardized sustainability metrics, including GPU training hours, inference energy per decision cycle, communication overhead during distributed learning, carbon-equivalent emissions, and end-to-end operational energy savings during deployment. Reward shaping in multi-objective optimization is a challenge for AI models. It is still difficult to design reward functions that simultaneously balance throughput, latency, energy efficiency, and fairness. Poorly designed rewards may lead to unanticipated behavioral trade-offs or unsatisfactory convergence. Hybrid PPO-DQN systems see this challenge. To attain steady performance, they need careful multi-objective reward design. Similarly, although PPO is said to be simpler to tune, hyperparameter sensitivity persists. Careful calibration is still required for clipping parameters, learning rates, entropy coefficients, and GAE lambda values.

Misconfigured hyperparameters can destabilize training, particularly in fast-varying vehicular or satellite environments. Scalability and observability further complicate real-world adoption. While MAPPO demonstrates strong cooperative performance, scaling a large number of heterogeneous agents introduces significant coordination overhead. Most of the challenges will be related to credit assignment. MAPPO-based energy-saving control strategy can be used for ultra-dense network base stations. In such deployments, each base station will be treated as an agent requiring collaboration and interference management [132]. Integrated Adaptive Communication Network (IACN) built on MAPPO that dynamically adjusts communication topology and optimizes content for efficient multi-agent coordination [133]. Non-stationarity in ultra-dense 6G scenarios exacerbates these challenges.

5.2. Limitations of the Present Review

The review process identified substantial content overlapping across studies, particularly in RIS-assisted beamforming and satellite handover domains. Such redundancy complicates synthesis and suggests a need for clearer differentiation of novel contributions. The study focused on the 2023–2026 timeframe to capture recent advances, but, in the process, might have necessarily excluded foundational pre-2023 works. This introduces some recency bias into the overall synthesis. Although the final set of 76 studies is sufficient for a focused review of recent PPO applications, some emerging work may not yet have appeared in peer-reviewed form or within the selected databases. Further, the analysis is limited to peer-reviewed studies and explicitly excludes preprints, industrial reports, and other unpublished results. This limitation matters because preprints often contain the latest work in fast-moving fields such as PPO, 6G, RIS, ISAC, and multi-agent reinforcement learning, several months before formal publication.

The review considered only English-language publications and omitted publications in Chinese, Korean, Japanese, and other languages. A large number of active research communities in the literature on wireless communications and AI were written in these languages [134,135]. Another limitation is the potential for publication bias. Studies reporting positive PPO performance improvements are more likely to be published than studies with negative, null, or inconclusive findings. As a result, the reviewed literature may overestimate the consistency of PPO’s advantages across communication domains. Considerable heterogeneity in datasets, simulation settings, and performance metrics limits direct comparison across studies. Thus, determining whether reported performance gains arise from the PPO method itself or from differences in experimental setup is difficult to establish. Finally, most reviewed works rely on simulation-based validation, leaving real-world deployment evidence relatively limited.

A further limitation of the present review is its deliberate focus on PPO-based and related DRL approaches for communication-network optimization. Although PPO represents one of the most widely adopted DRL algorithms in recent 5G, B5G, and 6G research, other optimization paradigms include mathematical optimization, game-theoretic approaches, and heuristic and metaheuristic algorithms. Also, alternative DRL methods such as SAC, TD3, DDPG, TRPO, and A3C were not comprehensively analyzed. Therefore, the conclusions of this survey should be interpreted within the scope of PPO-centered research.

6. Future Research Directions

Future studies may extend the scope of this review. Researchers could conduct comprehensive comparative analyses of PPO and alternative optimization approaches, including SAC, TD3, DDPG, TRPO, A3C, heuristic methods, and mathematical optimization frameworks. Such investigations would provide deeper insights into the trade-offs among learning stability, sample efficiency, computational complexity, and scalability. They would also make practical deployment in next-generation communication networks easier. In addition, PPO applications are emerging in areas such as vehicular edge computing, distributed edge intelligence, and AI-native 6G networks. These remain promising directions for future research.

Real-World Validation and Sim-to-Real Transfer: Future research should quantify the energy cost of PPO training [136]. They need to consider inference pipelines and explore lightweight PPO variants [137] suitable for deployment on energy-harvesting IoT nodes and edge devices in smart cities. Literature indicates a growing need for advanced multi-agent reinforcement learning frameworks to address scalability and coordination challenges in next-generation communication systems. These studies should examine whether the significant computational overhead required to train the models could eventually undermine the net energy benefits achieved during their operational life.

Scalable Multi-agent PPO for Ultra-Dense Networks: Recent dual-clip MAPPO implementations in energy-constrained aerial systems demonstrate improved convergence and coordination stability [138]. Convergence speeds during trajectory optimization improve significantly when employing adaptive discounting strategies. Future work should extend these design principles to heterogeneous multi-tier network architectures.

Semantic-aware communication reduces unnecessary data transmission, lowering energy consumption and enabling sustainable bandwidth utilization in IoT-heavy smart city networks. Integrating PPO-based resource control with adaptive, knowledge-driven mechanisms can enable efficient handling of semantic redundancy and improve system-level quality of experience. These developments point toward unified cross-layer optimization strategies that bridge semantic processing and physical-layer design in next-generation communication systems. Building upon foundational machine learning successes in maritime networks, future frameworks should incorporate deeper semantic awareness across communication layers [46]. Future investigations must prioritize hybrid architectures that balance computational efficiency with real-time decision-making capabilities in offshore environments.

7. Implications for Sustainable and Smart Infrastructure

7.1. Sustainability and Practical Implications

The review has direct implications for modernizing digital communication systems and infrastructure. Integrating sensing and communication functions can reduce dedicated hardware requirements and associated spectrum overhead in large IoT deployments [56,93]. Future communication architectures should jointly optimize throughput, latency, and operational energy consumption while accounting for the computational overhead of AI-driven control. Theoretical models should incorporate carbon footprint alongside throughput and latency as primary design objectives.

Network designers who are planning to create new communication system architectures in the near future can prioritize energy efficiency. Future 5G and 6G deployment strategies must explicitly account for algorithmic energy costs [92]. It is expected that collaboration between industry and academia will steadily improve the deployment of AI-driven solutions.

7.2. Implications for Smart Infrastructure

PPO effectively manages the spectrum during peak loads, making urban communication systems more resilient [139]. Adaptive public infrastructure will benefit from self-optimizing networks that automatically adjust to changing citizen demands. Such infrastructure will be a key component of adaptive governance, especially in smart cities, which, in turn, help ensure proper service delivery [140]. Sustainable transportation systems use V2X technologies to reduce congestion, significantly reduce delays and energy use, and thereby lead to positive environmental and economic impacts [141].

8. Conclusions

Across 76 peer-reviewed studies published between 2023 and 2026, PPO has emerged as one of the most effective deep reinforcement learning approaches for optimizing next-generation communication systems. Significant improvements are consistently reported in energy efficiency, spectral utilization, and system-level coordination across resource allocation, beamforming, routing, MEC, ISAC, UAV-assisted networks, satellite communications, and V2X systems. Across these domains, PPO performs particularly well in environments characterized by continuous control, dynamic network conditions, and competing optimization objectives.

Several limitations remain. The sim-to-real transfer gap continues to hinder practical deployment because operational network conditions are difficult to reproduce accurately in simulation environments. Reward design for multi-objective optimization remains challenging, while MAPPO-based coordination becomes increasingly complex as agent density scales toward ultra-dense 6G infrastructures.

Future research must prioritize real-world testbeds, standardized sustainability metrics, and reproducible multi-agent evaluation frameworks. The convergence of 6G, edge intelligence, and large-scale IoT environments creates conditions where PPO-based optimization is especially valuable, given the high-dimensional state spaces, heterogeneous agents, continuous action requirements, and strict energy constraints. PPO is steadily evolving from a task-specific optimization technique into a foundational control framework for intelligent, adaptive, and energy-aware communication ecosystems.