Joint 3D Trajectory and Power Optimization for UAV Swarms in Cell-Free Massive MIMO Networks: A CTDE-MAPPO Framework for Sensing-Aware Precision Agriculture

Highlights

What are the main findings?

• A joint sensing–communication utility function is introduced that balances communication energy efficiency and field coverage completeness in a single weighted objective that is embedded directly into a MAPPO-CTDE reward design with hard battery depletion and return-to-depot constraints—a formulation that, to our knowledge, is the first to jointly integrate these specific elements within a CF-mMIMO UAV setting, building on established CTDE, MAPPO, and reward-shaping techniques.
• The CF-mMIMO CPU serves as a global critic during training, resolving inter-agent non-stationarity while each UAV executes a lightweight local policy at runtime, achieving 93% field coverage and a 92% depot-return rate without any inter-UAV communication overhead at execution time.

What are the implications of the main findings?

• Jointly optimizing 3D rotary-wing trajectories and transmit power through a shared coverage map creates strong inter-agent coupling that makes the CF-mMIMO CPU a natural and architecturally motivated centralized critic—enabling scalable swarm coordination without dedicated inter-UAV communication links.
• CF-mMIMO macro-diversity, combined with sensing-aware trajectory adaptation, offers a practical and deployable pathway toward energy-autonomous UAV swarms for large-scale crop monitoring in 6G agricultural networks.

Abstract

The integration of Unmanned Aerial Vehicle (UAV) swarms with Cell-Free massive Multiple Input Multiple Output (CF-mMIMO) networks offers promising prospects for large-scale crop monitoring in precision agriculture. CF-mMIMO provides macro-diversity and uniform channel quality across large agricultural fields. However, practical deployment demands jointly optimizing 3D trajectories and transmit power to maximize energy efficiency and field coverage simultaneously. This is challenging due to the limited battery capacity, mandatory return-to-depot constraints, and collision avoidance requirements. In this paper, we introduce a joint sensing–communication utility function that captures the trade-off between energy efficiency and field coverage completeness. To provide a scalable and distributed solution for rotary-wing UAV swarms, we develop a multi-agent deep reinforcement learning (MADRL) methodology based on the Multi-Agent Proximal Policy Optimization (MAPPO) approach. We adopt the Centralized Training with Decentralized Execution (CTDE) strategy, in which a CF-mMIMO central processing unit (CPU) serves as a global critic during training. At execution time, each UAV independently runs a lightweight local policy that adapts its trajectory and transmit power in real time based on battery state and air-to-ground channel variations. Simulation results reveal that the proposed MAPPO-CTDE approach outperforms existing benchmarks. Unlike prior methods that require instantaneous global CSI or neglect the sensing–communication coupling, the proposed approach simultaneously achieves high field coverage completeness, robust communication energy efficiency, and a high depot-return rate under hard battery constraints without any inter-UAV communication overhead at execution time.

1. Introduction

The convergence of sixth-generation (6G) wireless infrastructure and autonomous aerial systems is reshaping large-scale environmental monitoring [1,2,3]. Precision agriculture represents one of the most demanding applications of this convergence: it requires centimeter-level spatial coverage of fields spanning hundreds of hectares, a real-time uplink of high-dimensional sensing data, and energy-autonomous operation under strict flight-time constraints [4]. Ground-based IoT sensors provide localized measurements of soil moisture, temperature, and vegetation indices [5], but fixed infrastructure cannot scale to the spatial extent of modern farms [6]. Unmanned Aerial Vehicles (UAVs) resolve this limitation through three-dimensional mobility that enables an on-demand surveying of priority zones across arbitrarily large fields, decoupling sensing coverage from the constraints of fixed deployment [7,8,9,10,11]. Meeting the joint requirements of coverage completeness, real-time data uplink, and energy-autonomous operation—simultaneously and at scale—defines a class of integrated sensing-and-communication problems that existing wireless network architectures were not designed to solve. Deploying swarms of UAVs further improves coverage speed and fault tolerance, enabling a parallel surveying of spatially distributed field zones within a single mission horizon [12]. Simultaneous multi-UAV operation introduces co-channel interference, spectrum contention, and uplink reliability degradation. These communication impairments are particularly problematic as agents move continuously across the field, exceeding the mitigation capabilities of traditional single-cell network architectures at scale. Cell-Free massive Multiple-Input Multiple-Output (CF-mMIMO) networks, widely recognized as a key enabling technology for 6G systems [13], offer a principled solution to these impairments. Rather than relying on a single base station, CF-mMIMO distributes numerous access points (APs) across the coverage area and connects them to a central processing unit (CPU) via dedicated fronthaul links [14,15]. This architecture provides macro-diversity, suppresses path shadowing, and supports the coherent joint processing of simultaneous uplink transmissions from multiple UAVs—delivering uniform signal quality across the entire field regardless of UAV position [16].

1.1. Motivation and Problem Statement

Realizing the full potential of UAV swarms in precision agriculture requires jointly optimizing 3D trajectories and transmit power to simultaneously maximize energy efficiency and field coverage completeness—a non-convex, long-horizon problem further complicated by hard battery depletion limits, mandatory return-to-depot deadlines, and collision avoidance requirements. Despite recent progress in UAV-assisted CF-mMIMO networks, existing frameworks address only a subset of these coupled constraints. Most prior studies optimize exclusively for communication metrics, neglecting sensing coverage as a primary mission objective, or treat battery depletion as a soft penalty rather than a hard operational limit.

In a real agricultural deployment, ignoring these coupled constraints is not a minor limitation but a point of mission failure:

• A UAV that hovers exclusively at the altitude yielding the best channel quality will maximize communication efficiency but fail to sweep or scan the field.
• A UAV that ignores aerodynamic battery depletion will drain its reserves prematurely and fail to return safely to the depot.
• A swarm mission that skips partially surveyed zones will leave the end-user with an incomplete, unusable crop map.

Taken together, these conflicting physical and network constraints render the joint optimization problem fundamentally harder to solve. Addressing these intertwined limitations and bridging the gap between communication-centric design and physical mission survival serves as the core motivation behind the methodology developed in this paper.

1.2. Paper Contributions

To tackle the above-mentioned limitations, this paper presents the following key contributions:

• We propose a CF-mMIMO multi-UAV remote sensing (RS) architecture for precision agriculture. To accurately reflect real-world deployments, this architecture incorporates (i) a three-dimensional (3D) rotary-wing mobility model, (ii) a probabilistic air-to-ground fading channel, and (iii) a full-mission battery depletion model.
• We formulate a joint sensing–communication (SC) utility function $\mathcal{U}$ that simultaneously rewards communication energy efficiency and field coverage completeness under a single weighted objective (Section 4). We cast the resulting optimization as Problem $\mathcal{P}$: a non-convex, long-horizon, battery-constrained trajectory and power allocation problem subject to hard battery depletion limits, mandatory return-to-depot constraints, and minimum uplink rate requirements. We demonstrate analytically that $\mathcal{P}$ cannot be decomposed slot-by-slot, thereby ruling out classic SCA and supervised DNN approaches, and firmly motivating our MADRL reformulation. This formulation is absent from prior CF-mMIMO UAV studies, which either omit sensing coverage entirely or treat battery constraints as soft penalties.
• Our proposed methodology centers on a Multi-Agent Proximal Policy Optimization (MAPPO) approach, which is selected specifically to mitigate the non-stationary dynamics of collaborative aerial swarms. By implementing a Centralized Training with Decentralized Execution (CTDE) strategy, we eliminate the need for inter-UAV communication during live missions. The training phase utilizes the CF-mMIMO CPU as a shared global critic that processes the complete system state to refine the behavior of all agents. Once deployed, each UAV operates autonomously, executing a localized policy that continuously adapts spatial trajectories and uplink power based on real-time battery levels and fading channel conditions. Furthermore, we carefully engineer the reward mechanisms and state-action spaces to ensure that the fleet naturally converges on strategies that maximize agricultural sensing coverage, avoid collisions, and respect absolute energy constraints.

The remainder of this paper is organized as follows. In Section 2, we review prior studies and highlight the key research gaps motivating this paper. Section 3 details the CF-mMIMO-assisted precision agriculture system model and outlines the key assumptions considered throughout this paper. In Section 4, we formulate the joint optimization problem. Section 5 describes the proposed MAPPO-CTDE framework. In Section 6, we examine the simulation results. Section 7 discusses the achieved performance gains. Section 8 presents concluding remarks and highlights directions for future research.

Mathematical Notation: Throughout this paper, ${(·)}^H$ denotes the conjugate transpose. Boldface lowercase letters indicate vectors, while boldface uppercase letters indicate matrices. $\parallel ·\parallel$ and $|·|$ denote the Euclidean norm and absolute value. $\mathbb{E}[·]$ is the expectation operator. We denote by $\mathcal{CN}(\mu ,{\sigma }^2)$ the circularly symmetric complex Gaussian distribution.

2. Related Work

The integration of multi-UAV swarms with CF-mMIMO networks brings together challenges from wireless resource allocation, trajectory planning, and multi-agent reinforcement learning. Although substantial progress has been made in each area, enabling fully autonomous operation for precision agriculture requires a joint design that considers both communication performance and UAV flight constraints. This section reviews the literature most relevant to the proposed framework. We first discuss power allocation methods for CF-mMIMO systems, including both conventional optimization techniques and recent learning-based approaches. We then review UAV trajectory optimization methods and their limitations in dynamic multi-agent environments. Next, we examine studies that jointly optimize UAV trajectories and transmit power, highlighting the challenges associated with decentralized control and non-convex optimization. Finally, we compare the proposed MAPPO-based framework with existing approaches and identify the remaining challenges related to sensing coverage, battery limitations, and return-to-depot requirements.

2.1. Power Allocation in CF-mMIMO Networks

Resource allocation—specifically power assignment—in CF-mMIMO networks has been extensively investigated under both convex optimization and learning-based frameworks. Björnson and Sanguinetti [17] established the spectral and energy efficiency fundamentals of CF-mMIMO. They demonstrated the superiority of distributed processing over conventional cellular systems. Nevertheless, the scalability of cell-free networks remains constrained by two primary factors: the computational complexity of collaborative signal processing and the extensive signaling overhead required for inter-AP synchronization and data exchange. Recent research has focused on replacing computationally expensive iterative optimization solutions—including weighted minimum mean square error (WMMSE) and Fractional Programming—with deep learning (DL) to manage the complexity of CF-mMIMO systems. Braga et al. [18] formulated the joint pilot and data power control problem in CF systems as an optimization task. To address this, they proposed a decentralized actor–critic DRL framework that operates with minimal information exchange among network nodes. The authors in [19] tackled the challenge of uplink and downlink power control in CF-mMIMO systems by developing an unsupervised deep learning framework. By using only large-scale fading coefficients as input features, their approach establishes a desirable trade-off between overall system performance and processing overhead. Crucially, their framework supports scalability through its compatibility with user-centric access point clustering architectures. Similarly, Zaher et al. [20] introduced a deep learning framework for downlink power allocation to improve sum spectral efficiency (SE) and proportional fairness (PF). They designed a DNN that maps local large-scale fading (LSF) parameters and heuristic power coefficients directly to the target allocation. This configuration allows the trained model to approximate centralized optimization performance while maintaining a distributed implementation at individual APs. Furthermore, they developed a clustered DNN architecture to jointly optimize power coefficients within local AP groups. This approach successfully balances the trade-off between utilizing global network knowledge and reducing computational overhead. However, these existing deep-learning paradigms are inherently limited in dynamic environments. Zaher et al. and similar contemporary studies [17,18,19,20] focused exclusively on standard physical-layer metrics, such as spectral efficiency and fairness, for stationary or generic user equipment. Consequently, they do not account for dynamic, mission-oriented scenarios like UAV-based precision agriculture. In such deployment contexts, 3D trajectory planning, strict battery capacity constraints, and spatial sensing coverage are equally critical to operational success.

2.2. UAV Trajectory Optimization

In [21], the authors derived a foundational analytical model for rotary-wing UAV propulsion power. Zeng et al. investigated a joint trajectory and time-allocation design to minimize the total energy consumption subject to ground-node throughput constraints. To address the continuous and non-convex problem, they employed path discretization coupled with SCA techniques. While effective for predictable, offline trajectory synthesis, such analytical optimization frameworks struggle to adapt in real time to the rapid, dynamic large-scale fading variations inherent to complex cell-free massive MIMO environments.

Similarly, Wu et al. [22] developed SCA-based frameworks for UAV trajectory but required perfect instantaneous CSI and centralized execution, limiting their applicability to multi-UAV deployments. The application of DRL to UAV trajectory planning was explored in cellular networks [23], where single-agent formulations demonstrated adaptability to dynamic channels. However, these approaches did not scale to multi-UAV settings or exploited the distributed structure of CF-mMIMO. The deployment of UAVs within CF-mMIMO frameworks was also explored from an infrastructure perspective, where aerial platforms acted as mobile base stations to assist ground networks. For instance, in [24], the authors utilized UAVs as mobile aerial access points (MAAPs) and deployed a Tabu-search algorithm alongside gradient-based location optimization to alleviate pilot contamination in user-centric CF-mMIMO networks. Moreover, Kenneth et al. [25] considered the deployment of UAVs as moving aerial providers (MAPs) to offload traffic and eliminate coverage dead zones. To handle the highly non-convex deployment problem under dynamic user distributions, the authors developed a hybrid framework that pairs an enhanced K-means clustering algorithm for user association with a soft actor–critic deep reinforcement learning algorithm for 3D location optimization. Nevertheless, the studies in [21,22,23,24,25] left several critical research gaps unaddressed. First, these studies focused narrowly on physical-layer metrics (downlink capacity and signal to interference plus noise ratio (SINR)). They overlooked dynamic, long-horizon operational constraints that are essential for autonomous fleet missions. These included continuous trajectory planning, hard battery depletion limits, and mandatory return-to-depot deadlines. Second, the centralized SAC approach adopted in these studies performed well for static positioning. It did not, however, scale to the distributed coordination demands of multiple moving aerial agents operating in real time. A related line of work in ground-based intelligent transportation addressed a structurally similar problem—generating smooth, collision-free trajectories under dynamic obstacle motion—using classical planning rather than learning. Huang et al. [26] proposed a sine resistance network (SRN) combined with a bias oval artificial potential field (BOAPF) for autonomous electric vehicles, meshing the road as a resistance network whose branch resistances were set by a potential field that incorporated the relative velocity of nearby obstacles. The planned path was then obtained by tracing the path of maximum current, which, by construction, avoided the local-minima failure mode of classical APF methods. However, like the SCA and APF-based UAV trajectory methods discussed above, this approach computed an explicit potential field or resistance network at every time step rather than learning a policy, and it targeted a single ground vehicle reacting to nearby obstacles. It did not address multi-agent coordination, energy-constrained long-horizon missions, or the joint sensing–communication objective central to this paper.

2.3. Joint Trajectory and Power Optimization for UAV-Based Systems

Jointly optimizing UAV trajectories and transmit power is considerably more challenging than optimizing either variable separately. The difficulty arises from the strong interdependence between these two decisions. A UAV’s trajectory determines the evolution of the air-to-ground channel conditions, which in turn affects the transmit power required to meet uplink rate requirements. At the same time, the energy consumed along a given trajectory directly constrains the mission duration and achievable flight range. As a result, approaches that optimize trajectory and power independently—by fixing one variable before optimizing the other—often lead to suboptimal performance. Early research explored the deployment of UAVs as amplify-and-forward relays to connect base stations and mobile devices that lacked reliable direct communication links. To address these connection challenges, Zhang et al. [27] proposed a framework that jointly optimized the flying trajectory of a single UAV alongside the transmit power of both the aerial relay and the ground-level mobile device. Their primary objective was to minimize the overall outage probability of the uplink network. Because the formulated problem was non-convex, Zhang et al. decoupled it into two distinct optimization stages, where they applied a gradient descent method for the trajectory design and extremum principles for the power control. Through an iterative alternating minimization algorithm, their joint solution significantly reduced the network’s outage probability when compared to baseline models that relied on fixed relay positions, equal power allocation, or simple circular flight paths. However, the work [27] relied on a highly simplified topology restricted to a single mobile device and a single UAV, leaving the complexities of multi-user environments and UAV swarms unaddressed. Furthermore, by decoupling the non-convex optimization problem, their alternating minimization approach yielded only an approximate local minimum rather than a globally optimal solution. Hur and Lee [28] explored the simultaneous adjustment of flight paths and transmission power when reacting to a suddenly detected ground eavesdropper. To ensure uninterrupted service, the authors introduced a consistent fairness constraint that maintained stable data transmission rates for all users at every moment while the UAV relocated between its initial and final optimal hovering points. Furthermore, they integrated an energy constraint to account for the UAV’s limited battery capacity during this redeployment, utilizing block coordinate descent and SCA approaches to solve the complex, non-convex optimization problem. However, despite demonstrating significant energy savings with minimal secrecy rate sacrifices, their approach hinged on several restrictive assumptions. Specifically, the proposed framework [28] was limited to a single UAV serving entirely stationary ground users over simplified, line-of-sight-dominated channels, which inherently restricts its immediate applicability in highly dynamic, multi-UAV environments or cluttered urban settings with complex signal obstructions. To address the scalability challenges associated with fully joint optimization, recent studies explored hierarchical and decentralized learning frameworks. For instance, Wu et al. [29] introduced a hierarchical framework utilizing a MARL method with decoupled critics (DCs) tailored for loosely coupled multi-UAV emergency communication tasks. In their architecture, a high-level clustering algorithm first assigned ground users to specific UAVs, while the lower-level MARL-DC layer jointly optimized UAV trajectories and transmit power to satisfy a proportional fairness objective. By decoupling the critics, each agent could effectively isolate shared global coordination objectives, such as collision avoidance, from localized decision-making processes. Although the approach of [29] demonstrated superior learning efficiency and robust collision avoidance in dynamic disaster-relief environments, its application to precision agriculture remained limited. The target scenario in [29] is disaster-relief emergency communication, where UAV tasks are loosely coupled, and ground users are distributed across a post-disaster area—a fundamentally different operational context from crop monitoring, where sensing coverage completeness and energy-autonomous return-to-depot are non-negotiable mission requirements. Specifically, the work [29] did not incorporate CF-mMIMO infrastructure. Moreover, that formulation omitted critical domain-specific constraints, such as rotary-wing aerodynamic propulsion energy consumption modeling, strict battery capacities, return-to-depot requirements, and sensing coverage objectives. Shah et al. [30] developed a three-stage framework that jointly optimized UAV 3D trajectories, transmit power, and user association. The authors used an Accelerated Proximal Gradient (APG) framework to manage the inherent non-convexity of the formulated optimization problem. Their approach reduced the number of deployed drones and improved per-user throughput, but it focused entirely on communication metrics. Yet, classical optimization approaches, such as Successive Convex Approximation (SCA) or the APG method utilized by Shah et al. [30], typically yield centralized policies. These frameworks require aggregating instantaneous channel state information (CSI) at every slot to jointly optimize trajectories and power. This creates massive communication overhead and processing delays, making such centralized approaches inherently incompatible with the decentralized execution requirements of multi-UAV systems. In addition, the work in [30] did not consider energy efficiency or sensing coverage. This leaves a significant gap for precision agriculture applications, which demand full spatial coverage under strict battery constraints. Ma et al. [31] examined the problem of resource allocation within UAV-aided uplink CF-mMIMO systems, where the system was required to satisfy the stringent demands of ultra-reliable and low-latency communications. To maximize energy efficiency (EE) and minimize interference to ground users, the authors proposed a power allocation approach driven by a self-supervised deep neural network (DNN). Although effective in their setting, supervised DNNs are poorly suited to this problem for three reasons. Generating training labels requires solving the globally optimal trajectory-power-coverage sequence for every possible system state—a combinatorially intractable task. A DNN also produces a reactive, one-shot mapping with no ability to reason about how current decisions affect battery or coverage later in the mission. Finally, DNNs trained on a fixed channel distribution generalize poorly to new environments, swarm sizes, or seasonal changes in vegetation. Multi-Agent Reinforcement Learning (MARL) addresses all three of these limitations. It requires no pre-computed labels, optimizes long-term cumulative reward through temporal credit assignment, and adapts to non-stationary environments through continued interaction. Liu et al. [32] tackled UAV trajectory optimization within CF-mMIMO networks by proposing a Single Critic Multi-Actor (SCMA) Multi Agent Deep Deterministic Policy Gradient (MADDPG) methodology. This approach achieved an EE gain over traditional SCA methods. Their work considered fixed-wing UAVs acting as flying base stations and optimized the trajectory solely for communication energy efficiency. Sensing coverage, battery constraints, and return-to-depot requirements were not addressed. Several recent studies integrated deep reinforcement learning (DRL) to manage the multi-faceted operations of mobile agents. For instance, Lee and Ko [33] proposed an online DRL framework, termed DeepUAV, that optimized a UAV’s trajectory alongside data relay decisions and downlink wireless power transfer (WPT) for ground sensor nodes. While their approach successfully navigated the trade-offs between data freshness and node energy replenishment, it relied on a single-UAV setup operating over a simplified, grid-based state space. In contrast, our framework expands this paradigm to joint 3D trajectory tracking and physical sensing coverage within multi-UAV CF-mMIMO systems, accounting for explicit aerodynamic depletion profiles and macro-scale field completeness metrics.

Saraiva et al. [34] explored how aerial and ground users can share the same CF-mMIMO network. To achieve this, they used distributed DRL to manage uplink power and maximize the overall spectral efficiency (SE). Their formulation, however, treated UAVs as static communication nodes competing for network resources with no consideration of trajectory planning, sensing coverage, or battery management. To bridge the gap between computationally expensive offline trajectory optimization and real-time onboard deployment, Bianchi et al. [35] introduced a real-time trajectory control framework for UAVs using a rule-based minimum-energy reference generation strategy. By extracting practical operating rules from offline optimal control formulations, their method determines energy-optimal mission durations and reference trajectories online. This significantly reduces the onboard computational burden while preserving near-optimal energy performance during path tracking under dynamic environmental constraints. However, such rule-based heuristics optimize physical trajectories independently of the underlying wireless network infrastructure. In a collaborative 6G-enabled aerial swarm, trajectory design is intrinsically coupled with time-varying fading channels and multi-user co-channel interference. Consequently, static operating rules cannot dynamically adapt to joint sensing-and-communication (SC) objectives where aerodynamic energy conservation must be balanced online against real-time data uplink rates and field-wide coverage completeness. Zhang et al. [36] proposed a scalable constrained policy optimization paradigm for safe Multi-Agent Reinforcement Learning (MARL). Their algorithm, Scalable MAPPO-Lagrangian (Scal-MAPPO-L), exploits the decay of spatial correlations in large-scale networks to optimize $\kappa$-hop localized policies. By integrating localized trust-region bounds with a truncated localized advantage function, the framework guarantees both strict safety constraint satisfaction and scalable policy optimization using only local interactions, successfully mitigating the need for a global coordinator. Despite providing robust safety guarantees via decentralized interactions, such localized constrained frameworks are ill suited for architectures governed by centralized infrastructure, such as Cell-Free massive MIMO (CF-mMIMO) networks, where joint signal processing at a central processing unit (CPU) is essential to maximize macro-diversity and mitigate spectrum contention. Furthermore, these methods typically manage constraints as localized, soft penalties, failing to address long-horizon, non-separable global mission objectives like field-wide coverage completeness alongside hard physical survival constraints, such as absolute battery depletion limits and mandatory return-to-depot deadlines.

2.4. Positioning Relative to Recent UAV Swarm Optimization Frameworks

This paper addresses these limitations by redefining the role of the UAVs toward precision agriculture tasks. Rather than adapting existing frameworks, we build a scalable, online MAPPO architecture that natively embeds battery mechanics and physical coverage constraints into the optimization loop. The closest prior work to this paper is that of Liu et al. [32], who propose an MARL algorithm with a shared critic for UAV trajectory optimization in CF-mMIMO networks. Their target application is energy-efficiency maximization within a space–air–ground integrated network. Key contributions include a ground user equipment (GUE) grouping scheme based on density distribution and a two-timescale optimization framework that combines SCA with MARL. Despite these merits, their formulation differs from ours in four fundamental respects.

• First, their UAVs act as flying access points serving ground users. In contrast, the UAVs in this paper are mobile remote sensors whose primary function is to collect crop and soil data. This sensing role introduces a coverage completeness requirement that is absent in [32].
• Second, their energy model targets fixed-wing UAVs, which have fundamentally different propulsion characteristics from the rotary-wing platforms prevalent in agricultural settings. This paper adopts the Zeng et al. [21] rotary-wing model with per-UAV battery depletion enforced as a hard mission constraint.
• Third, the formulation of [32] includes no return-to-depot requirement. Such a requirement is a practical necessity in agricultural deployments, where UAVs must land at a charging station upon mission completion.
• Fourth, their objective is purely communication-centric, maximizing energy efficiency defined as throughput per unit energy. This paper instead introduces a joint sensing–communication (SC) utility that simultaneously rewards field coverage and communication efficiency, reflecting the dual purpose of the UAV swarm.

These distinctions produce a fundamentally different Markov Decision Process (MDP) structure, reward design, and set of active constraints, making the proposed framework an innovative extension beyond prior studies.

Table 1. Comparison with closest related works. ✓ indicates that the feature is addressed by the corresponding work; × indicates that it is not.

Feature	[29]	[32]	[34]	This Work
CF-mMIMO	×	✓	✓	✓
UAV trajectory opt.	✓	✓	×	✓
Power control	✓	×	✓	✓
Rotary-wing model	×	×	×	✓
Battery hard constr.	×	×	×	✓
Return-to-depot	×	×	×	✓
Sensing coverage	×	×	×	✓
Joint SC utility	×	×	×	✓
CTDE	×	✓ *	×	✓

* Liu et al. use a shared critic but a different MARL algorithm and objective.

provides the key distinctions between this paper and the closest related papers [29,32,34].

Beyond the CF-mMIMO-specific works discussed above, several recent studies address related but structurally distinct UAV swarm optimization problems. Zhang et al. [37] proposed TTBO-FL, which is a soft-actor–critic framework that jointly optimizes trajectory, local training epochs, and 3D beamforming for energy-efficient federated learning across a two-level UAV hierarchy; unlike our formulation, its objective is federated model accuracy rather than sensing–communication utility, and it does not model a cooperative CF-mMIMO uplink. Mu et al. [38] introduced ABDDPG, combining MADDPG with an attention-biased Transformer for scalable UAV flocking; its attention mechanism allows the actor to condition on a variable-size neighbor set, though ABDDPG (Attention-Biased Deep Deterministic Policy Gradient) targets collision-free flocking rather than joint communication–sensing optimization. Dong et al. [39] proposed a hybrid RL-tuned particle-swarm-optimization (PSO) scheme for a single UAV-assisted MIMO relay, which—being a single-agent, single-UAV formulation—does not address swarm coordination or CF-mMIMO cooperation. Ao et al. [40] proposed GATAC, which is a graph-attention-network multi-agent actor–critic framework for heterogeneous (fixed-wing/rotary-wing) post-disaster search. Its graph-attention encoding lets the policy handle a growing number of agents. Finally, Han et al. [41] solved a static UAV deployment-and-trajectory problem via a bi-level dandelion/iterated-greedy meta-heuristic. Being a non-learning, offline optimization method, it cannot adapt online to time-varying channel conditions or battery state as our CTDE-MAPPO policy does.

Table 2. Comparison of UAV swarm optimization frameworks. ✓ indicates the feature is supported by the corresponding work; ✗ indicates it is not.

Reference	Domain	Learning Paradigm	Joint Action Space	Energy Model	Scales to Larger Swarms?
Zhang et al. (TTBO-FL) [37]	Federated learning over UAV relay swarm	Soft Actor–Critic (single-agent)	Trajectory + local epochs + 3D beamforming	UAV propulsion (unspecified detail)	✗
Mu et al. (ABDDPG) [38]	Flocking/obstacle avoidance	MADDPG + attention-biased Transformer, LoRA fine-tuning	3D trajectory (velocity)	Not modeled	✓ *
Dong et al. [39]	Single UAV-assisted MIMO relay	RL-tuned PSO (hybrid, single-agent)	Trajectory + beamforming/power	Not modeled	N/A (single UAV)
Ao et al. (GATAC) [40]	Heterogeneous post-disaster target search	MARL + Graph Attention Network actor–critic	Trajectory (role-specific: fixed-wing leader/rotor follower)	Multi-rotor + fixed-wing propulsion	✓
Han et al. (bi-level) [41]	Static IoT data collection	Dandelion algorithm + iterated greedy (meta-heuristic, non-RL)	Deployment (footholds) + trajectory	Communication + flight energy	N/A (offline, re-solved per instance)
This paper	CF-mMIMO sensing–communication, precision agriculture	CTDE-MAPPO (on-policy, multi-agent)	3D trajectory + transmit power	Rotary-wing propulsion [21] + Tx power	✓ †

* Attention over neighbors permits a variable-size input, but flocking arrival rate is evaluated, not swarm-size transfer explicitly. ^† Requires a warm-start fine-tuning phase when the swarm size differs from the training configuration (see Section 6.6 and the discussion in Section 7).

summarizes these distinctions.

Individually, CTDE architectures, MAPPO, trajectory–power co-optimization, battery-aware reward shaping, and coverage-driven UAV missions have each been investigated extensively in the literature reviewed above. The contribution of this paper lies in its joint integration into a single formulation tailored to CF-mMIMO-enabled precision agriculture rather than in a fundamentally new theoretical or algorithmic mechanism.

3. CF-mMIMO-Assisted Precision Agriculture: System Model and Assumptions

3.1. CF-mMIMO Network Topology

In Figure 1, we adopt a precision agriculture system in which a swarm of K rotary-wing UAVs, indexed by $\mathcal{K}=\{1,2,\dots ,K\}$, operates over a rectangular agricultural field $\mathcal{F}\subset {\mathbb{R}}^2$ of dimensions $L_x\times L_y$ meters. The field is monitored by a CF-mMIMO network comprising M single-antenna APs, indexed by $\mathcal{M}=\{1,2,\dots ,M\}$, deployed at fixed positions ${\mathbf{b}}_m={[x_m^b,y_m^b,0]}^T$ on the ground. Each AP communicates with a central CPU via dedicated error-free fronthaul links. The entire network is assumed to function over a finite mission horizon of T time slots of equal duration ${\delta }_t$, during which each UAV continuously collects remote sensing data (multispectral imagery, soil moisture readings, vegetation indices) and transmits these data to the AP network in the uplink.

3.2. UAV Mobility Model

At time slot t, the three-dimensional coordinates of UAV k are represented by ${\mathbf{q}}_k\left(t\right)={[x_k\left(t\right),y_k\left(t\right),z_k\left(t\right)]}^T\in {\mathbb{R}}^3$. UAVs are constrained to fly within the altitude range $[H_{min},H_{max}]$, and their horizontal displacement per time slot is bounded by a maximum speed $v_{max}$:

$$∥{\mathbf{q}}_k(t+1)-{\mathbf{q}}_k\left(t\right)∥\le v_{max}{\delta }_t,\forall k,t.$$

(1)

To guarantee collision-free operation, a strict inter-UAV separation threshold $d_{min}$ is imposed at every time slot:

$$∥{\mathbf{q}}_k\left(t\right)-{\mathbf{q}}_{k^{\prime }}\left(t\right)∥\ge d_{min},\forall k\ne k^{\prime },\forall t.$$

(2)

UAVs depart from and return to a designated ground depot ${\mathbf{q}}_0$ so that ${\mathbf{q}}_k\left(1\right)={\mathbf{q}}_k\left(T\right)={\mathbf{q}}_0$ for all k.

3.3. Air-to-Ground Channel Model

The uplink channel between UAV k at position ${\mathbf{q}}_k\left(t\right)$ and AP m at position ${\mathbf{b}}_m$ is modeled as a composite air-to-ground fading channel that accounts for small-scale fading as well as large-scale path loss. The three-dimensional distance from UAV k to AP m at time t is  

$$d_{mk}\left(t\right)=∥{\mathbf{q}}_k\left(t\right)-{\mathbf{b}}_m∥.$$

(3)

The elevation angle from AP m to UAV k is

$${\theta }_{mk}\left(t\right)=arcsin\left({\displaystyle \frac{z_k\left(t\right)}{d_{mk}\left(t\right)}}\right).$$

(4)

Following the probabilistic Line-of-Sight (LoS) model widely adopted in UAV communications literature [42], the probability of a LoS link is given by

$$P_{\mathrm{LoS},mk}\left(t\right)={\displaystyle \frac{1}{1+\alpha exp\left(-\beta \left[{\theta }_{mk}\left(t\right)-\alpha \right]\right)}}.$$

(5)

In this model, $\alpha$ and $\beta$ represent parameters that account for different environmental conditions, including suburban and dense urban landscapes. The large-scale fading coefficient is

$$\begin{array}{cc}\hfill {\beta }_{mk}\left(t\right)& ={\eta }_{\mathrm{LoS}}{P}_{\mathrm{LoS},mk}\left(t\right)+{\eta }_{\mathrm{NLoS}}\left(1-P_{\mathrm{LoS},mk}\left(t\right)\right)\hfill \\ \hfill & -20{log}_{10}\left(d_{mk}\left(t\right)\right)-20{log}_{10}\left(f_c\right)+C_0,\hfill \end{array}$$

(6)

where ${\eta }_{\mathrm{LoS}}$ and ${\eta }_{\mathrm{NLoS}}$ are the mean additional attenuation factors for LoS and NLoS links in dB, $f_c$ is the carrier frequency, and $C_0$ is a constant offset. Here, ${\eta }_{\mathrm{LoS}}$ and ${\eta }_{\mathrm{NLoS}}$ are expressed in dB, and all of the remaining terms are evaluated in dB, so that ${\beta }_{mk}\left(t\right)$ is obtained in dB and then converted to linear scale. The instantaneous uplink channel vector from UAV k to AP m at time slot t is modeled as

$$g_{mk}\left(t\right)=\sqrt{{\beta }_{mk}\left(t\right)}{h}_{mk}\left(t\right),$$

(7)

where $h_{mk}\left(t\right)\sim \mathcal{CN}(0,1)$ represents the small-scale fading coefficient. We assume $h_{mk}\left(t\right)$ to remain constant within each time slot (block fading).

3.4. CF-mMIMO Uplink Transmission

At time slot t, UAV k transmits its sensing data using transmit power $p_k\left(t\right)\in [0,P_{max}]$. The received signal at AP m is

$$r_m\left(t\right)={\displaystyle \sum _{k=1}^K}{g}_{mk}\left(t\right)\sqrt{p_k\left(t\right)}{s}_k\left(t\right)+n_m\left(t\right),$$

(8)

where $s_k\left(t\right)\sim \mathcal{CN}(0,1)$ denotes the unit-power data symbol transmitted by the UAV k. $n_m\left(t\right)\sim \mathcal{CN}(0,{\sigma }_n^2)$ denotes the additive white Gaussian noise (AWGN) at AP m. All AP signals are forwarded to the CPU, which applies a coherent large-scale fading decoding (LSFD) combiner ${\mathbf{v}}_k\left(t\right)\in {\mathbb{C}}^M$ to recover the data of UAV k:

$${\widehat{s}}_k\left(t\right)={\mathbf{v}}_k^H\left(t\right)\mathbf{r}\left(t\right),$$

(9)

where $\mathbf{r}\left(t\right)={[r_1\left(t\right),\dots ,r_M\left(t\right)]}^T$. Under the minimum mean square error (MMSE) combiner, ${\mathbf{v}}_k\left(t\right)$ is given by   

$${\mathbf{v}}_k\left(t\right)=p_k\left(t\right){\left({\displaystyle \sum _{k^{\prime }=1}^K}{p}_{k^{\prime }}\left(t\right){\mathbf{g}}_{k^{\prime }}\left(t\right){\mathbf{g}}_{k^{\prime }}^H\left(t\right)+{\sigma }_n^2{\mathbf{I}}_M\right)}^{-1}{\mathbf{g}}_k\left(t\right),$$

(10)

where ${\mathbf{g}}_k\left(t\right)={[g_{1k}\left(t\right),\dots ,g_{Mk}\left(t\right)]}^T$. The uplink SINR achieved by UAV k at time slot t is given by

$${\gamma }_k\left(t\right)={\displaystyle \frac{p_k\left(t\right){\left|{\mathbf{v}}_k^H\left(t\right){\mathbf{g}}_k\left(t\right)\right|}^2}{{\displaystyle {\displaystyle \sum _{\begin{array}{c}{k}^{\prime }=1\\ k^{\prime }\ne k\end{array}}^K}{p}_{k^{\prime }}\left(t\right){\left|{\mathbf{v}}_k^H\left(t\right){\mathbf{g}}_{k^{\prime }}\left(t\right)\right|}^2+{\sigma }_n^2{∥{\mathbf{v}}_k\left(t\right)∥}^2}}},$$

(11)

and the instantaneous achievable rate is

$$R_k\left(t\right)={log}_2\left(1+{\gamma }_k\left(t\right)\right)[\mathrm{bits}/\mathrm{s}/\mathrm{Hz}].$$

(12)

3.5. Energy Consumption Model

The total energy consumed by UAV k at time slot t comprises both communication energy and propulsion energy:

$$E_k\left(t\right)=p_k\left(t\right){\delta }_t+P_{\mathrm{prop},k}\left(t\right){\delta }_t,$$

(13)

where the propulsion power for a rotary-wing UAV is modeled as [21]

$$\begin{array}{cc}\hfill P_{\mathrm{prop},k}\left(t\right)& =P_0\left(1+{\displaystyle \frac{3v_k^2\left(t\right)}{U_{\mathrm{tip}}^2}}\right)\hfill \\ \hfill & +P_i{\left(\sqrt{1+{\displaystyle \frac{v_k^4\left(t\right)}{4v_0^4}}}-{\displaystyle \frac{v_k^2\left(t\right)}{2v_0^2}}\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{d}_0\rho sA{v}_k^3\left(t\right),\hfill \end{array}$$

(14)

with $v_k\left(t\right)=∥{\mathbf{q}}_k(t+1)-{\mathbf{q}}_k\left(t\right)∥/{\delta }_t$ being the UAV speed, $P_0$ the blade profile power, $P_i$ the induced power in hover, $U_{\mathrm{tip}}$ the rotor tip speed, $v_0$ the mean rotor velocity in hover, $d_0$ the fuselage drag ratio, $\rho$ the air density, s the rotor solidity, and A the rotor disc area. The cumulative energy consumption of the UAV k up to time t is

$$E_k^{\mathrm{cum}}\left(t\right)={\displaystyle \sum _{\tau =1}^t}{E}_k\left(\tau \right),$$

(15)

and the residual battery level is

$$B_k\left(t\right)=B_k^{max}-E_k^{\mathrm{cum}}\left(t\right),$$

(16)

where $B_k^{max}$ is the initial battery capacity of UAV k.

3.6. Sensing Coverage Model

Each UAV k is equipped with a downward-facing multispectral sensor whose footprint on the ground approximates a circular disc. At altitude $z_k\left(t\right)$, the sensing radius is

$$r_k\left(t\right)=z_k\left(t\right)tan(\psi /2),$$

(17)

where $\psi ={120}^{\circ }$ is the sensor’s half-angle field of view. The agricultural field $\mathcal{F}$ is discretized into a uniform grid of $N_c=N_x\times N_y$ cells, each of area ${\delta }_c=(L_x/N_x)(L_y/N_y)$ m². A cell c at position ${\mathbf{p}}_c\in {\mathbb{R}}^2$ is considered sensed at time slot t if at least one UAV is within the following sensing range:   

$$\mathcal{C}\left(t\right)={\displaystyle \bigcup _{k=1}^K}\left\{c:\parallel {\mathbf{q}}_k{\left(t\right)}_{xy}-{\mathbf{p}}_c\parallel \le r_k\left(t\right)\right\},$$

(18)

where ${\mathbf{q}}_k{\left(t\right)}_{xy}={[x_k\left(t\right),y_k\left(t\right)]}^T$ denotes the horizontal coordinates of the k-th UAV. The cumulative coverage fraction at time t is

$$\rho \left(t\right)={\displaystyle \frac{\left|\mathcal{C}\right(t\left)\right|}{N_c}}\in [0,1].$$

(19)

The incremental coverage gained at slot t is $\Delta \rho \left(t\right)=\rho \left(t\right)-\rho (t-1)$ with $\rho \left(0\right)=0$. Language editing and the system model illustration (Figure 1) were assisted by Claude ([Sonnet 5], Anthropic, San Francisco, CA, USA, claude.ai). The authors have reviewed all outputs and take full responsibility for the content of this publication.

4. Joint Optimization Problem

This section formalizes the problem of jointly optimizing UAV trajectories and transmit power in the multi-UAV CF-mMIMO system. Since precision agriculture requires both physical field coverage and reliable data transmission, we begin by defining a utility metric that explicitly balances sensing coverage against communication energy efficiency. We then state the full optimization problem under four operational constraints: finite battery capacity, collision avoidance, minimum uplink rate, and mandatory return-to-depot.

4.1. Joint Sensing–Communication Utility

The system-wide energy efficiency (EE) is defined as the total information delivered by all UAVs per unit of total energy consumed over the mission horizon:

$${\eta }_{\mathrm{EE}}={\displaystyle \frac{{\displaystyle {\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{t=1}^T}{R}_k\left(t\right)}}{{\displaystyle {\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{t=1}^T}{E}_k\left(t\right)}}}[\mathrm{bits}/\mathrm{J}].$$

(20)

Unlike prior studies that restrict their objective to communication energy efficiency, this paper defines a joint sensing–communication utility that reflects the two core objectives of the UAV swarm in precision agriculture. Because the raw energy efficiency ${\eta }_{\mathrm{EE}}$ and the coverage metric possess fundamentally different physical meanings, units, and numerical ranges, optimizing an unscaled linear combination could result in an ill-conditioned formulation that is highly sensitive to slight variations in the weighting parameter. To reconcile these heterogeneous dimensions and ensure optimization robustness, we map both metrics onto a uniform, dimensionless scale. We define the normalized joint utility function as

$$U=\alpha ·\left({\displaystyle \frac{{\eta }_{\mathrm{EE}}}{{\eta }_{\mathrm{EE}}^{max}}}\right)+(1-\alpha )·\overline{\rho }\left(T\right),$$

(21)

where ${\eta }_{\mathrm{EE}}$ [bits/J] is the communication energy efficiency defined in (20), $\overline{\rho }\left(T\right)$ is the normalized cumulative coverage fraction at the end of the mission, and $\alpha \in [0,1]$ is a weighting coefficient that controls the operational priority between communication and sensing objectives. The scaling factor ${\eta }_{\mathrm{EE}}^{max}$ is a normalization constant representing the maximum nominal or theoretical upper-bound energy efficiency achievable under ideal line-of-sight (LoS) channel conditions. By dividing ${\eta }_{\mathrm{EE}}$ by ${\eta }_{\mathrm{EE}}^{max}$, both terms in (21) become dimensionless and strictly bounded within $[0,1]$, preventing either objective from numerically dominating the underlying multi-agent reward structure during policy training. Setting $\alpha =1$ reduces the utility to a pure normalized communication energy efficiency objective that is analogous to the non-normalized formulations in [32]. Setting $\alpha =0$ yields a pure coverage maximization problem. In the precision agriculture context, finding an equilibrium along the Pareto optimal trade-off frontier between these two metrics is essential.

4.2. Optimization Problem

We formulate the joint trajectory and power allocation problem as follows: We formulate the joint trajectory and power allocation problem as follows:

$$\begin{array}{}\mathrm{(22)}& \mathcal{P}:{\displaystyle \underset{\left\{p_k\left(t\right)\right\},\left\{{\mathbf{q}}_k\left(t\right)\right\}}{max}}U\mathrm{(23)}& \mathrm{s}.\mathrm{t}.0\le p_k\left(t\right)\le P_{max},\forall k,t,\mathrm{(24)}& \hspace{1em}\hspace{1em}\hspace{1em}\parallel {\mathbf{q}}_k(t+1)-{\mathbf{q}}_k\left(t\right)\parallel \le v_{max}{\delta }_t,\forall k,t,\mathrm{(25)}& \hspace{1em}\hspace{1em}\hspace{1em}{H}_{min}\le z_k\left(t\right)\le H_{max},\forall k,t,\mathrm{(26)}& \hspace{1em}\hspace{1em}\hspace{1em}\parallel {\mathbf{q}}_k\left(t\right)-{\mathbf{q}}_{k^{\prime }}\left(t\right)\parallel \ge d_{min},\forall k\ne k^{\prime },\forall t,\mathrm{(27)}& \hspace{1em}\hspace{1em}\hspace{1em}{B}_k\left(t\right)\ge 0,\forall k,t,\mathrm{(28)}& \hspace{1em}\hspace{1em}\hspace{1em}{\mathbf{q}}_k\left(1\right)={\mathbf{q}}_k\left(T\right)={\mathbf{q}}_0,\forall k,\mathrm{(29)}& \hspace{1em}\hspace{1em}\hspace{1em}{\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=1}^T}{R}_k\left(t\right)\ge R_{min},\forall k,\mathrm{(30)}& \hspace{1em}\hspace{1em}\hspace{1em}\overline{\rho }\left(T\right)\ge {\rho }_{min}.\end{array}$$

where the following apply:

• ${\rho }_{min}$ is the minimum required field coverage fraction.
• Constraint (23) enforces per-UAV transmit power limits.
• Constraints (24) and (25) enforce mobility and altitude bounds.
• Constraint (26) is the collision avoidance condition.
• Constraint (27) enforces that the battery state cannot fall below zero. This is a physical, state-space invariant enforced by construction in the environment dynamics (Section 5.1.4) rather than a constraint requiring active enforcement by the learned policy. Avoidance of the depletion event itself—i.e., discouraging a UAV from actually reaching zero battery mid-mission—is instead handled through reward shaping, as detailed in Section 5.1.
• Constraint (28) enforces the return-to-depot requirement, which is essential for battery-charged agricultural UAVs but is absent from the formulation of [32].
• Constraints (29) and (30) enforce minimum uplink rate and coverage completeness, respectively.

4.3. Why This Problem Cannot Be Solved by Existing Methods?

Problem $\mathcal{P}$ is non-convex for several reasons: the fractional structure of ${\eta }_{\mathrm{EE}}$, the coupling between trajectory and channel quality in ${\gamma }_k\left(t\right)$, the non-convex coverage set union in (18), and the return-to-depot constraint (28). Non-convexity alone, however, is not the main difficulty. $\mathcal{P}$ is a long-horizon sequential problem: the trajectory chosen at slot t directly affects the battery level, coverage state, and channel quality at every future slot. Any method that optimizes one slot at a time (including SCA and supervised DNNs) cannot account for these dependencies and will therefore fail to find a good policy. The return-to-depot constraint (28) adds a further complication, as it links the start and end of the mission and forces the agent to plan the full trajectory from the outset rather than reacting slot by slot. These properties make Multi-Agent Deep Reinforcement Learning (MADRL) the appropriate choice, as developed in the following section.

5. Proposed MAPPO-Based MADRL Framework

Due to the structural intractability of Problem $\mathcal{P}$ under conventional optimization methods, we reformulate the joint trajectory and power allocation task as an MARL problem. Specifically, our system is modeled as a Partially Observable Markov Decision Process (POMDP) comprising K agents. This section defines the fundamental components of this environment: the local observation space for individual UAVs, the global state utilized by the centralized critic, the continuous action space, and a reward function designed to drive the fleet toward the joint sensing-communication utility.

5.1. Markov Decision Process Formulation

The sequential problem $\mathcal{P}$ is cast as a POMDP with K agents (one per UAV).

5.1.1. Local Observation Space

The k-th agent’s local observation at time t is given by

$${\mathbf{o}}_k\left(t\right)=\left[{\tilde{\mathbf{q}}}_k\left(t\right),{\tilde{B}}_k\left(t\right),{\tilde{\mathit{\beta }}}_k\left(t\right),{\left\{{\tilde{\mathbf{q}}}_{k^{\prime }}\left(t\right)\right\}}_{k^{\prime }\ne k},\Delta \rho \left(t\right),\tilde{t}\right],$$

(31)

where tildes denote normalized quantities, ${\tilde{\mathit{\beta }}}_k\left(t\right)\in {\mathbb{R}}^M$ are log-scale large-scale fading estimates normalized to $[0,1]$, $\Delta \rho \left(t\right)$ is the incremental coverage gained at slot t, and $\tilde{t}=t/T$ is the normalized mission time. Including $\tilde{t}$ is important because it allows the agent to anticipate the return-to-depot deadline and begin navigating homeward before the battery runs out. Per-slot methods have no such capability. We note that ${\mathbf{o}}_k\left(t\right)$ is a decentralized observation space reliant on passive onboard sensing mechanisms (such as radar or visual tracking) rather than active inter-agent communication, preserving the zero-communication overhead protocol during execution.

5.1.2. Global State

The centralized critic receives the full global state:

$$\mathbf{s}\left(t\right)=\left[{\left\{{\tilde{\mathbf{q}}}_k\left(t\right)\right\}}_{k=1}^K,{\left\{{\tilde{B}}_k\left(t\right)\right\}}_{k=1}^K,{\left\{{\tilde{\mathit{\beta }}}_k\left(t\right)\right\}}_{k=1}^K,\rho \left(t\right),\tilde{t}\right].$$

(32)

5.1.3. Action Space

$${\mathbf{a}}_k\left(t\right)=[\Delta x_k,\Delta y_k,\Delta z_k,p_k\left(t\right)]\in {[-1,1]}^4,$$

(33)

with the displacement scaled to $v_{max}{\delta }_t$ and power mapped linearly to $[0,P_{max}]$.

5.1.4. Reward Function

The reward function is the central design contribution of this paper. It is structured to incentivize all terms of the joint utility (21) while enforcing the hard constraints of $\mathcal{P}$:

$$\begin{array}{cc}\hfill r_k\left(t\right)=& \underset{\mathrm{EE}\mathrm{incentive}}{\underbrace{w_1·\alpha ·tanh\left({\eta }_{\mathrm{EE}}\left(t\right)/{\eta }_0\right)}}+\underset{\mathrm{coverage}\mathrm{gain}}{\underbrace{w_2·(1-\alpha )·\Delta \rho \left(t\right)}}\hfill \\ \hfill & +\underset{\mathrm{battery}\mathrm{conservation}}{\underbrace{w_3·{\tilde{B}}_k\left(t\right)}}-\underset{\mathrm{QoS}\mathrm{penalty}}{\underbrace{w_4·⊮[R_k\left(t\right)<R_{min}]}}\hfill \\ \hfill & -\underset{\mathrm{battery}\mathrm{depletion}\mathrm{penalty}}{\underbrace{w_5·⊮[B_k\left(t\right)\le 0]}}-\underset{\mathrm{collision}\mathrm{penalty}}{\underbrace{w_6·⊮\left[\mathrm{collision}\right]}}\hfill \\ \hfill & +\underset{\text{return-to-depot}\mathrm{bonus}}{\underbrace{w_7·⊮[t=T,\parallel {\mathbf{q}}_k\left(T\right)-{\mathbf{q}}_0\parallel \le d_{\mathrm{depot}}]}},\hfill \end{array}$$

(34)

where ${\eta }_0$ is a normalization constant, $w_1,\dots ,w_7>0$ are tunable weights and $d_{\mathrm{depot}}$ is the acceptable landing radius around the depot. The formulation in (34) introduces several design elements that set it apart from prior literature. First, the battery conservation term $w_3{\tilde{B}}_k\left(t\right)$ provides a continuous incentive to preserve energy throughout the mission. It guides the policy to balance aggressive sensing trajectories against the reserve capacity needed for the return flight. This trade-off is absent from [32]. Second, the return-to-depot bonus at $t=T$ enforces constraint (28) through positive reinforcement. This accelerates training convergence compared to a standard terminal penalty alone. The penalty term $w_5$ in Equation  (34) discourages battery depletion during the mission (cf. the hard state-space clipping of constraint  (27) in Section 4.2, which prevents $B_k\left(t\right)$ from becoming negative but does not by itself discourage the UAV from reaching zero battery). Finally, the normalized mission time $\tilde{t}$ is included in the observation space as an implicit deadline signal. Early in the mission, the agent prioritizes coverage. As the deadline approaches, it shifts toward return-trajectory planning.

5.2. MAPPO Algorithm

The proposed solution adopts the Multi-Agent Proximal Policy Optimization (MAPPO) approach under the Centralized Training with Decentralized Execution (CTDE) strategy. Each agent k maintains a local actor network ${\pi }_{{\theta }_k}\left({\mathbf{a}}_k\right|{\mathbf{o}}_k)$, parameterized by ${\theta }_k$, which maps the local observation to a probability distribution through continuous actions. A single centralized critic network $V_{\varphi }\left(\mathbf{s}\right)$, parameterized by $\varphi$, takes the full global state as input and estimates the joint state-value function. This asymmetry between actor (local) and critic (global) is the key feature of CTDE: during training, the critic resolves the non-stationarity caused by simultaneously learning agents; during deployment, each UAV executes its actor independently with no inter-UAV communication overhead. Unlike loosely coupled communication tasks [29], where per-agent decoupled critics may outperform a single centralized critic, the shared coverage map $\rho \left(t\right)$ and collision avoidance constraints in our formulation create genuine strong inter-agent coupling, making the single centralized critic the appropriate architectural choice.

Both networks—actor and critic—are structured as multi-layer perceptrons (MLPs) with two hidden layers of 256 neurons, utilizing ReLU nonlinearities. At the output stage, the actor network applies a Tanh activation, which is appropriately adjusted to conform to the action space constraints. The MAPPO training objective for agent k is the clipped surrogate loss:

$${\mathcal{L}}_k\left({\theta }_k\right)=-\mathbb{E}\left[min\left({\rho }_k\left(t\right){\widehat{A}}_k\left(t\right),\mathrm{clip}\left({\rho }_k\left(t\right),1-ϵ,1+ϵ\right){\widehat{A}}_k\left(t\right)\right)\right],$$

(35)

where the following apply:

• ${\rho }_k\left(t\right)={\pi }_{{\theta }_k}\left({\mathbf{a}}_k\right|{\mathbf{o}}_k)/{\pi }_{{\theta }_k^{\mathrm{old}}}\left({\mathbf{a}}_k\right|{\mathbf{o}}_k)$ is the importance sampling ratio;
• ${\widehat{A}}_k\left(t\right)$ is the Generalized Advantage Estimate (GAE), which is given by

  $${\widehat{A}}_k\left(t\right)={\displaystyle \sum _{l=0}^{T-t}}{\left(\gamma \lambda \right)}^l{\delta }_{t+l},{\delta }_t=r_k\left(t\right)+\gamma V_{\varphi }\left(s(t+1)\right)-V_{\varphi }\left(s\left(t\right)\right);$$
• $ϵ$ is the clipping coefficient.

The critic is trained by minimizing the mean squared Bellman error:

$${\mathcal{L}}_{\mathrm{critic}}\left(\varphi \right)=\mathbb{E}\left[{\left(V_{\varphi }\left(\mathbf{s}\left(t\right)\right)-{\widehat{V}}_k^{\mathrm{target}}\left(t\right)\right)}^2\right].$$

(36)

The complete training procedure is summarized in Algorithm 1.

Algorithm 1 MAPPO Training for Joint Sensing–Communication Optimization in CF-mMIMO UAV Networks

1: Initialize actor networks ${\pi }_{{\theta }_k}$ for all k, shared centralized critic $V_{\varphi }$, Adam optimizers.   2: Initialize CF-mMIMO channel simulator, UAV propulsion energy model, and coverage grid ${\left\{{\mathbf{p}}_c\right\}}_{c=1}^{N_c}$.   3: for episode $=1$ to $N_{\mathrm{ep}}$ do   4:    Reset positions to depot ${\mathbf{q}}_0$, batteries $B_k\leftarrow B_k^{max}$, coverage map $\mathcal{C}\leftarrow Ø$, rollout buffer ${\mathcal{B}}_k\leftarrow Ø$ for all k.   5:    for $t=1$ to T do   6:      Set normalized time $\tilde{t}\leftarrow t/T$; form local observations ${\mathbf{o}}_k\left(t\right)$ (Equation (31)) and global state $\mathbf{s}\left(t\right)$ (Equation (32)).   7:      Compute value estimate: $\widehat{V}\left(t\right)\leftarrow V_{\varphi }\left(\mathbf{s}\left(t\right)\right)$   8:      for each UAV k do   9:         Sample action and log-probability: ${\mathbf{a}}_k\left(t\right),log{\pi }_{{\theta }_k}\sim {\pi }_{{\theta }_k}\left(·|{\mathbf{o}}_k\left(t\right)\right)$; clip to ${[-1,1]}^4$. 10:      end for 11:      Execute joint action; update UAV positions, transmit powers, and batteries via Equations (1)–(16). 12:      Update coverage map $\mathcal{C}\left(t\right)$ via Equation (18); compute incremental gain $\Delta \rho \left(t\right)$. 13:      Query CF-mMIMO uplink: obtain ${\gamma }_k\left(t\right)$ and $R_k\left(t\right)$ via Equations (11) and (12). 14:      Compute per-slot EE: ${\eta }_{\mathrm{EE}}\left(t\right)={\sum }_k{R}_k\left(t\right)/{\sum }_k{E}_k\left(t\right)$ 15:      Compute rewards $r_k\left(t\right)$ for all k via Equation (34), including battery conservation term $w_3{\tilde{B}}_k\left(t\right)$ and terminal depot bonus $w_7{\delta }_k^{\mathrm{dep}}\left(t\right)$. 16:      Store $\left({\mathbf{o}}_k\left(t\right),{\mathbf{a}}_k\left(t\right),r_k\left(t\right),log{\pi }_k,\widehat{V}\left(t\right),\mathbf{s}\left(t\right)\right)$ in ${\mathcal{B}}_k$ for all k. 17:    end for 18:    Bootstrap terminal value: ${\widehat{V}}^{\mathrm{last}}\leftarrow V_{\varphi }\left(\mathbf{s}\left(T\right)\right)$ 19:    for each agent k do 20:      Compute GAE advantages ${\widehat{A}}_k\left(t\right)$ and discounted returns ${\widehat{R}}_k\left(t\right)$ from ${\mathcal{B}}_k$ and ${\widehat{V}}^{\mathrm{last}}$; normalize ${\widehat{A}}_k$. 21:      for each update epoch $=1$ to $N_{\mathrm{epochs}}$ do 22:         Update actor by minimizing clipped surrogate loss with entropy bonus: ${\mathcal{L}}_k\left({\theta }_k\right)$ via Equation (35). 23:         Update shared critic by minimizing Bellman MSE loss: ${\mathcal{L}}_{\mathrm{critic}}\left(\varphi \right)$ via Equation (36). 24:         Clip gradient norms to $0.5$ for both actor and critic. 25:      end for 26:    end for 27:    Compute and log joint utility: $U=\alpha {\eta }_{\mathrm{EE}}+(1-\alpha )\rho \left(T\right)$ (Equation (21)), coverage $\rho \left(T\right)$, and depot-return fraction. 28: end for 29: Output: Trained decentralized actor policies ${\left\{{\pi }_{{\theta }_k^{*}}\right\}}_{k=1}^K$.

6. Simulation Results

This section evaluates the proposed MAPPO framework through numerical simulations. We first describe the simulation setup and benchmark methods; then, we present results across three experiments: training convergence, coverage probability versus SINR threshold, and energy efficiency under varying swarm size. Finally, we analyze the sensitivity to the communication–sensing trade-off coefficient $\alpha$.

6.1. Simulation Setup

We conduct simulations in a custom Python environment that integrates a CF-mMIMO uplink model with a physics-based UAV mobility and energy simulator [21,42]. We model the agricultural field as a 500 m × 500 m area with $M=16$ APs uniformly distributed on a $4\times 4$ grid at height $H_{\mathrm{AP}}=5$ m. Unless stated otherwise, we deploy $K=4$ UAVs over a horizon of $T=200$ time slots with ${\delta }_t=1$ s. Each data point reported in the comparative figures and tables is computed by averaging the corresponding metric over 20 independently seeded evaluation episodes. For each comparison point flagged in the text, we additionally provide the $95\%$ confidence intervals and paired Wilcoxon signed-rank tests reported at the comparison points discussed in the text. 

Table 3. Simulation parameters.

Parameter	Symbol	Value
Field dimensions	$L_x\times L_y$	500 m × 500 m
Number of APs	M	16
Number of UAVs	K	4
Carrier frequency	$f_c$	2.4 GHz
Noise power	${\sigma }_n^2$	$-96$ dBm
Max UAV power	$P_{max}$	200 mW
UAV battery	$B_k^{max}$	50 kJ
Max UAV speed	$v_{max}$	15 m/s
Altitude range	$[H_{min},H_{max}]$	$[20,100]$ m
Min separation	$d_{min}$	10 m
Mission horizon	T	200 s
Min rate	$R_{min}$	1 bit/s/Hz
MAPPO clip	$ϵ$	0.2
Discount factor	$\gamma$	0.99
GAE parameter	$\lambda$	0.95
Training episodes	$N_{\mathrm{ep}}$	5000

details the principal simulation parameters used in this paper.

6.2. Experimental Protocol and Hyperparameter Configuration

6.2.1. Network Architecture and Initialization

Both the actor and centralized critic are implemented as multi-layer perceptrons with two hidden layers of 256 units each, using Tanh activations at every hidden layer. The actor outputs a diagonal Gaussian policy: a Tanh-squashed mean and a state-independent, per-dimension log-standard-deviation parameter, which is clamped to $[-2.0,1.0]$ during training. All hidden layers are orthogonally initialized with gain $\sqrt{2}$; the actor’s output (mean) layer uses gain $0.01$ to avoid early saturation of the policy distribution, while the critic’s output layer uses gain $1.0$.

6.2.2. Optimization and Exploration Schedule

Both networks are trained with Adam, using an actor learning rate of $3\times {10}^{-4}$ and a critic learning rate of $1\times {10}^{-3}$, each decayed via cosine annealing over the training horizon to a floor of ${\mathrm{lr}}_{\mathrm{actor}}/30$ and ${\mathrm{lr}}_{\mathrm{critic}}/10$, respectively. The PPO clipping parameter is $ϵ=0.2$, the GAE parameter $\lambda =0.95$, and the discount factor $\gamma =0.99$, with 10 update epochs per episode over mini-batches of size 64. The entropy coefficient is annealed linearly from $0.03$ to $0.003$ over the course of training to balance early exploration against late-stage policy sharpening.

6.2.3. Reward Weight Configuration and Tuning

The step-by-step per-agent reward function uses the following explicit weight configuration: $w_1=2.0$ (energy-efficiency incentive), $w_2=1.5$ (coverage gain), $w_3=0.1$ (battery conservation), $w_4=1.0$ (QoS violation penalty), $w_5=3.0$ (battery depletion penalty), $w_6=1.0$ (collision penalty), and $w_7=10.0$ (return-to-depot terminal bonus). These scalar weights were tuned manually through an iterative, reward-decomposition-based diagnostic process rather than an automated hyperparameter search. At each tuning iteration, the cumulative episodic return was decomposed into its isolated constituent terms. This allowed us to systematically track and evaluate the numerical trajectory of each penalty and reward component, enabling the manual adjustment of values that either dominated the gradient or became negligible relative to the primary energy-efficiency and coverage signals.

6.2.4. Training Budget and Checkpoint Selection

The training process is executed over a fixed budget of 5000 episodes at $T=200$ time steps each (1,000,000 environment steps total) rather than an early-stopping criterion. Because the joint utility does not increase monotonically over training, the actor and critic weights are checkpointed at every episode that improves on the best joint utility observed so far. This best-performing checkpoint—not the final-episode weights—is restored prior to evaluation to guarantee testing stability.

6.2.5. Computational Infrastructure

All simulation models and training pipelines were implemented in Python 3.13 using PyTorch 2.11.0 and executed within a cloud-hosted JupyterHub cluster. The allocated hardware resource configuration consists of 16 CPU cores, $48\mathrm{GiB}$ of system RAM, and $two$ dedicated NVIDIA L4 GPUs (each providing $24\mathrm{GB}$ of VRAM).

6.3. Impact Analysis of Proposed Framework Components

Table 4. Ablation study quantifying the contribution of each model component to joint utility (U), coverage ratio ($\rho \left(T\right)$), and depot-return rate, which were evaluated at $K=4$ UAVs and SNR $=10$ dB across $N=20$ independent seeds. Each entry reports the mean ± 95% confidence interval; the $\alpha$ column lists the utility weight used for that variant.

Variant	$\mathit{\alpha }$	U	$\mathit{\rho }\left(\mathit{T}\right)$	Depot
EE only	1	0.418 ± 0.028	0.68 ± 0.042	0.88 ± 0.058
Coverage only	0	0.391 ± 0.024	0.95 ± 0.015	0.91 ± 0.061
w/o battery term ($w_3=0$)	0.6	0.431 ± 0.033	0.89 ± 0.026	0.38 ± 0.047
w/o mission time $\tilde{t}$	0.6	0.428 ± 0.029	0.91 ± 0.022	0.32 ± 0.053
w/o shared critic (indep. PPO)	0.6	0.445 ± 0.035	0.90 ± 0.031	0.85 ± 0.049
Proposed: Full model	0.6	0.531 ± 0.021	0.93 ± 0.018	0.92 ± 0.031

Boldface indicates the proposed (full) model.

reports the impact of each design component of the proposed framework on the three primary metrics: joint utility U, field coverage $\rho \left(T\right)$, and depot-return fraction. Each row removes exactly one component from the full proposed scheme while keeping all others fixed. In this section, to calculate the joint utility U, we choose $\alpha =0.6$ for all variants.

The ablation results in Table 4 reveal a critical design insight: the 54-percentage-point collapse in depot-return fraction when $w_3=0$ (from 92% to 38%) demonstrates that the terminal return bonus $w_7$ alone is insufficient to enforce mission feasibility under hard battery constraints. Without a continuous energy-preservation signal throughout the mission, the agent aggressively pursues coverage gains in early time slots, depleting its battery before it can execute the return trajectory—a reward hacking failure mode in which the agent maximizes the terminal bonus probability by exploiting the coverage reward, inadvertently violating the battery constraint in 62% of episodes. The removal of the normalized mission time $\tilde{t}$ from the observation produces a comparable depot-return collapse (to 32%), confirming that the agent requires both a continuous energy signal and an explicit deadline horizon to plan feasible return trajectories. Taken together, $w_3{\tilde{B}}_k\left(t\right)$ and $\tilde{t}$ constitute the mission-feasibility mechanism of the proposed framework; neither component alone is sufficient, and removing either one renders the learned policy operationally infeasible in the majority of episodes.

6.4. Convergence of MAPPO Training

Figure 2 shows the normalized cumulative reward per episode for MAPPO ($K=4$) and the single-agent DRL baseline. MAPPO converges within approximately 2500 episodes, after which the reward curve stabilizes with low variance—a direct consequence of the centralized critic resolving inter-agent non-stationarity during training. The single-agent baseline plateaus at a reward approximately 35% lower, confirming that independent PPO agents, operating without shared value estimation, cannot resolve the credit-assignment problem in a cooperative multi-UAV environment. Notably, the variance band of the single-agent baseline remains wide throughout training, indicating persistent instability rather than a stable suboptimal policy.

Training does not proceed monotonically. Because the centralized critic is trained on a target distribution that shifts as every agent’s policy shifts, the joint utility we observe during training tends to rise for a while and then degrade under continued updates—a moving-target effect that is well known in on-policy multi-agent RL. In practice, this means the policy sitting at the end of the training run is not necessarily the best one produced along the way. Evaluating only the final checkpoint, as is common practice, can therefore understate what the policy is actually capable of. We address this with three mechanisms built into the training loop. The first is best-checkpoint tracking: at every episode, we check the joint utility $\mathcal{U}$ against the best value seen so far, and if it improves, we save the actor and critic weights. It is this saved checkpoint, not the final one, that we restore before evaluation. Second, we anneal the learning rates of the actor and critic with a cosine schedule over training, so updates become smaller as training progresses and the policy is less likely to overshoot past a good solution late in training. Third, the PPO entropy coefficient is annealed linearly from a higher to a lower value, which keeps exploration alive early on while letting the policy settle into something closer to deterministic by the end. None of this amounts to a convergence guarantee—non-convex, multi-agent, continuous-control problems like ours do not admit one—but the fact that these mechanisms are needed at all is itself informative: it shows that training in this setting is not simply well behaved by default, and that naively taking the last checkpoint would report a weaker policy than the one actually learned.

6.5. Uplink SINR Coverage Analysis

Figure 3 presents the empirical uplink SINR coverage probability $\mathbb{P}({\gamma }_k>{\gamma }_{\mathrm{th}})$ in terms of the required SINR threshold ${\gamma }_{\mathrm{th}}$ for $K=4$ UAVs. $\mathbb{P}({\gamma }_k>{\gamma }_{th})$ is estimated from the per-UAV SINR samples accumulated over 50 independent channel realizations per episode. To isolate the respective contributions of AP cooperation and trajectory adaptability, the proposed MAPPO over CF-mMIMO is benchmarked against three baselines:

• Static lawnmower (CF-mMIMO): UAVs follow fixed parallel sweep trajectories with equal power allocation while retaining the full cell-free AP cooperation. This baseline isolates the gain of the adaptive trajectory.
• MADRL (Cellular mMIMO): The same MAPPO policy is deployed over a non-cooperative cellular architecture, where each UAV is served by its single strongest AP. This baseline isolates the gain of cell-free cooperation.
• Cellular mMIMO: A non-cooperative single-AP network is used that has no learned trajectory optimization.

In Figure 3, the proposed MAPPO over the CF-mMIMO network achieves the highest coverage at every threshold, sustaining near-unity probability up to ${\gamma }_{\mathrm{th}}\approx 22$ dB, with a median SINR of approximately 48 dB. Replacing CF cooperation with a cellular architecture while retaining MADRL reduces the median SINR to $38.3$ dB—a degradation of approximately 10 dB. This confirms that the coverage gain stems jointly from the learned policy and the distributed AP architecture with neither factor alone being sufficient. Specifically, the static lawnmower with CF reception sits below both MADRL-enhanced variants at high thresholds, with a median SINR of $33.7$ dB. Such a result indicates that while AP cooperation recovers substantial spatial diversity, the learned adaptive trajectory and power control account for the remaining $14.3$ dB gain relative to the static baseline. Removing AP cooperation entirely (single-AP baseline) incurs the steepest penalty: the median SINR collapses to $-8.5$ dB. This highlights the indispensability of cell-free operation for reliable uplink reception in UAV-dense deployments.

6.6. System Energy Efficiency

Figure 4 illustrates the system energy efficiency ${\eta }_{\mathrm{EE}}$ as a function of the UAV swarm size K at an average $\mathrm{SNR}=0\mathrm{dB}$. To evaluate our proposed MAPPO approach, we consider five distinct baselines:

• Supervised DNN: a neural network trained via behavioral cloning using SCA-generated optimal power labels at $K=4$.
• Static Lawnmower: a non-adaptive baseline where UAVs follow pre-computed parallel sweep trajectories with a fixed power allocation of $p_k=P_{max}/2,\forall k$.
• SCA: an iterative successive convex approximation algorithm with perfect instantaneous CSI and a fixed lawnmower trajectory. SCA assumes perfect instantaneous CSI and optimizes transmit power only along the same fixed lawnmower trajectory used by the static lawnmower baseline; it therefore bounds fixed-trajectory power-control methods and is not a bound on the joint trajectory-and-power problem addressed by MAPPO.
• SCMA-MADDPG: the MARL-based scheme proposed in [32], which optimizes for communication metrics without sensing awareness. This baseline was reimplemented (not taken from the original paper), under our own environment, sharing our network architecture and training procedure, differing only in the reward function, with hyperparameters matched to the proposed method for fairness.
• Parameter-Shared MAPPO: a parameter-shared MAPPO baseline, trained under the same procedure, budget, and hyperparameters as our proposed method, differing only in that its actor weights are shared across all UAVs rather than trained independently.

The shaded region denotes the 95% confidence interval calculated across 20 independent random seeds for the proposed MAPPO approach; baseline variance envelopes are omitted exclusively to preserve visual clarity. As seen from Figure 4, the system energy efficiency decreases monotonically with K for all schemes, since the co-channel interference among UAVs intensifies. For $K<8$, the SCA and the static lawnmower methods dominate. This crossover arises from the interaction between inter-UAV interference and the exploration burden of joint trajectory-power learning. At low swarm sizes, co-channel interference is weak, so a near-optimal, low-variance power allocation along a fixed trajectory (SCA) or a simple, deterministic, full-coverage sweep (lawnmower) already achieves high energy efficiency: neither baseline is handicapped by exploration noise nor must it learn the larger joint (position, power) action space that MAPPO optimizes simultaneously. As K increases, interference comes to dominate the achievable energy efficiency, and the ability to actively reposition UAVs to reduce mutual interference—a capability the fixed-trajectory baselines structurally lack—becomes decisive. This explains the crossover near $K\approx 8$: MAPPO trails SCA and the static lawnmower baseline at $K=4$ (1.408 vs. 1.684 and 1.544 Mbits/J, respectively) but leads every baseline from $K=8$ onward with the gap widening substantially by $K=16$ (0.997 vs. 0.774 SCA and 0.588 static). This reflects the agent’s ability to learn radio-aware flight patterns: by coordinating the 3D position alongside the transmit power, the UAVs actively minimize interference in a way non-cooperative or static methods cannot—an advantage that is particularly evident relative to the SCA baseline, which optimizes power only.

Moreover, the DNN approach suffers a severe performance collapse beyond its training point ($K=4$), dropping to $472.7$ kbits/J at $K=12$ ($-58.1\%$ relative to MAPPO). This is a direct result of a distribution shift caused by an observation–dimension mismatch at $K\ne 4$. Furthermore, the proposed MAPPO outperforms SCMA-MADDPG [32], whose reward is limited to communication EE. Without a coverage objective, the agents in [32] tend to cluster near the access points offering the strongest channel gains, which intensifies mutual co-channel interference as K grows and ultimately degrades the very EE metric they seek to maximize. In contrast, the joint reward in MAPPO enforces spatial dispersion of the swarm, implicitly reducing inter-UAV interference and sustaining higher per-agent SINR across all values of K. The proposed MAPPO also outperforms the parameter-shared MAPPO baseline across all tested swarm sizes, with the gap widening from a marginal 0.005–0.02 Mbit/J at K = 4–8 to a more pronounced $0.06$ Mbit/J at $K=16$, suggesting that per-agent actor specialization becomes increasingly valuable as swarm size—and the coordination burden it imposes—grows.

It is important to clarify the nature of this cross-K evaluation: the policy trained at $K=4$ is not applied zero-shot to other swarm sizes. Because each agent’s observation and the centralized critic’s global state both scale in dimension with K (and similarly with AP density M), a change in K requires instantiating a new policy, transferring weights only for the subset of UAVs present in the original swarm, and a short warm-start fine-tuning phase (approximately 10% of the original training budget) before evaluation. All cross-K results reported above for $K\ne 4$ were obtained after this warm-start procedure. This distinction is important: the reported scalability with K (Table 5) refers to computational/execution scaling for a fixed, already-trained architecture, not to zero-shot representational generalization across swarm sizes. Field size and battery capacity, by contrast, do not alter observation or global-state dimensionality and so require no architectural retraining, though some fine-tuning remains advisable when these change substantially, since the value ranges the policy was trained on shift accordingly.

To evaluate how the network scales architecturally, Figure 5 illustrates the system energy efficiency of the proposed MAPPO approach as a function of the number of deployed access points (M) across different swarm sizes (K). Two distinct trends emerge from this analysis. First, fixing the user load and scaling M from 8 to 32 yields a steady improvement in energy efficiency. This initial gain highlights the benefits of the cell-free architecture, where increasing the spatial density of APs brings the infrastructure physically closer to the flying UAVs, mitigating severe path loss and providing strong macro-diversity that the MAPPO framework exploits for smarter power allocation. However, this trend hits an inflection point at $M=32$, after which the energy efficiency begins to degrade as the network scales up to 64 APs. This drop illustrates a classic diminishing returns threshold: beyond $M=32$, the incremental throughput gains unlocked by additional spatial multiplexing are entirely eclipsed by the surging energy costs of operating the extra hardware circuits and handling the denser fronthaul signaling. Second, looking across the curves, scaling the swarm size from $K=20$ to $K=50$ consistently shifts the entire performance curve downward. This behavior is expected, as a higher density of concurrently transmitting UAVs inevitably creates a more intense mutual interference environment, driving up the aggregate network power expenditure required to satisfy user data rate constraints.

6.7. Sensitivity to the Communication–Sensing Trade-Off Coefficient $\alpha$

To evaluate how the trade-off between sensing and communication objectives impacts overall system performance, we analyze the sensitivity of the MAPPO framework by varying the weighting coefficient $\alpha$ from $0.0$ to $1.0$ in steps of $0.2$. As illustrated in Figure 6, the joint utility U follows a distinct concave trajectory at $\mathrm{SNR}=10$ dB, highlighting the inefficiencies of optimizing for a single objective. When $\alpha =0.0$, the network operates in a pure-sensing mode. The UAVs prioritize expansive, exploratory trajectories to maximize field coverage, but they completely neglect communication performance, resulting in a low joint utility of approximately $0.39$. Conversely, at the other extreme ($\alpha =1.0$), the framework focuses solely on communication. In this mode, the policy drives the UAVs to hover near the access point (AP) clusters to maximize macro-diversity and uplink throughput, which leaves the wider field unmonitored and drops the joint utility to around $0.42$. The best performance is achieved in the intermediate range where $0.4\le \alpha \le 0.6$. The joint utility peaks at $\alpha =0.6$, reaching approximately $0.53$. This demonstrates that the MAPPO framework successfully balances the competing objectives, capturing the synergy between field coverage and data transmission. This predictable, smooth response curve indicates that network operators can easily fine-tune fleet behavior to match specific mission requirements simply by adjusting $\alpha$ without needing to restructure the underlying reward mechanics.

6.8. Battery-Coverage Dynamics

The temporal dynamics illustrated in Figure 7 demonstrate the trade-off between field coverage acceleration and battery preservation over the normalized mission horizon $\tilde{t}=t/T$. During the early-to-mid stage of the mission ($0.0\le \tilde{t}\le 0.5$), the multi-UAV fleet rapidly expands across the search region, causing the global coverage fraction $\rho \left(t\right)$ to rise sharply from $0.0$ to approximately $0.85$. Concurrently, all four UAVs exhibit near-identical, monotonic battery depletion rates, dropping smoothly from full capacity ($1.0$) down to roughly $0.40$ near $\tilde{t}=0.70$. This synchronized energy consumption indicates that the learned policy balances work allocation across the agents without over-burdening any single UAV. As the mission progresses into the shaded Emergent Return Triggering Zone ($\tilde{t}\ge 0.70$), marginal coverage gains naturally saturate at $\rho \left(t\right)\approx 0.93$. Rather than aggressively pursuing the final $7\%$ of the unmapped peripheral area—which would incur high energy risk—the trained policy shifts its primary objective toward homeward transit. This behavioral shift is emergent rather than hard-coded: the agents leverage the normalized time signal $\tilde{t}$ within their local observations to initiate their return flight, arriving safely at the depot $q_0$ with a reserve energy margin of 10–15%. Reserving this $30\%$ time window ($\tilde{t}\in [0.7,1.0]$) provides a necessary physical buffer to handle kinetic return maneuvers and potential environmental disturbances such as headwind resistance.

6.9. 3D Spatial Trajectories and Territory Partitioning

Figure 8 visualizes the multi-agent spatial flight paths generated in 3D space ($X,Y,Z$) alongside their corresponding swept sensor footprints projected on the ground plane. Upon departure from the central depot $q_0(0,0,0)$, the multi-agent policy exhibits emergent territorial partitioning: each UAV assigned to a specific quadrant sweeps its designated subregion while operating at distinct operational altitudes ranging between $20\mathrm{m}$ and $100\mathrm{m}$. This spatial segregation minimizes inter-agent sensor redundancy and eliminates mid-air collision risks during the intensive coverage phase. The structure of individual trajectories reflects a clear two-phase operational pattern. First, during the sensing phase, UAVs perform loitering patterns to maximize the local footprint coverage over target subfields, as evidenced by the dashed circular projections on the ground plane. Second, upon reaching the temporal return threshold ($\tilde{t}\approx 0.7$), the agents cease loitering and execute near-linear, direct vector trajectories straight back to $q_0$. This transition back to direct lines minimizes the flight path distance during the final leg, ensuring that kinetic energy is conserved when flying home from distant field boundaries.

7. Discussion

7.1. Computational Complexity and Scalability Analysis

To contextualize the proposed framework within the broader landscape of existing methods, we characterize each algorithm’s computational complexity in terms of the following:

• The number of UAVs K;
• The number of APs M;
• The mission horizon T;
• The hidden-layer width H; and
• The number of PPO update epochs $N_e$.

The local observation dimension is $d_{\mathrm{obs}}=3+1+M+3(K-1)+3=M+3K+4$ (position, battery, large-scale fading, relative positions of $K-1$ neighbors, incremental coverage, normalized time), and the global state dimension is $d_{\mathrm{gs}}=K(M+4)+2$ (concatenation of all UAV states, coverage fraction, and normalized time). Table 5 summarizes the training and execution complexities, CSI requirements, and scalability properties.

Table 5 shows that the proposed MAPPO framework provides a balance between training expressiveness and execution efficiency through the CTDE paradigm. During training, the CF-mMIMO CPU hosts a centralized critic $V_{\varphi }\left(\mathbf{s}\right)$ whose input dimension $d_{\mathrm{gs}}=K(M+4)+2$ grows linearly with both K and M. This global view resolves the non-stationarity caused by simultaneously learning agents, facilitating stable GAE training [43]. The dominant training cost is the per-slot MMSE matrix inversion, $\mathcal{O}\left(M^3\right)$, which is unavoidable for any method that faithfully simulates the CF-mMIMO uplink, yielding a total training budget of $\mathcal{O}(N_{\mathrm{ep}}·N_e·T·(M^3+K{H}^2))$. Once training is complete, the critic is discarded. Each UAV k then runs only its local actor ${\pi }_{{\theta }_k}\left({\mathbf{a}}_k\right|{\mathbf{o}}_k)$, which is a two-hidden-layer MLP requiring $\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$ operations per slot. Since the K actors run in parallel with no inter-UAV communication at execution time, the total cost scales linearly with K.

Table 5. Computational complexity and scalability comparison (K UAVs, M APs, T slots, H hidden neurons, $N_e$ update epochs per episode, $I_{\mathrm{SCA}}$ iterations per slot).

Algorithm	Training Complexity	Execution Complexity	CSI Needed	Scales with K	Adaptive
Proposed MAPPO	$\mathcal{O}\left(N_{\mathrm{ep}}{N}_{e}T(M^3+K{H}^2)\right)$	$\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$	Local ${\tilde{\mathit{\beta }}}_k$ only	Linear	Online
SCA (Perfect CSI)	—	$\mathcal{O}\left(T{I}_{\mathrm{SCA}}(K^3+M^3)\right)$	Full inst. CSI	Cubic	Offline
Supervised DNN	$\mathcal{O}\left(N_{\mathrm{demo}}{d}_{\mathrm{obs}}{H}^2\right)$	$\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$	Local ${\tilde{\mathit{\beta }}}_k$ only	Fixed	Static
Static Lawnmower	$\mathcal{O}\left(KT\right)$ precompute	$\mathcal{O}\left(K\right)$ lookup	None	Trivially	None
SCMA-MADDPG [32]	$\mathcal{O}\left(N_{\mathrm{ep}}{N}_{e}T(M^3+K{H}^2)\right)$	$\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$	Local ${\tilde{\mathit{\beta }}}_k$ only	Linear	Online

Table 5. Computational complexity and scalability comparison (K UAVs, M APs, T slots, H hidden neurons, $N_e$ update epochs per episode, $I_{\mathrm{SCA}}$ iterations per slot).

Algorithm	Training Complexity	Execution Complexity	CSI Needed	Scales with K	Adaptive
Proposed MAPPO	$\mathcal{O}\left(N_{\mathrm{ep}}{N}_{e}T(M^3+K{H}^2)\right)$	$\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$	Local ${\tilde{\mathit{\beta }}}_k$ only	Linear	Online
SCA (Perfect CSI)	—	$\mathcal{O}\left(T{I}_{\mathrm{SCA}}(K^3+M^3)\right)$	Full inst. CSI	Cubic	Offline
Supervised DNN	$\mathcal{O}\left(N_{\mathrm{demo}}{d}_{\mathrm{obs}}{H}^2\right)$	$\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$	Local ${\tilde{\mathit{\beta }}}_k$ only	Fixed	Static
Static Lawnmower	$\mathcal{O}\left(KT\right)$ precompute	$\mathcal{O}\left(K\right)$ lookup	None	Trivially	None
SCMA-MADDPG [32]	$\mathcal{O}\left(N_{\mathrm{ep}}{N}_{e}T(M^3+K{H}^2)\right)$	$\mathcal{O}\left(K{d}_{\mathrm{obs}}H\right)$	Local ${\tilde{\mathit{\beta }}}_k$ only	Linear	Online

The SCA baseline, by contrast, requires a perfect instantaneous CSI at every slot and re-solves a fractional program with per-episode complexity $\mathcal{O}\left(T{I}_{\mathrm{SCA}}(K^3+M^3)\right)$ —cubic in K—which may limit real-time deployment beyond moderate swarm sizes. The supervised DNN matches MAPPO’s execution cost but is trained at a fixed fleet size and cannot generalize to other values of K without full retraining, as reflected by the distribution-shift collapse in Figure 4. The static lawnmower achieves the lowest execution cost $\mathcal{O}\left(K\right)$ via a precomputed trajectory, at the cost of any adaptivity to battery state, channel variations, or dynamic field conditions. While the SCMA-MADDPG baseline [32] operates within the same complexity class as MAPPO, its optimization objective is limited to the EE reward component. In a sensing-heavy application like precision agriculture, this narrow focus encourages “greedy” behavior where UAVs hover near the strongest APs to conserve energy, ultimately leaving large areas of the field unsurveyed. Because the computational cost of [32] is effectively identical to ours, it becomes clear that our approach’s advantage lies in its joint reward design—which balances coverage with communication—rather than raw architectural capacity. Overall, the proposed MAPPO-CTDE framework combines scalable execution, online adaptability, and low deployment overhead while jointly optimizing energy efficiency and sensing coverage. From a deployment perspective, the computational requirements of the proposed actor network are modest. Since inference consists primarily of straightforward matrix multiplications within a small neural network, the associated processing energy is expected to be negligible compared to the energy consumed by mechanical propulsion and wireless transmission. In addition, because execution relies on locally observed large-scale fading information rather than instantaneous CSI, each UAV can continue operating even when communication with the network is temporarily unavailable. This feature improves operational robustness compared to fully centralized control schemes. Looking forward, while the current state representation effectively scales linearly with the swarm size K and the number of APs M, it assumes a rigid topological input structure. Because $d_{\mathrm{obs}}$ explicitly concatenates the relative positions of $K-1$ neighbors, deploying a pre-trained policy on a dynamically resizing swarm—where UAVs might enter or exit the active fleet mid-mission for emergency recharging—remains challenging. Future extensions of this paper could integrate attention mechanisms into the actor network to process a variable-length sequence of neighboring states. This would reduce the network architecture’s dependence on a fixed K, allowing for seamless, mid-flight swarm elasticity while maintaining the robust coverage and energy-efficiency benefits demonstrated by our current MAPPO approach.

7.2. Battery-Depletion Avoidance: Reward Shaping Versus Hard Constraints

Our battery-depletion avoidance mechanism is enforced only through reward shaping (Section 5.1.4) rather than a hard safety layer, so it can reduce but not strictly guarantee zero probability of depletion during execution; a formally constrained or safe-RL formulation—e.g., a control-barrier-function or Lagrangian-constrained-MDP layer, along the lines of Zhang et al. [36]—would be needed to provide a probabilistic or worst-case depletion guarantee, and we identify this as a promising direction for future work.

7.3. Agricultural Field Realism

The evaluation environment models the agricultural field as a uniform 500 m × 500 m area with a homogeneous coverage grid, in which every cell carries equal sensing priority. This abstraction does not incorporate real farm topography, crop-type-dependent revisit priorities, or heterogeneous sensing-demand maps that would arise in an operational deployment. This simplification was adopted deliberately to isolate the joint trajectory–power–coverage optimization problem in a controlled, reproducible setting that is decoupled from confounds specific to any single field geometry or crop layout. Validating the proposed framework on real agricultural field geometries and heterogeneous priority maps—for instance, sensing-demand maps derived from satellite NDVI imagery to reflect crop-health-dependent revisit priorities—is an important direction for future work.

7.4. Discussion on Modeling Assumptions and Practical Limitations

While the proposed framework establishes a foundational paradigm for joint sensing-and-communication (SC) optimization in CF-mMIMO-assisted UAV swarms, several modeling assumptions represent an optimistic baseline relative to a real agricultural deployment. We discuss these limitations along two axes: the wireless channel model and the sensing model.

7.4.1. Wireless Channel Model

The air-to-ground channel adopted in this paper (Section 3.3) assumes error-free fronthaul links between the APs and the CPU together with a standard probabilistic LoS/NLoS model governed by fixed environmental parameters $\alpha ,\beta$. A real agricultural deployment introduces several impairments this formulation does not capture. Canopy density varies both spatially across a field and temporally across the growing season, producing foliage attenuation that a static LoS probability cannot represent; incorporating empirical vegetation-attenuation models, such as those standardized by the ITU-R, would be a natural refinement. Terrain morphology—furrows, slopes, irrigation infrastructure—can introduce scattering and shadowing beyond what the elevation-angle-dependent LoS probability in Equation (5) accounts for, and weather conditions such as rain or fog further attenuate the link without being modeled here. The fronthaul itself is assumed to be instantaneous and lossless; a finite-capacity, delay-constrained fronthaul, more representative of rural deployment infrastructure would introduce additional latency and potential packet loss into the uplink pipeline. Extending the channel model along these lines, particularly with a season- and terrain-aware attenuation term, is an important step toward field-validated performance figures. We note that the deployed policy’s information requirements are already limited to large-scale fading knowledge ${\tilde{\beta }}_k$ rather than instantaneous CSI (Section 5.1.1), so the imperfect CSI concern specifically applies to the fidelity of the training and evaluation channel simulator rather than to the execution-time policy. Extending the evaluated envelope to estimation-noise-corrupted large-scale fading and fronthaul-constrained operation is a natural and scoped next step.

7.4.2. Sensing Model Fidelity and Practical Limitations

The geometric footprint model used for coverage (Section 3.6) is computationally convenient for trajectory optimization, but it abstracts away several factors that govern actual crop-monitoring quality in the field:

• Altitude-Dependent Resolution (GSD): Ground Sampling Distance scales directly with UAV altitude $z_k$. Flying near $H_{max}$ maximizes geometric coverage but coarsens spatial resolution, so fine-grained tasks such as early-stage weed identification or localized crop-stress classification demand a lower operating altitude—a direct trade-off between coverage speed and imaging quality that our model does not currently arbitrate.
• Image Overlap Requirements: Photogrammetric pipelines for orthomosaic generation typically require 70–80% frontal and lateral image overlap. Our coverage metric treats sensing as an instantaneous geometric footprint, whereas a deployable system would need to coordinate flight speed, camera trigger rate, and inter-path spacing to guarantee this overlap—a constraint that limits how abruptly the trajectory can change direction.
• Environmental and Atmospheric Factors: We assume clear-sky, static imaging conditions. In practice, wind-induced attitude oscillations (yaw, pitch, roll) cause gimbal jitter and motion blur, while cloud cover, shadowing, and diurnal solar angle shift illumination and introduce radiometric noise into the sensing pipeline.
• Probabilistic Sensing Uncertainty: Rather than the binary disc model in Equation (18), real sensor detection probability decays with distance from the camera’s nadir point rather than dropping sharply at the footprint boundary. Incorporating a probabilistic sensing kernel, together with robust optimization under wind-induced trajectory uncertainty, is a promising direction for future work.

7.5. Coverage and Depot-Return Gains

We note that the reported $93\%$ coverage and $92\%$ depot-return figures are metrics of the simulated mission under the idealized channel, propulsion, and sensing models discussed above, and they do not directly equate to “mission success” in an operational agricultural sense, which would additionally require accounting for heterogeneous sensing priorities across crop zones, obstacle avoidance (e.g., trees, structures, other aircraft), weather-induced flight disruptions, and mid-mission replanning capability—none of which are modeled in the current framework. We frame closing this gap via integration with a higher-level mission-planning/replanning layer as an important direction for translating the current trajectory-and-power optimization layer into a deployable end-to-end system.

7.6. Analysis of Simulation-to-Reality (Sim-to-Real) Gaps

While our custom simulation environment establishes a mathematically rigorous baseline for evaluating the joint optimization framework, we acknowledge that the lack of direct hardware-in-the-loop (HIL) testing or platform-specific validation introduces a distinct Simulation-to-Reality (Sim-to-Real) gap. The theoretical propulsion model derived from [21] assumes steady-state flight dynamics in an unperturbed environment. In a practical field deployment, several unmodeled physical disturbances will inevitably affect the reported performance metrics, specifically the peak 93% field coverage completeness and 92% communication energy efficiency achieved by our MADRL framework. These core discrepancies are categorized as follows:

• Stochastic Wind Disturbances: Real-world agricultural environments are subject to unpredictable wind vectors and localized thermal updrafts. To maintain a stable hover or track the target 3D trajectory under wind resistance, the UAV’s flight controller must constantly apply rapid, high-frequency motor adjustments. This continuous attitude correction drastically scales up the aerodynamic power consumption relative to the idealized, smooth propulsion equations, accelerating battery depletion and prematurely forcing a return-to-depot sequence. This would directly contract the available mission horizon, likely degrading the 93% field coverage completeness threshold.
• GPS Localized Noise and Sensor Drift: Standard onboard Global Positioning System (GPS) receivers exhibit inherent positioning errors and multi-path fading, resulting in a localized noise envelope (typically $\pm 1$ to 3 m). Because our decentralized policy maps continuous spatial observations directly to actions, positional drift introduces state estimation errors. This error forces the UAVs into jagged, suboptimal trajectory corrections, causing overlapping camera footprints and slight misalignments with the cell-free APs. This spatial jitter would introduce a minor performance penalty across both objectives, shaving percentage points off the peak 92% communication efficiency due to suboptimal beamforming and uplink channel matching.
• Non-Linear Battery Discharge and Internal Resistance: Our model utilizes an idealized, linear energy consumption model where remaining capacity is proportional to power drawn over time. In real lithium-polymer (LiPo) cells, the discharge curve is highly non-linear, and it is characterized by a rapid voltage drop near the end of the discharge cycle and severe internal resistance losses (I²R heating) under peak current loads (e.g., during aggressive high-speed climbing maneuvers). Neglecting these electrochemical dynamics means our simulator under-represents the risk of voltage sags under low-charge states, which would force the safety-aware policy to adopt a more conservative return-to-depot margin in reality, slightly reducing overall mission utility.

Acknowledging these factors, the reported 93% and 92% efficiency bounds should be viewed as an optimistic upper envelope. Bridging this Sim-to-Real gap through domain randomization (injecting synthetic wind and GPS noise distributions into the MADRL training phase) and HIL validation represents a crucial vector for our ongoing research.

8. Conclusions

This paper tackled the problem of jointly optimizing UAV trajectories and transmit power for rotary-wing UAV swarms operating over a CF-mMIMO network in a precision agriculture scenario. Unlike existing studies that either optimize power control for static aerial users or trajectory for communication energy efficiency alone, we introduced a joint sensing–communication utility function and enforced several operational constraints absent from existing formulations: transmit power limits, mobility and altitude bounds, collision avoidance, hard battery depletion, minimum uplink rate, and mandatory return-to-depot. We developed a MAPPO-based MADRL methodology under the CTDE paradigm, where the CF-mMIMO CPU acted as a global critic during training. At execution time, each UAV ran an independent local policy, adjusting its trajectory and transmit power in real time based on its battery state and current channel conditions. We designed the reward function to simultaneously incentivize communication energy efficiency, incremental coverage gain, battery conservation, and return to the depot before battery depletion. Numerical results confirmed the effectiveness of our proposed joint utility function, showing that the framework balanced real-time scalability and multi-objective optimization. By dynamically adapting to the interaction between communication quality and sensing coverage, the proposed approach maintained high energy efficiency and reliable full-field coverage while satisfying hardware and safety constraints.

Several directions can be explored in future work to further improve the proposed framework and facilitate its deployment in real-world agricultural environments.

• Heterogeneous Fleet Optimization and Asymmetric Dynamics: While the proposed framework focuses on homogeneous rotary-wing UAVs, large-scale agricultural monitoring may benefit from heterogeneous UAV swarms. An interesting extension is to support mixed fleets consisting of fixed-wing UAVs for long-range coverage and rotary-wing UAVs for high-resolution sensing tasks. Such a setting introduces heterogeneous flight dynamics, distinct energy consumption models, and a more complex joint action space. Future work will investigate hierarchical coordination and reward design strategies to enable efficient cooperation among different UAV types.
• Privacy-Preserving Federated Multi-Farm Learning: Another promising direction is the integration of Federated Learning (FL) to enable collaborative training across multiple farms while keeping local operational data private. In this setting, CF-mMIMO CPUs deployed at different farms can serve as local FL clients, performing policy updates using local data and sharing only model parameters or gradients with a central aggregation server. Future research will also explore differential privacy techniques to enhance data protection while maintaining learning performance.