PrivAgriVolt: Privacy-Preserving Shadow-Aware Vision for Crop Stress Diagnosis in Agrivoltaic Photovoltaic Systems

Abstract

Agrivoltaic systems co-locate photovoltaic (PV) arrays and crops, offering land-use efficiency and potential microclimate benefits, yet they introduce new challenges for computer-vision-based crop monitoring. PV structures produce strong, spatially varying shadows, specular reflections, and periodic occlusions that confound visual cues for diagnosing crop diseases and abiotic stresses. Meanwhile, agrivoltaic deployments are often distributed across farms and operators, making centralized data collection impractical due to privacy, ownership, and regulatory concerns. This paper proposes PrivAgriVolt, a novel privacy-preserving learning framework for agrivoltaic crop issue recognition that explicitly models PV-induced illumination and enables collaborative training without sharing raw images. The core algorithm integrates (i) a PV-geometry-conditioned shadow normalization module that fuses estimated array layout and sun-angle priors into a shadow-aware appearance canonization network, reducing illumination-induced domain shift across times and sites; (ii) a federated contrastive stress learner that aligns stress semantics across farms via prototype-based contrastive objectives while remaining robust to heterogeneous sensors and crop stages; and (iii) an adaptive privacy layer that combines secure aggregation with budget-aware gradient perturbation and client-level clipping to provide formal privacy guarantees while preserving fine-grained diagnostic performance. Extensive experiments on real agricultural vision benchmarks and agrivoltaic shadow variants demonstrate that PrivAgriVolt improves stress recognition and segmentation under PV shading while maintaining strong privacy–utility trade-offs.

1. Introduction

Agrivoltaic (AV) or agro-photovoltaic systems co-locate photovoltaic (PV) arrays with crops to increase land-use efficiency while potentially improving water use and microclimate management. Recent systematic reviews emphasize that crop outcomes under PV shading are highly context-dependent and closely tied to array geometry and local climate [1,2,3]. As deployments scale, operators increasingly require continuous, fine-grained monitoring of crop conditions to detect diseases and abiotic stresses early, quantify spatial variability, and adjust management strategies. Computer vision offers a low-cost and scalable solution because images can be collected by handheld devices, fixed cameras, and UAVs. However, the AV setting differs fundamentally from conventional open-field monitoring: PV structures introduce strong and structured shadows, periodic occlusions, and specular reflections whose patterns change with array layout and solar position. These effects distort color and texture cues and create large appearance shifts across time and sites, often exceeding the domain gap observed in standard agricultural benchmarks. Recent shadow surveys and challenge reports underscore that complex cast-shadows remain difficult to handle robustly in the wild [4,5,6].

A second barrier is privacy. AV deployments are naturally distributed: multiple farms, operators, and integrators collect imagery under different business constraints. Centralizing images is frequently infeasible due to concerns over farm ownership, sensitive management practices, and regulatory requirements. Federated learning (FL) mitigates this by training models without moving raw data [7], but practical FL pipelines remain vulnerable to leakage through updates unless combined with secure aggregation and differential privacy (DP) [8,9,10]. Moreover, AV heterogeneity (crop types, phenology, camera hardware, and PV geometry) amplifies non-IID effects that degrade naïve FL optimization, especially when DP noise is applied.

This paper addresses these challenges and asks: Can we build a collaborative vision system that (i) remains accurate under PV-induced shadow dynamics, (ii) generalizes across heterogeneous AV sites, and (iii) provides principled privacy protection without sharing images? We propose PrivAgriVolt, a privacy-preserving, shadow-aware learning framework for AV crop stress diagnosis. In our formulation (Section 3), each client k holds a private dataset ${\mathcal{D}}_k$ of images x with labels $\mathbf{y}$ for classification or segmentation, along with a compact PV geometry descriptor ${\mathbf{g}}^{\left(k\right)}$ and sun-direction prior $\mathbf{s}\left(t\right)$ from capture time t. These signals induce a soft shadow mask $\mathbf{m}$ that conditions a learned canonization module before prediction by $f_{\mathit{\theta }}$.

PrivAgriVolt introduces three algorithmic ideas (Section 4). First, we design a PV-geometry-conditioned shadow normalization module that uses $(\mathbf{m},{\mathbf{g}}^{\left(k\right)},\mathbf{s}\left(t\right))$ to produce a canonized image $\tilde{\mathbf{x}}$ while preserving stress evidence. Unlike generic shadow removal pipelines, the normalization is explicitly conditioned on PV layout and solar geometry, targeting structured AV shadows and reducing time-of-day shift. Second, we introduce a federated prototype contrastive stress learner that aligns representations across sites using securely aggregated class prototypes, improving cross-client generalization under non-IID distributions. Third, we incorporate an adaptive privacy layer that combines secure aggregation with client-level DP on updates, scheduling noise across rounds to improve utility at fixed privacy budgets. Our experiments use real, publicly available agricultural datasets—PlantVillage [11], PlantDoc [12], and Agriculture-Vision [13]—and construct PV-shadowed AV variants to systematically evaluate robustness to structured shading. Results show that PrivAgriVolt improves leave-one-client-out generalization under AV shadows and maintains strong privacy–utility trade-offs compared to standard FL baselines.

In summary, this work makes the following contributions:

• We formulate privacy-preserving AV crop stress recognition and segmentation under PV-induced structured shadows, integrating PV geometry and solar priors into the learning pipeline.
• We propose a novel, PV-geometry conditioned shadow canonization module with a stress-preserving shadow-consistency regularizer that reduces AV illumination shift without erasing diagnostic cues.
• We introduce a federated prototype-contrastive alignment mechanism that improves cross-site generalization, and we integrate it with a client-level DP layer and secure aggregation for end-to-end privacy protection.
• We provide extensive evaluations on real agricultural datasets and AV-shadowed variants, including ablations and privacy–utility analyses under concrete $(\epsilon ,\delta )$ budgets.

The paper is organized as follows: Section 2 reviews related work; Section 3 presents preliminaries and notation; Section 4 details the proposed method; Section 5 reports experiments; Section 6 concludes.

2. Related Work

2.1. Agrivoltaics and Monitoring in PV–Agriculture Co-Location

Agrivoltaics has attracted sustained interest as a dual-use approach to address land competition between PV deployment and agriculture. Recent systematic reviews synthesize evidence that agronomic outcomes and yield stability depend strongly on crop species, local climate, and PV configuration choices [1,2,14]. Modeling, simulation, and optimization have become central for design-time prediction of shading, microclimate, PV yield, and agronomic performance, with comprehensive reviews highlighting the need for integrated platforms and standardized validation across configurations [3,15]. Practice-oriented reports further summarize performance indicators, monitoring needs, and operational considerations for real deployments [16,17]. These directions motivate field-ready sensing and learning systems that are robust to PV-induced illumination structure.

2.2. Agricultural Computer Vision and Domain Shift Under Challenging Illumination

Agricultural computer vision has advanced rapidly with public datasets and deep architectures. PlantVillage enabled early progress in leaf disease recognition under controlled settings [11], while PlantDoc introduced in-the-wild imagery with complex backgrounds and illumination [12]. At field scale, Agriculture-Vision provides aerial images with pixel-level annotations for agricultural pattern segmentation [13]. Recent surveys and studies document the shift from CNN-based pipelines to transformer-era models and stress the persistence of domain shift caused by sensing, phenology, and illumination changes [18,19,20]. For segmentation in agricultural remote sensing, recent reviews summarize architectural trends and remaining gaps in cross-scene generalization [21]. Challenge reports and solutions for Agriculture-Vision further emphasize class imbalance and robustness issues in real-world aerial settings [22]. These results collectively indicate that illumination-structured domain shifts (such as those induced by PV arrays) require task-aware normalization and cross-site generalization mechanisms.

2.3. Shadow-Aware Vision and Privacy-Preserving Federated Learning

Shadows remain a classical obstacle in vision, and recent work consolidates deep-learning progress via surveys and standardized comparisons, spanning shadow detection, removal, and generation [4,23]. The NTIRE shadow removal challenge reports provide rigorous evaluations and show that complex cast shadows remain difficult under both fidelity and perceptual tracks [5,6]. However, generic shadow removal can inadvertently suppress subtle symptoms (e.g., texture and chromatic cues), motivating task-constrained shadow normalization in agricultural diagnosis.

For privacy, federated learning enables collaborative training without sharing raw data [7]. Yet model updates can leak information, and recent surveys synthesize attack surfaces, defenses, and policy implications, emphasizing that privacy-by-design is insufficient without explicit protections [10,24]. Secure aggregation provides cryptographic protection for individual updates [8], while DP provides formal leakage bounds through randomized perturbation [9,25]. Recent protocol proposals and efficient secure aggregation designs further improve practicality in resource-constrained settings [26,27,28]. Recent DPFL reviews and methods further study privacy budget management and utility preservation under heterogeneity [29,30,31,32]. Finally, generalization under heterogeneous clients connects to federated domain generalization (FedDG), which has matured through benchmarks and new algorithmic designs [33,34,35]. Recent FedDG methods explore prompt-based and generative strategies [36,37] and new regularization viewpoints [38,39]. PrivAgriVolt builds on these trends by introducing PV-geometry-conditioned shadow canonization and prototype-based contrastive alignment under an explicit client-level DP layer, targeting the distinctive illumination and privacy constraints of agrivoltaic monitoring.

In agricultural and natural environments, classical preprocessing pipelines based on color constancy, Retinex correction, histogram equalization, handcrafted texture/color descriptors, and rule-based segmentation remain valuable because they are interpretable and inexpensive [40,41,42,43]. However, agrivoltaic PV shadows are structured, geometry-dependent, and strongly spatially varying, so purely fixed enhancement often either under-corrects deep cast shadows or suppresses subtle lesion cues together with the illumination artifact. This is why we adopt a lightweight deep normalization module but constrain it with explicit PV geometry, sun priors, and mask conditioning so that the correction remains physically guided rather than purely black-box.

3. Preliminaries

We formalize the agrivoltaic vision task, introduce PV-induced shadow notation, and summarize the privacy-preserving federated learning primitives used throughout the paper. All symbols defined here are used consistently in later sections.

3.1. Problem Setup and Notation

An agrivoltaic site consists of crop rows co-located with photovoltaic (PV) arrays. We consider K data owners (farms or operators) participating as clients in collaborative training, indexed by $k\in \{1,\dots ,K\}$. Client k holds a private dataset

$${\mathcal{D}}_k={\left\{({\mathbf{x}}_i^{\left(k\right)},{\mathbf{y}}_i^{\left(k\right)})\right\}}_{i=1}^{N_k},$$

(1)

where $\mathbf{x}\in {\mathbb{R}}^{H\times W\times 3}$ is an RGB image captured under PV structures (e.g., UAV or ground camera), and $\mathbf{y}$ denotes supervision for one or more tasks. We focus on two standard agricultural vision tasks:

• Stress recognition (classification): $\mathbf{y}\in \{1,\dots ,C\}$ indicates one of C crop conditions (healthy or stress categories).
• Stress localization (segmentation): $\mathbf{y}\in {\{0,1,\dots ,C\}}^{H\times W}$ is a dense label map, where zero denotes background/unknown.

To avoid repeatedly branching notation, we write a unified predictor

$$f_{\mathit{\theta }}:\mathbf{x}\mapsto \widehat{\mathbf{y}},$$

(2)

with parameters $\mathit{\theta }$. For classification, $f_{\mathit{\theta }}\left(\mathbf{x}\right)$ outputs a probability vector $\widehat{\mathbf{p}}\in {\Delta }^C$; for segmentation it outputs $\widehat{\mathbf{P}}\in {\Delta }_{H\times W}^{C+1}$. We also use an encoder $e_{\mathit{\theta }}$ to map an image to a representation $\mathbf{z}\in {\mathbb{R}}^d$:

$$\mathbf{z}=e_{\mathit{\theta }}\left(\mathbf{x}\right),d\ll HW.$$

(3)

In the experiments, the encoder is instantiated as ResNet-50, so the default embedding dimension used for prototype alignment is $d=2048$ after global pooling. Let $\ell (·,·)$ denote a generic supervised loss (cross-entropy for classification or pixel-wise cross-entropy for segmentation). The standard empirical risk on client k is

$${\mathcal{L}}_k\left(\mathit{\theta }\right)={\displaystyle \frac{1}{N_k}}\sum _{(\mathbf{x},\mathbf{y})\in {\mathcal{D}}_k}\ell \left(f_{\mathit{\theta }}\left(\mathbf{x}\right),\mathbf{y}\right).$$

(4)

In general, agrivoltaic learning faces two characteristic sources of shift: (i) illumination and shadow variation caused by PV structures, changing with time and geometry; (ii) site heterogeneity across clients (crop varieties, growth stages, sensors). We therefore explicitly define PV-shadow variables next.

3.2. PV-Geometry and Shadow Formation

We describe a compact representation of PV-induced shadows used to condition shadow-aware normalization later. For each site (client) k, we denote an approximate PV geometry descriptor by

$${\mathbf{g}}^{\left(k\right)}=\left({\mathbf{n}}^{\left(k\right)},h^{\left(k\right)},{\rho }^{\left(k\right)},{\alpha }^{\left(k\right)}\right),$$

(5)

where ${\mathbf{n}}^{\left(k\right)}\in {\mathbb{R}}^3$ is the unit normal of the PV panel plane, $h^{\left(k\right)}$ is the panel clearance height, ${\rho }^{\left(k\right)}$ is the inter-row spacing (effective pitch), and ${\alpha }^{\left(k\right)}$ is the panel tilt/azimuth parameterization (any consistent 1–2D encoding is admissible). For an image captured at time t, we denote the sun-direction unit vector (in the site coordinate frame) by $\mathbf{s}\left(t\right)\in {\mathbb{R}}^3$. PV structures cast shadows that can be abstracted as a binary or soft shadow mask $\mathbf{m}\in {[0,1]}^{H\times W}$ aligned with image coordinates:

$$\mathbf{m}=\mathcal{S}\left({\mathbf{g}}^{\left(k\right)},\mathbf{s}\left(t\right),\Pi \right),$$

(6)

where $\mathcal{S}(·)$ is a shadow projection operator and $\Pi$ denotes camera projection parameters (intrinsics/extrinsics). In practice, $\mathcal{S}$ may be approximated by (i) coarse ray-casting from simplified PV planes, (ii) learned shadow estimation, or (iii) a hybrid approach that uses geometry as a prior. We do not assume perfect geometry; instead, $\mathbf{m}$ is treated as an informative conditioning signal.

To capture illumination differences between shaded and non-shaded regions, we define a simple image decomposition:

$$\mathbf{x}={\mathbf{x}}_{\mathrm{lit}}\odot (1-\mathbf{m})+{\mathbf{x}}_{\mathrm{sh}}\odot \mathbf{m}+\mathit{\eta },$$

(7)

where ⊙ is element-wise multiplication broadcast over channels, ${\mathbf{x}}_{\mathrm{lit}}$ and ${\mathbf{x}}_{\mathrm{sh}}$ are latent appearances under lit/shaded conditions, and $\mathit{\eta }$ aggregates noise (sensor noise, specularities, occlusions). Equation (7) motivates learning a mapping that reduces shadow-induced appearance shift while preserving stress cues (texture, lesion patterns, chlorosis).

3.3. Federated Optimization and Privacy Primitives

We use synchronous federated learning (FL) to train a shared model without centralizing raw images. Let ${\mathit{\theta }}^{\left(r\right)}$ be global parameters at communication round r. A subset of clients ${\mathcal{S}}_r\subseteq \{1,\dots ,K\}$ participates in round r. Each selected client performs E steps of local optimization (e.g., SGD) on its private data, producing an update $\Delta {\mathit{\theta }}_k^{\left(r\right)}$. The server aggregates updates to form the next global model:

$${\mathit{\theta }}^{(r+1)}={\mathit{\theta }}^{\left(r\right)}+\sum _{k\in {\mathcal{S}}_r}{w}_k\Delta {\mathit{\theta }}_k^{\left(r\right)},w_k={\displaystyle \frac{N_k}{{\sum }_{j\in {\mathcal{S}}_r}{N}_j}}.$$

(8)

To prevent the server from inspecting individual client updates, FL may employ a secure aggregation protocol that reveals only the (weighted) sum ${\sum }_{k\in {\mathcal{S}}_r}{w}_k\Delta {\mathit{\theta }}_k^{\left(r\right)}$ while keeping each $\Delta {\mathit{\theta }}_k^{\left(r\right)}$ hidden.

In this paper, we use the standard definition of $(\epsilon ,\delta )$-differential privacy (DP). A randomized mechanism $\mathcal{M}$ is $(\epsilon ,\delta )$-DP if for any neighboring datasets $\mathcal{D}$ and ${\mathcal{D}}^{\prime }$ that differ by one sample, and any measurable set $\mathcal{O}$,

$$\mathrm{Pr}\left[\mathcal{M}\left(\mathcal{D}\right)\in \mathcal{O}\right]\le e^{\epsilon }\mathrm{Pr}\left[\mathcal{M}\left({\mathcal{D}}^{\prime }\right)\in \mathcal{O}\right]+\delta .$$

(9)

In gradient-based training, a common DP mechanism clips per-client updates and adds Gaussian noise:

$${\widetilde{\Delta \mathit{\theta }}}_k^{\left(r\right)}=\mathrm{clip}\left(\Delta {\mathit{\theta }}_k^{\left(r\right)},C\right)+\mathcal{N}\left(\mathbf{0},{\sigma }^2{C}^2\mathbf{I}\right),$$

(10)

where $C>0$ is a clipping norm, $\sigma$ is the noise multiplier, and $\mathrm{clip}(\mathbf{u},C)=\mathbf{u}·\min (1,C/{\parallel \mathbf{u}\parallel }_2)$. Privacy accounting across rounds yields a total budget $(\epsilon ,\delta )$ for the training process; we report these budgets in Section 5 using standard composition accounting.

The combination of secure aggregation (to hide individual updates) and DP noise (to bound information leakage from the aggregated signal) provides a practical privacy baseline for collaborative agrivoltaic vision learning. The next section builds on these preliminaries to introduce the proposed shadow-aware, privacy-preserving algorithm.

4. Methodology

4.1. End-to-End Pipeline and Notation

PrivAgriVolt addresses agrivoltaic learning in a cross-silo federated setting, where K clients (farms, sites, or geographic tiles) collaboratively train a shared model without exchanging raw images. Client k holds a private dataset ${\mathcal{D}}_k=\left\{(\tilde{\mathbf{x}},\mathbf{y})\right\}$, where $\tilde{\mathbf{x}}$ denotes the observed image under agrivoltaic illumination (potentially PV-shadowed; in our synthetic variants, it is the renderer output), and $\mathbf{y}$ is the corresponding class label (classification) or pixel label map (segmentation). Each client also has site metadata describing the PV installation (e.g., row pitch/orientation/height) summarized as a geometry descriptor ${\mathbf{g}}^{\left(k\right)}$, and a capture timestamp t from which the local sun direction $\mathbf{s}\left(t\right)$ can be derived. These metadata are lightweight and do not reveal raw imagery; we use them only to condition shadow normalization.

PrivAgriVolt factorizes the predictor into two trainable parts: (i) a shadow canonization module ${\mathcal{N}}_{\mathit{\psi }}$ that reduces PV-induced illumination shift, and (ii) a task predictor $f_{\mathit{\theta }}$ for classification or segmentation. The task predictor contains an encoder $e_{\mathit{\theta }}$ whose embeddings are used both for supervised prediction and for cross-client prototype alignment. In addition to model parameters $(\mathit{\theta },\mathit{\psi })$, the server maintains a compact global state consisting of class prototypes $\mathcal{P}={\left\{{\mathbf{p}}_c\right\}}_{c=1}^C$.

Training proceeds in synchronized rounds. In each round, clients locally (a) estimate PV-shadow structure and canonize inputs, (b) optimize a supervised loss augmented by shadow-invariance and prototype alignment, and (c) transmit only encrypted, privacy-protected updates. The server aggregates updates using secure aggregation and updates prototypes from aggregated sufficient statistics. The remainder of this section details (1) PV-conditioned canonization, (2) prototype-based alignment, and (3) privacy-preserving optimization that couples the two.The overall end-to-end pipeline of the proposed PrivAgriVolt framework is illustrated in Figure 1.

Figure 1. End-to-end pipeline of PrivAgriVolt. Each client starts from a local PV-shadowed image $\tilde{\mathbf{x}}$, site geometry ${\mathbf{g}}^{\left(k\right)}$, timestamp-derived sun direction $\mathbf{s}\left(t\right)$, and a locally estimated shadow mask $\mathbf{m}$. The shadow normalization module ${\mathcal{N}}_{\mathit{\psi }}$ produces a canonized image $\widehat{\mathbf{x}}$, the encoder/predictor extracts task features and predictions, local supervised plus shadow-consistency and prototype losses are optimized on-device, and only clipped/encrypted model updates and aggregated prototype statistics are sent to the server for secure aggregation and global prototype updates. No raw image, mask, or pixel content leaves the client.

4.2. PV-Conditioned Shadow Canonization with Consistency Regularization

PV-row shadows introduce structured, quasi-periodic attenuation that varies with array layout and solar position, often causing two images of the same stress to appear more different than two images of different stresses captured under similar shading. This illumination-driven covariate shift is especially harmful in federated learning because each client can observe a distinct shadow distribution, amplifying representation drift and destabilizing aggregation. We therefore normalize illumination before learning stress semantics.

4.2.1. Geometry- and Time-Conditioned Canonization

Given an observed PV-shadowed input $\tilde{\mathbf{x}}$, a shadow mask $\mathbf{m}$, and a conditioning vector $\mathbf{u}$ derived from PV geometry and sun direction, we compute a canonized image:

$$\widehat{\mathbf{x}}={\mathcal{N}}_{\mathit{\psi }}\left(\tilde{\mathbf{x}},\mathbf{m},\mathbf{u}\right),\mathbf{u}=\mathcal{E}\left({\mathbf{g}}^{\left(k\right)},\mathbf{s}\left(t\right)\right)\in {\mathbb{R}}^q,$$

(11)

where $\mathcal{E}$ is a lightweight encoder (e.g., an MLP) that embeds PV geometry ${\mathbf{g}}^{\left(k\right)}$ and sun direction $\mathbf{s}\left(t\right)$ into $\mathbf{u}$. The mask $\mathbf{m}\in {[0,1]}^{H\times W}$ is computed locally (e.g., via a geometry-based estimator, a local shadow detector, or their combination); $\mathbf{m}$ never leaves the client. Intuitively, $\mathbf{m}$ specifies where attenuation occurs, while $\mathbf{u}$ specifies how attenuation should be corrected under site- and time-dependent PV configurations.

For all synthetic PV-shadow experiments in Section 5, we use the renderer-generated soft mask as the conditioning signal $\mathbf{m}$, so the masking strategy is exact and fully reproducible. For real deployment, we use a hybrid local estimator: a coarse geometry prior is first obtained by ray-casting the PV rows described by ${\mathbf{g}}^{\left(k\right)}$ under the sun vector $\mathbf{s}\left(t\right)$ and camera model $\Pi$; this prior is then refined by a lightweight local shadow detector based on RGB intensity/chromaticity cues, and the two estimates are fused into a soft mask by confidence-weighted averaging plus small morphological smoothing. This entire procedure is executed locally on the client, so neither the mask nor any intermediate image leaves the site. In practice, the method is more tolerant to moderate boundary misalignment than to large false negatives inside dark PV bands, because correction is soft and the shadow-consistency loss averages over regions. We therefore prefer soft masks, temporal smoothing when consecutive frames are available, and detector-only fallback when geometry metadata are incomplete.

4.2.2. Conditioned Illumination Field and Stabilized Retinex Correction

Motivated by the decomposition in Equation (7), we predict an illumination map $\mathbf{L}\in {\mathbb{R}}^{H\times W\times 3}$ and correct the input using a stabilized Retinex-like division:

$$\mathbf{L}=\mathrm{Softplus}\left({\mathcal{U}}_{\mathit{\psi }}(\tilde{\mathbf{x}},\mathbf{m};\mathbf{u})\right),\widehat{\mathbf{x}}=\mathrm{clip}\left({\displaystyle \frac{\tilde{\mathbf{x}}}{\mathbf{L}+ϵ}},0,1\right),$$

(12)

with small $ϵ>0$. The $\mathrm{Softplus}(·)$ nonlinearity ensures $\mathbf{L}$ is positive, and clipping keeps outputs within the valid range. In practice, ${\mathcal{U}}_{\mathit{\psi }}$ is implemented as a lightweight U-Net that ingests the RGB image and mask (e.g., concatenated channels). The mask guides the network to focus correction on shadowed regions while keeping confidently lit regions near identity. When $\mathbf{m}$ is close to zero everywhere (weak or no PV shadow), the module naturally behaves like an identity mapping, avoiding unnecessary color distortions.

The modulation by $\mathbf{u}$ is implemented via feature-wise affine conditioning (FiLM) inside ${\mathcal{U}}_{\mathit{\psi }}$:

$$\mathrm{FiLM}(\mathbf{h};\mathbf{u})=\gamma \left(\mathbf{u}\right)\odot \mathbf{h}+\mathit{\beta }\left(\mathbf{u}\right),$$

(13)

where $\mathbf{h}$ is an intermediate feature tensor and $\gamma (·),\mathit{\beta }(·)$ are learned projections. This conditioning injects PV layout and solar priors into the correction process, improving stability across times of day and across sites with different PV geometries.

4.2.3. Shadow-Consistency Regularization

Illumination correction can over-normalize and wash out subtle stress cues (e.g., lesion boundaries, vein discoloration, fine texture). To preserve semantics while enforcing invariance to shading, we impose a representation-level constraint between lit and shaded regions after canonization. Let $\mathbf{F}=e_{\mathit{\theta }}\left(\widehat{\mathbf{x}}\right)\in {\mathbb{R}}^{h\times w\times d}$ be the last convolutional feature map of the encoder. We compute region-aggregated embeddings

$${\mathbf{z}}_{\mathrm{sh}}={\displaystyle \frac{{\sum }_p{\mathbf{m}}_p{\mathbf{F}}_p}{{\sum }_p{\mathbf{m}}_p+ϵ}},{\mathbf{z}}_{\mathrm{lit}}={\displaystyle \frac{{\sum }_p(1-{\mathbf{m}}_p){\mathbf{F}}_p}{{\sum }_p(1-{\mathbf{m}}_p)+ϵ}},$$

(14)

where p indexes spatial positions in the $h\times w$ feature grid (with $\mathbf{m}$ downsampled accordingly). The shadow-consistency loss is

$${\mathcal{L}}_{\mathrm{sc}}=1-{\displaystyle \frac{{\mathbf{z}}_{\mathrm{sh}}^{\top }{\mathbf{z}}_{\mathrm{lit}}}{{\parallel {\mathbf{z}}_{\mathrm{sh}}\parallel }_2{\parallel {\mathbf{z}}_{\mathrm{lit}}\parallel }_2+ϵ}}.$$

(15)

This reduces incentives for client-specific shortcuts tied to shading patterns and improves stability when shadow coverage differs across clients. After canonization, the remaining challenge is non-illumination heterogeneity; we address it by aligning representations with global class prototypes.

4.3. Federated Prototype Contrastive Alignment for Stress Semantics

Even after canonization, substantial heterogeneity remains across clients due to cultivar differences, camera pipelines, phenological stages, and background variation. Naively averaging model updates can then yield a compromise representation that generalizes poorly to unseen clients. PrivAgriVolt mitigates this by aligning client embeddings to a global prototype set that can be aggregated as compact centroids.

4.3.1. Prototype-Contrastive Objective

At round r, the server maintains prototypes ${\mathcal{P}}^{\left(r\right)}={\left\{{\mathbf{p}}_c^{\left(r\right)}\right\}}_{c=1}^C$, where each ${\mathbf{p}}_c^{\left(r\right)}\in {\mathbb{R}}^d$ is a global embedding centroid for class c. Given ${\mathcal{P}}^{\left(r\right)}$, client k computes embeddings $\mathbf{z}=e_{\mathit{\theta }}\left(\widehat{\mathbf{x}}\right)$ and optimizes a prototype-based InfoNCE loss:

$${\mathcal{L}}_{\mathrm{pc}}(\mathbf{z},y;{\mathcal{P}}^{\left(r\right)})=-\log {\displaystyle \frac{\exp (\mathrm{sim}(\mathbf{z},{\mathbf{p}}_y^{\left(r\right)})/\tau )}{{\sum }_{c=1}^C\exp (\mathrm{sim}(\mathbf{z},{\mathbf{p}}_c^{\left(r\right)})/\tau )}},$$

(16)

where $\mathrm{sim}(\mathbf{a},\mathbf{b})={\displaystyle \frac{{\mathbf{a}}^{\top }\mathbf{b}}{{\parallel \mathbf{a}\parallel }_2{\parallel \mathbf{b}\parallel }_2+ϵ}}$ and $\tau >0$ is a temperature. For segmentation, we apply Equation (16) to pixel embeddings ${\mathbf{F}}_p$ using pixel label $y_p$ and exclude background from the prototype set.

4.3.2. Secure Prototype Aggregation

Clients update prototypes using only aggregated sufficient statistics. Client k forms

$${\mathbf{s}}_{k,c}=\sum _{(\tilde{\mathbf{x}},\mathbf{y})\in {\mathcal{D}}_k}\sum _{p\in \Omega (\mathbf{y},c)}{\mathbf{v}}_p,n_{k,c}=\sum _{(\tilde{\mathbf{x}},\mathbf{y})\in {\mathcal{D}}_k}|\Omega (\mathbf{y},c)|,$$

(17)

where ${\mathbf{v}}_p$ is the embedding contributing to class c (a single $\mathbf{z}$ for classification or a pixel embedding ${\mathbf{F}}_p$ for segmentation), and $\Omega (\mathbf{y},c)$ selects contributing positions. With secure aggregation, the server receives only ${\sum }_k{\mathbf{s}}_{k,c}$ and ${\sum }_k{n}_{k,c}$ and updates prototypes by an exponential moving average:

$${\overline{\mathbf{p}}}_c^{\left(r\right)}={\displaystyle \frac{{\sum }_{k\in {\mathcal{S}}_r}{\mathbf{s}}_{k,c}}{{\sum }_{k\in {\mathcal{S}}_r}{n}_{k,c}+ϵ}},{\mathbf{p}}_c^{(r+1)}=(1-\mu ){\mathbf{p}}_c^{\left(r\right)}+\mu {\overline{\mathbf{p}}}_c^{\left(r\right)}.$$

(18)

Prototypes provide global semantic anchors that reduce cross-client drift and make subsequent federated aggregation less sensitive to client-specific appearances. We next describe how the canonization and prototype objectives are trained under client-level DP with secure aggregation.

4.4. Privacy-Preserving Federated Optimization and Training Objective

PrivAgriVolt couples supervised learning, shadow-invariance, and prototype alignment while enforcing privacy through secure aggregation and client-level DP on model updates.

4.4.1. Local Training Objective

On client k at round r, we jointly optimize $(\mathit{\theta },\mathit{\psi })$ using

$$\underset{\mathit{\theta },\mathit{\psi }}{\min }{\mathcal{J}}_k={\mathcal{L}}_k(\mathit{\theta },\mathit{\psi })+{\lambda }_{\mathrm{sc}}{\mathcal{L}}_{\mathrm{sc}}+{\lambda }_{\mathrm{pc}}{\mathcal{L}}_{\mathrm{pc}},$$

(19)

where ${\mathcal{L}}_k$ is the supervised risk evaluated on canonized images $\widehat{\mathbf{x}}={\mathcal{N}}_{\mathit{\psi }}(\tilde{\mathbf{x}},\mathbf{m},\mathbf{u})$:

$${\mathcal{L}}_k(\mathit{\theta },\mathit{\psi })={\displaystyle \frac{1}{N_k}}\sum _{(\tilde{\mathbf{x}},\mathbf{y})\in {\mathcal{D}}_k}\ell \left(f_{\mathit{\theta }}\left(\widehat{\mathbf{x}}\right),\mathbf{y}\right).$$

(20)

The shadow-consistency term ${\mathcal{L}}_{\mathrm{sc}}$ operates on features extracted from canonized images, while the prototype-contrastive term ${\mathcal{L}}_{\mathrm{pc}}$ uses server-provided prototypes ${\mathcal{P}}^{\left(r\right)}$ to enforce cross-client semantic alignment.

4.4.2. Client-Level DP with Budget-Aware Noise Schedule

Before sending updates to the server, each client clips and perturbs its model updates (covering both $\mathit{\theta }$ and $\mathit{\psi }$) as in Equation (10). We use a round-dependent noise multiplier to allocate more utility to later rounds:

$${\sigma }_r={\sigma }_{\max }-({\sigma }_{\max }-{\sigma }_{\min }){\left({\displaystyle \frac{r}{R}}\right)}^{\beta },r=1,\dots ,R,$$

(21)

where R is the total number of rounds and $\beta \ge 1$ controls the decay. The schedule is calibrated so that a standard privacy accountant yields the target overall $(\epsilon ,\delta )$ for the full training run (reported in Section 5). To avoid disclosing norm statistics, clients can set a stable clipping norm C using an exponential moving average of recent local update norms computed entirely on-device.

4.4.3. Secure Aggregation and Global Updates

Clients transmit encrypted DP-perturbed updates $({\widetilde{\Delta \mathit{\theta }}}_k^{\left(r\right)},{\widetilde{\Delta \mathit{\psi }}}_k^{\left(r\right)})$, and the server recovers only the aggregated update and applies FedAvg (Equation (8)). In parallel, prototype sufficient statistics from Equation (17) are securely aggregated to update ${\mathcal{P}}^{\left(r\right)}$ using Equation (18). This yields a single training loop in which (i) canonization reduces illumination shift, (ii) prototypes align stress semantics under heterogeneity, and (iii) secure aggregation plus DP provide end-to-end privacy. The details of the pseudo code are shown in Algorithm 1.

Algorithm 1 PrivAgriVolt training (server and clients)

1: Server initializes model parameters ${\mathit{\theta }}^{\left(0\right)},{\mathit{\psi }}^{\left(0\right)}$, prototypes ${\mathcal{P}}^{\left(0\right)}$, total rounds R, and noise schedule ${\left\{{\sigma }_r\right\}}_{r=1}^R$.   2: for $r=0$ to $R-1$ do   3:  Server samples clients ${\mathcal{S}}_r$ and broadcasts $({\mathit{\theta }}^{\left(r\right)},{\mathit{\psi }}^{\left(r\right)},{\mathcal{P}}^{\left(r\right)},{\sigma }_{r+1})$.   4:  for all clients $k\in {\mathcal{S}}_r$ in parallel do   5:   Client computes $\mathbf{u}=\mathcal{E}({\mathbf{g}}^{\left(k\right)},\mathbf{s}\left(t\right))$ and estimates shadow masks $\mathbf{m}$ locally.   6:   Client canonizes inputs $\widehat{\mathbf{x}}={\mathcal{N}}_{\mathit{\psi }}(\tilde{\mathbf{x}},\mathbf{m},\mathbf{u})$ and updates $(\mathit{\theta },\mathit{\psi })$ for E steps by minimizing Equation (19).   7:   Client forms updates $(\Delta {\mathit{\theta }}_k^{\left(r\right)},\Delta {\mathit{\psi }}_k^{\left(r\right)})$ and prototype stats ${\left\{({\mathbf{s}}_{k,c},n_{k,c})\right\}}_{c=1}^C$.   8:   Client clips and perturbs updates using Equation (10) with noise ${\sigma }_{r+1}$ to obtain DP updates $({\widetilde{\Delta \mathit{\theta }}}_k^{\left(r\right)},{\widetilde{\Delta \mathit{\psi }}}_k^{\left(r\right)})$; encrypts for secure aggregation.   9:  end for 10:  Server securely aggregates ${\sum }_{k\in {\mathcal{S}}_r}{w}_k{\widetilde{\Delta \mathit{\theta }}}_k^{\left(r\right)}$ and ${\sum }_{k\in {\mathcal{S}}_r}{w}_k{\widetilde{\Delta \mathit{\psi }}}_k^{\left(r\right)}$, then updates $({\mathit{\theta }}^{(r+1)},{\mathit{\psi }}^{(r+1)})$ by FedAvg (Equation (8)). 11:  Server securely aggregates prototype stats and updates ${\mathcal{P}}^{(r+1)}$ by Equation (18). 12: end for

5. Experiments

5.1. Datasets and Evaluation Protocol

We evaluated PrivAgriVolt on real agricultural vision benchmarks and on PV-shadowed variants that emulate agrivoltaic (AV) illumination. Our goal was to stress-test cross-client generalization when periodic shading perturbed color constancy, local contrast, and fine texture cues, while keeping the semantic label space unchanged. Unless otherwise stated, all methods used identical training/test partitions, identical shadowed inputs (when applicable), and identical client splits to ensure fair comparison.

Benchmarks

We covered both ground-level plant disease recognition and aerial field pattern segmentation. For classification, we used (i) PlantVillage (38 classes; controlled leaf images) and (ii) PlantDoc (27 classes in the standard classification split: 10 healthy and 17 diseased labels; in-the-wild backgrounds and capture conditions). For segmentation, we used Agriculture-Vision (six non-background pattern labels over aerial agricultural tiles). URLs to obtain these datasets are as follows:

• PlantVillage (https://www.kaggle.com/datasets/emmarex/plantdisease) (accessed on 20 August 2025),
• PlantDoc (https://github.com/pratikkayal/PlantDoc-Dataset) (accessed on 10 August 2025),
• Agriculture-Vision (https://www.agriculture-vision.com/) (accessed on 23 August 2025).

To emulate PV-row shadows without requiring per-image 3D scene recovery, we overlaid physically plausible strip shadows produced by a simplified PV row geometry and sun direction. Concretely, for an RGB image $\mathbf{x}\in {[0,1]}^{H\times W\times 3}$, we synthesized a soft shadow mask $\mathbf{m}\in {[0,1]}^{H\times W}$ and produced the shadowed observation $\tilde{\mathbf{x}}=\mathbf{x}\odot \left(\alpha \mathbf{m}+(1-\alpha )\mathbf{1}\right),$ where $\alpha \in (0,1]$ controls attenuation strength and $\mathbf{m}$ captures spatially varying shading (consistent with the notation in Section 3). The mask was generated from PV-row pitch $\rho$, height h, row direction (derived from normal $\mathbf{n}$), and sun direction $\mathbf{s}\left(t\right)$, producing quasi-periodic bands. To avoid unrealistically hard edges, we applied a penumbra model by blurring and slightly warping the binary band mask, yielding smooth transitions that matched typical PV-shadow softness on vegetation canopies.

We created PlantVillage-AV and PlantDoc-AV by applying the renderer to all images, sampling a per-image severity (target mean shaded area $\approx 35\%$). Labels remained unchanged. For aerial segmentation, we constructed AV-Shadow from Agriculture-Vision by sampling a broader geometry/time prior (to mimic changing PV layouts across farms) and adding mild specular streaks (low-amplitude, high-frequency highlights) to reflect occasional glints on leaves or soil under strong directional lighting. That renderer captured periodicity, attenuation strength, and penumbra width, but it did not model panel soiling, atmospheric scattering, canopy self-shadowing, or wind-driven motion. The synthetic results should therefore be read as controlled robustness measurements rather than a full substitute for field validation.

For each dataset, we simulated $K=10$ clients to reflect realistic heterogeneity: PlantDoc clients were formed by source subsets (e.g., capture device and background style), PlantVillage clients by crop type and capture conditions (controlled but class-imbalanced), and Agriculture-Vision clients by geographic tiles (region-dependent appearance and label prevalence). We report leave-one-client-out generalization: trained on nine clients and evaluated on the held-out client, repeating for all 10 held-out clients and averaging. This protocol directly measured robustness to domain shifts induced by farm/site differences and shadow distributions. We used $K=10$ because the public benchmarks admitted only a limited number of coherent source/region partitions while still preserving label coverage in each fold; this should therefore be interpreted as a controlled cross-silo benchmark rather than a large-population federation study.The pipeline for generating PV-shadowed inputs is illustrated in Figure 2 and the shadow severity distributions across datasets are shown in Figure 3.

Figure 2. Construction of PV-shadowed inputs. The renderer samples PV-row geometry and sun direction to generate a soft strip-shadow mask $\mathbf{m}$; attenuation strength $\alpha$ controls overall darkness. The same synthesis pipeline was used to create PlantVillage-AV, PlantDoc-AV, and AV-Shadow, with broader geometry priors for AV-Shadow.

Figure 3. Illustrative severity distributions used for PV-shadow synthesis. Filled circles denote PlantVillage-AV, filled squares denote PlantDoc-AV, and filled triangles denote AV-Shadow; AV-Shadow intentionally places more mass on higher shaded-area fractions to reflect stronger periodic shading in aerial agrivoltaic deployments.

Classification was evaluated by top-one accuracy (%). Segmentation was evaluated by mean IoU (%) over non-background classes (background excluded to prevent trivial gains from dominant empty regions). All images were resized to a fixed shorter side (classification) or to the dataset-native tile resolution (segmentation) with standard normalization; augmentations (random crop/flip, mild color jitter) were applied identically across methods and were also applied before PV shadow overlay to avoid augmentations inadvertently “undoing” shadow structure.The datasets and their PV-shadowed variants used in this study are summarized in Table 1 and the detailed parameter settings of the PV shadow renderer are listed in Table 2.

Table 1. Datasets and derived PV-shadowed variants used in our evaluation. Counts are reported after standard filtering (corrupted files removed) and before federated client partitioning.

Dataset	Task	Variant(s) Used	#Classes	Typical Resolution
PlantVillage	Classification	PlantVillage, PlantVillage-AV	38	256–512 px images
PlantDoc	Classification	PlantDoc, PlantDoc-AV	27	In-the-wild (var.)
Agriculture-Vision	Segmentation	Agriculture-Vision, AV-Shadow	6	Aerial tiles (fixed)

Table 2. PV shadow renderer parameterization. Ranges were sampled per image (classification) or per tile (segmentation), and the table lists the concrete intervals used in the synthetic renderer.

Parameter	Symbol	Sampling/Range
PV-row pitch	$\rho$	PlantVillage-AV/PlantDoc-AV: $\mathcal{U}[4.0,6.5]$ m; AV-Shadow: $\mathcal{U}[3.5,8.0]$ m
PV-row height	h	PlantVillage-AV/PlantDoc-AV: $\mathcal{U}[2.2,3.4]$ m; AV-Shadow: $\mathcal{U}[2.2,4.0]$ m
Row orientation	$\mathbf{n}$	Uniform azimuth in $[0°,180°)$ with $\pm 7.5°$ jitter
Sun direction	$\mathbf{s}\left(t\right)$	Solar elevation in $[25°,65°]$; relative azimuth in $[-60°,60°]$
Attenuation strength	$\alpha$	Uniform in $[0.55,0.90]$
Penumbra softness	–	Gaussian blur radius in $[3,7]$ px (classification) and $[5,15]$ px (AV-Shadow)
Shaded-area target	–	Mean $\approx 35\%$ for PlantVillage-AV/PlantDoc-AV and ≈45% for AV-Shadow

5.2. Implementation and Privacy Settings

This subsection details the exact training stack used in our federated experiments, including model architectures, optimization hyperparameters, and privacy mechanisms. We emphasize settings that materially affect stability under (i) strong illumination shifts induced by PV-row shadows and (ii) client-level DP noise applied to updates. Unless stated otherwise, all baselines are tuned under the same compute budget (same number of rounds, local steps, and batch size) and use the same data preprocessing/augmentations.

For classification (PlantVillage/PlantDoc and their AV variants), we used a ResNet-50 encoder with a linear classification head. The global pooled backbone output was therefore $d=2048$. For segmentation (Agriculture-Vision and AV-Shadow), we used a DeepLabV3+style architecture with a ResNet-50 backbone, atrous spatial pyramid pooling, and a light decoder head. The choice of ResNet-50 was deliberate: it is strong enough to expose meaningful robustness differences while remaining feasible for multi-client FL.

PrivAgriVolt introduces a shadow normalization module ${\mathcal{N}}_{\mathit{\psi }}$ placed before the task model, mapping shadowed inputs $\tilde{\mathbf{x}}$ to normalized images $\widehat{\mathbf{x}}={\mathcal{N}}_{\mathit{\psi }}(\tilde{\mathbf{x}},\mathbf{m},\mathbf{u})$. The exact architecture used for overhead accounting was a four-scale U-Net with channel widths $[16,32,64,128]$, two $3\times 3$ convolutions per stage, bilinear upsampling, RGB+mask input (four channels), and a two-layer FiLM MLP ($8\to 64\to 480$) that predicts the affine parameters used in the encoder blocks. This separation is important: the task model is not forced to learn illumination invariance from scratch, and the normalization module can be regularized by shadow-consistency (Equation (15)) even when labels are scarce or heterogeneous.

We used synchronous federated training with $K=10$ clients. Each global round sampled all available clients (full participation) unless a baseline explicitly required otherwise; this kept the privacy accounting consistent and prevented performance differences from being confounded by client subsampling. Each client ran $E=5$ local epochs (or equivalently E local steps per mini-epoch, depending on dataset size) with batch size 32 and SGD momentum 0.9. We applied cosine learning rate decay over $R=200$ rounds. The default loss weights were ${\lambda }_{\mathrm{sc}}=0.5$ for shadow-consistency and ${\lambda }_{\mathrm{pc}}=0.2$ for prototype contrastive learning, with temperature $\tau =0.2$ (Equation (16)). Prototype momentum was $\mu =0.3$ in Equation (18), which empirically stabilized cross-client alignment when label distributions differed.

For segmentation, we additionally used class-balanced sampling at the tile level when feasible to reduce the dominance of easy/background-heavy regions. All methods—baselines and ours—shared the same augmentation stack (random crop/flip, mild color jitter) to isolate algorithmic contributions from augmentation choices.

When DP was enabled, we applied client-level DP to each client update before aggregation (Equation (10)). Each client’s model update was clipped to ${\ell }_2$ norm $C=1.0$ and perturbed with Gaussian noise. We used $C=1.0$ as a conservative normalized threshold that suppressed a small number of very large client updates without forcing aggressive clipping on every round; because the best value can vary with the dataset and optimizer scale, the clipping sensitivity analysis in Section 5.6 shows that this value provided the best trade-off among the tested settings. We report privacy as $(\epsilon ,\delta )$ with $\delta ={10}^{-5}$, accounting over R rounds. Secure aggregation was assumed whenever DP was enabled so that the server only observed the noisy aggregate, not individual updates.

A key implementation detail was the adaptive noise schedule (Equation (21)), parameterized by $({\sigma }_{\max },{\sigma }_{\min },\beta )=(1.2,0.6,2)$. Intuitively, adding higher noise early prevents memorization and suppresses unstable cross-client gradients, while lower noise later preserves fine-grained improvements once representations have partially aligned. This is especially beneficial under PV shadows because early rounds often exhibit exaggerated feature drift between shaded and unshaded clients, which DP noise can otherwise amplify.The unified training and model hyperparameters across tasks are presented in Table 3.

Table 3. Training and model hyperparameters used across tasks. The same federated schedule was used for all baselines unless a method required a specific deviation (in which case we matched total update steps).

Setting	Classification	Segmentation
Backbone	ResNet-50	ResNet-50 (DeepLabV3+ head)
Optimizer	SGD + momentum 0.9	SGD + momentum 0.9
Batch size (local)	32	32
Local steps/epochs	$E=5$	$E=5$
Global rounds	$R=200$	$R=200$
LR schedule	Cosine decay	Cosine decay
Shadow norm ${\mathcal{N}}_{\mathit{\psi }}$	U-Net (4 scales) + FiLM	U-Net (4 scales) + FiLM
Loss weights	${\lambda }_{\mathrm{sc}}=0.5,{\lambda }_{\mathrm{pc}}=0.2$	${\lambda }_{\mathrm{sc}}=0.5,{\lambda }_{\mathrm{pc}}=0.2$
Prototype temperature	$\tau =0.2$	$\tau =0.2$
Prototype momentum	$\mu =0.3$	$\mu =0.3$

For each leave-one-client-out fold, we fixed the held-out client and trained on the remaining clients under identical schedules. Reported numbers are the average over all 10 held-out clients. For DP runs, we kept the same clipping and schedule across datasets to avoid per-dataset privacy tuning. We also fixed the PV-shadowed variant generation seed per dataset split so that all compared methods saw identical shadow patterns.The key privacy configurations for client-level differential privacy are summarized in Table 4.

Table 4. Privacy settings for client-level DP. “Ada” denotes the adaptive noise schedule in Equation (21). We assume secure aggregation for DP runs so the server observes only aggregated noisy updates.

DP Component	Setting
DP granularity	Client-level (per-round client update)
Failure probability	$\delta ={10}^{-5}$
Clipping norm	$C=1.0$
Noise type	Gaussian
Noise schedule	Ada: $({\sigma }_{\max },{\sigma }_{\min },\beta )=(1.2,0.6,2)$
Total rounds accounted	$R=200$
Aggregation security	Secure aggregation assumed (DP enabled)
Reported budget	$(\epsilon ,\delta )$ after R rounds (e.g., $\epsilon \approx 4$)

5.3. Practical Overhead, Communication Cost, and Deployment Considerations

To quantify deployment feasibility, Table 5 reports the incremental size of the shadow normalization module, together with parameter counts, approximate multiply–accumulate operations (MACs), and a reference single-thread CPU latency for a straightforward PyTorch 1.13 implementation. Exact embedded latency depends on the target SoC, compiler, and quantization stack, so these values should be interpreted as hardware-agnostic complexity indicators rather than universal edge-device timings.

Table 5. Practical overhead analysis. Communication assumes FP32 transmission of trainable weights only and excludes the small extra metadata used by secure aggregation. Prototype exchange is negligible because the server only aggregates class sums and counts.

Component	Parameters	Approx. MACs/Image	Ref. Latency	One-Way Payload
ResNet-50 backbone ($224\times 224$)	25.56M	4.09 GMAC	105.8 ms	102.2 MB
Shadow normalization U-Net ($224\times 224$)	0.49M	2.05 GMAC	64.9 ms	1.97 MB
ResNet-50 backbone ($512\times 512$)	25.56M	21.34 GMAC	610.9 ms	102.2 MB
Shadow normalization U-Net ($512\times 512$)	0.49M	10.72 GMAC	562.4 ms	1.97 MB
Prototype table ($C\times d$, $d=2048$)	–	–	–	0.05–0.31 MB

The added shadow normalization module increases the ResNet-50 parameter count by only 0.49M parameters (about 1.9%) and adds 1.97 MB to the one-way FP32 client payload. For classification, the backbone+normalization pair therefore requires about 104.2 MB uplink and 104.2 MB downlink per round, or roughly 208.4 MB total traffic per client per round; across 200 rounds this corresponds to about 41.7 GB/client. The combined reference latency is about 170.7 ms per $224\times 224$ image and about 1.17 s per $512\times 512$ image on a single CPU thread. Prototype statistics remain negligible by comparison. In bandwidth-limited AIoT deployments, 8-bit quantization would reduce this model traffic by approximately $4\times$ (to about 10.4 GB/client over training), and sparsified/top-k uplinks could reduce it further; integrating such communication compression is a practical complement to the present privacy layer and does not alter the core shadow-aware learning objective.

5.4. Main Results

We compared PrivAgriVolt to representative federated baselines under the leave-one-client-out protocol described in Section 5.1. Our evaluation targeted two complementary stressors that commonly co-occur in agrivoltaic deployments: (i) illumination shift induced by periodic PV-row shadows, and (ii) statistical heterogeneity across farms, crops, and geographic tiles. We therefore report results on PV-shadowed classification variants (PlantVillage-AV, PlantDoc-AV) and aerial segmentation (Agriculture-Vision, AV-Shadow), where AV-Shadow represents the most challenging setting with broader PV-geometry priors and stronger shading.

Table 6 shows that PrivAgriVolt improves performance across all benchmarks compared to FedAvg, FedProx, FedBN, and MOON under identical partitions and shadowed inputs. The gains are consistent across tasks (classification and segmentation) and persist when privacy is enabled. In particular, AV-Shadow highlights the benefit of explicitly addressing illumination distortion: periodic shading perturbs local contrast and color constancy in a structured way that is not well handled by generic federated objectives alone. By contrast, PrivAgriVolt combines (i) PV-conditioned shadow normalization to reduce domain gap at the pixel/feature level and (ii) prototype alignment to stabilize semantics across clients, leading to the strongest cross-client generalization among federated methods.

Table 6. Leave-one-client-out performance on PV-shadowed agrivoltaic variants and aerial segmentation. “DP” uses client-level DP with $\epsilon \approx 4.0$ and $\delta ={10}^{-5}$. Higher is better.

Method	PlantVillage-AV Acc.	PlantDoc-AV Acc.	Agriculture-Vision mIoU	AV-Shadow mIoU
Centralized (no privacy)	94.8	79.1	55.4	47.2
FedAvg [7]	90.2	71.8	49.6	39.8
FedAvg + DP [9]	88.1	69.5	47.3	37.1
FedProx [44]	90.9	72.6	50.1	40.5
FedBN [45]	91.6	73.2	50.8	41.3
MOON [46]	91.3	73.5	50.6	41.0
PrivAgriVolt (no DP)	93.7	77.4	54.2	46.1
PrivAgriVolt (full)	92.9	76.1	53.0	44.7

The largest improvements appear on the most shadow-sensitive settings. Compared to FedBN, PrivAgriVolt improves AV-Shadow by $+4.8$ mIoU without DP (46.1 vs. 41.3) and retains a substantial advantage with DP enabled (44.7 vs. 39.0 in the same visualization as Figure 4). For classification, the improvements are also meaningful: PlantDoc-AV benefits strongly because background clutter and capture variability already induce a domain gap, and PV shadows compound that gap; PrivAgriVolt reduces this compounding effect through normalization plus cross-client prototype alignment.

Figure 4. AV-Shadow segmentation comparison with and without DP. For each method, the solid bar corresponds to no DP, and the hatched bar corresponds to client-level DP with $\epsilon \approx 4$ and $\delta ={10}^{-5}$. PrivAgriVolt maintains the highest mIoU in both regimes, indicating that PV-conditioned normalization and prototype alignment improve robustness to structured shading and reduce the effective amplification of DP noise under heterogeneity.

Client-level DP reduces accuracy/mIoU for all methods due to clipping and additive noise. However, PrivAgriVolt exhibits stronger utility retention than the DP baseline (FedAvg + DP): the proposed normalization and prototype alignment dampen the sensitivity of the learning dynamics to DP perturbations, particularly under heterogeneous shading patterns. Table 7 summarizes these effects using (i) gains over FedBN (a strong heterogeneity baseline) and (ii) gains over FedAvg + DP (a privacy baseline), computed directly from Table 6.

Table 7. Derived summary from Table 6. “Gain over FedBN” uses PrivAgriVolt (no DP) minus FedBN; “Gain over FedAvg + DP” uses PrivAgriVolt (full) minus FedAvg + DP. “DP drop” is PrivAgriVolt (no DP) minus PrivAgriVolt (full), measuring utility loss when enabling DP. All values are in percentage points.

Dataset	Gain Over FedBN	Gain Over FedAvg + DP	DP Drop (Ours)
PlantVillage-AV (Acc.)	$+2.1$	$+4.8$	$0.8$
PlantDoc-AV (Acc.)	$+4.2$	$+6.6$	$1.3$
Agriculture-Vision (mIoU)	$+3.4$	$+5.7$	$1.2$
AV-Shadow (mIoU)	$+4.8$	$+7.6$	$1.4$

While centralized training remains an upper bound (all data pooled, no privacy), PrivAgriVolt substantially narrows the gap without violating federated constraints. On AV-Shadow, PrivAgriVolt (no DP) reaches 46.1 mIoU versus 47.2 for centralized training, closing most of the difference despite client/domain separation. With DP enabled, performance remains close to the non-private federated setting, indicating the method is compatible with practical privacy requirements.The overall performance improvements over baselines are displayed in Figure 5.

Figure 5. Aggregate improvements across benchmarks. The solid bars report PrivAgriVolt (no DP) gains over FedBN (a strong heterogeneity baseline) on PlantVillage-AV (PV-AV), PlantDoc-AV (PD-AV), Agriculture-Vision (Agri-V), and AV-Shadow. The dotted bars report PrivAgriVolt (full, DP enabled) gains over FedAvg + DP at $\epsilon \approx 4$, highlighting that our privacy-compatible configuration delivers the largest absolute advantage in the most shadow-intensive regime (AV-Shadow).

5.5. Ablation Study

This subsection disentangles the contribution of each component in PrivAgriVolt under the most challenging setting, AV-Shadow, where periodic PV-row shading (and mild specular artifacts) induces strong appearance shift. We ablated (i) PV-conditioned shadow normalization (SN), (ii) shadow-consistency regularization (SC), (iii) prototype contrastive learning (PC), and (iv) the adaptive DP noise schedule (Ada-DP) when client-level DP was enabled. Unless explicitly stated, all variants followed the same federated protocol (leave-one-client-out over $K=10$ clients), the same training budget ($R=200$ rounds, $E=5$ local steps, batch size 32), and the same optimizer settings as Section 5.2. For privacy experiments, we fixed $\delta ={10}^{-5}$ and computed the final $(\epsilon ,\delta )$ after accounting over rounds; when comparing DP variants, we held the final $\epsilon$ approximately constant to ensure fairness.

We adopted an incremental ablation starting from a standard federated baseline (FedAvg) and adding modules one at a time. This ordering reflects the dependency structure of the method: SN is the primary mechanism that removes PV-shadow bias in the input domain; SC regularizes SN to be stable across shadow severities; PC aligns semantics across clients after normalization; Ada-DP is only meaningful once DP is enabled and was compared against a fixed-noise DP baseline with the same final privacy budget. Importantly, when SN was removed, SC was also removed because SC was defined on normalized outputs. Similarly, when PC was removed, we still trained the task model with the same supervised loss and (if present) SC.

Table 8 shows that each component improves AV-Shadow mIoU, with the largest single jump coming from adding SN (+3.4 mIoU over FedAvg). SC provides additional improvement (+0.9 mIoU), suggesting that enforcing illumination-consistent predictions across shadow severities prevents overfitting to a particular band pattern. PC provides the final large boost (+2.0 mIoU), consistent with the idea that even after normalization, client heterogeneity remains and benefits from representation alignment. With DP enabled, performance drops compared to the non-private variant, but Table 9 shows that Ada-DP recovers a meaningful portion of the gap versus fixed-noise DP at the same approximate $\epsilon$, indicating that time-varying noise is helpful under non-IID shading conditions.

Table 8. Component ablation on AV-Shadow (mIoU %). “SN” denotes PV-conditioned shadow normalization; “SC” denotes shadow-consistency; “PC” denotes prototype contrastive learning. All entries are leave-one-client-out averages over 10 held-out clients.

Variant	SN	SC	PC	AV-Shadow mIoU
FedAvg baseline [7]				39.8
+ SN	✓			43.2
+ SN + SC	✓	✓		44.1
+ SN + SC + PC (PrivAgriVolt, no DP)	✓	✓	✓	46.1

Table 9. DP-related ablation on AV-Shadow (mIoU %). “Fixed DP” uses a constant noise multiplier across rounds; “Ada-DP” uses Equation (21). Both were configured to achieve approximately the same final privacy budget (e.g., $\epsilon \approx 4$ with $\delta ={10}^{-5}$) under the same clipping norm.

Variant (DP Enabled)	SN + SC + PC	Noise Schedule	AV-Shadow mIoU
FedAvg + DP [9]		Fixed DP	37.1
PrivAgriVolt + DP (fixed noise)	✓	Fixed DP	43.5
PrivAgriVolt (full)	✓	Ada-DP	44.7

Figure 6 visualizes the incremental improvements on AV-Shadow as modules are added. The shape is informative: SN yields a large early gain (removing the dominant illumination bias), while PC yields a large later gain (addressing remaining cross-client semantic drift). Figure 7 compares fixed-noise DP and Ada-DP under matched $\epsilon$, showing that Ada-DP consistently improves mIoU, especially in later rounds where reduced noise helps consolidate fine-grained boundary predictions and minority class patterns. Figure 7 complements the main tables by visualizing the privacy–utility trade-off across multiple privacy budgets $\epsilon \in \{1,2,4,8\}$ at fixed $\delta ={10}^{-5}$.

Figure 6. Incremental ablation on AV-Shadow. Filled circles show mIoU as components are added to a FedAvg baseline: adding SN yields the largest single gain, SC further improves stability under varying shadow severity, and PC provides the final substantial improvement by aligning cross-client semantics after normalization.

Figure 7. Privacy–utility trade-off on AV-Shadow. Filled squares denote FedAvg + DP, and filled circles denote PrivAgriVolt (full, DP enabled with Ada-DP). Across privacy budgets $\epsilon \in \{1,2,4,8\}$ (with fixed $\delta ={10}^{-5}$), PrivAgriVolt maintains higher mIoU, indicating that PV-conditioned normalization and prototype alignment reduce heterogeneity-induced sensitivity to DP noise.

5.6. Robustness and Sensitivity Analyses

We complemented the main ablation with two practical sensitivity tests directly relevant to deployment: robustness to imperfect shadow masks and stability under different DP clipping norms. Unless otherwise stated, these experiments used the same leave-one-client-out protocol and training schedule as the main results. We compared three conditioning masks for the full PrivAgriVolt model without DP: (i) the perfect renderer mask, (ii) the hybrid geometry+detector mask described in Section 4.2, and (iii) a perturbed mask obtained by boundary jitter and random hole dropout. Table 10 shows that the method degrades gracefully as mask quality decreases. Relative to the perfect-mask upper bound, the hybrid estimator loses only 0.6 percentage points on PlantDoc-AV and 0.7 mIoU on AV-Shadow, while the stronger perturbed mask causes larger but still moderate drops. This behavior is consistent with the soft correction mechanism in Equation (12) and the region-averaged shadow-consistency loss in Equation (15).

Table 10. Sensitivity to shadow-mask quality for the full PrivAgriVolt model (no DP). The hybrid mask uses local geometry-guided estimation followed by detector refinement.

Mask Condition	PlantDoc-AV Acc.	AV-Shadow mIoU
Perfect renderer mask	77.4	46.1
Hybrid geometry+detector mask	76.8	45.4
Perturbed mask	75.9	44.8

We further analyzed the DP-enabled configuration on AV-Shadow at the representative privacy budget $\epsilon \approx 4$ with $\delta ={10}^{-5}$. Table 11 reports mean ± std over three random seeds for clipping norms $C\in \{0.5,1.0,2.0\}$. The standard deviation remains below 0.35 mIoU in all cases, indicating stable optimization. The best average is obtained at $C=1.0$, which balances excessive clipping at smaller C against higher effective noisy update energy at larger C.

Table 11. Sensitivity of DP performance to the clipping norm C on AV-Shadow. Results are mean ± std mIoU over three random seeds at $\epsilon \approx 4$ and $\delta ={10}^{-5}$.

Clipping Norm C	AV-Shadow mIoU	Seeds
0.5	44.0 ± 0.31	3
1.0	44.7 ± 0.24	3
2.0	44.2 ± 0.29	3

5.7. Pilot Transfer to Real Agrivoltaic Scenes

To examine transfer beyond the synthetic renderer, we summarize a compact pilot evaluation on real agrivoltaic imagery using the hybrid mask estimator from Section 4.2. We considered two task-aligned subsets: a ground-view disease-recognition subset under row-cast PV shadows and a UAV subset for field-pattern segmentation. Table 12 shows that the ranking observed on the synthetic-shadow benchmarks is preserved on the pilot real scenes. Compared with the strong heterogeneity baseline FedBN, PrivAgriVolt improves ground-view accuracy by 2.5 percentage points and UAV mIoU by 2.5 points. The absolute values are slightly lower than on the synthetic-shadow benchmarks, which is consistent with additional appearance shifts from canopy self-shadowing, foliage motion, and camera response differences that are not explicitly modeled by the renderer.

Table 12. Pilot transfer results on real agrivoltaic scenes. Both methods used the same backbone and federated schedule; PrivAgriVolt used the hybrid local mask estimator.

Method	Ground-View AV Pilot Acc.	UAV AV Pilot mIoU
FedBN	74.3	41.7
PrivAgriVolt	76.8	44.2

Overall, the additional analyses indicate that PrivAgriVolt is not overly sensitive to moderate mask noise, remains stable under reasonable clipping choices, and continues to provide a measurable advantage when transferred to a small real agrivoltaic pilot. At the same time, the broader benchmark should still be interpreted as a controlled cross-silo study with $K=10$, and larger multi-season field campaigns remain an important next step.

6. Conclusions

This paper presented PrivAgriVolt, a privacy-preserving and shadow-aware learning framework for crop stress recognition and segmentation in agrivoltaic photovoltaic systems, where structured PV-induced shadows and cross-farm heterogeneity jointly degrade conventional vision models and complicate centralized data collection. By conditioning a stress-preserving shadow canonization network on PV geometry and solar priors, aligning representations through securely aggregated class prototypes, and enforcing client-level differential privacy with secure aggregation and an adaptive noise schedule, PrivAgriVolt improves cross-client generalization under severe shading while maintaining strong privacy–utility trade-offs. Experiments on real agricultural benchmarks and PV-shadowed variants demonstrated consistent gains over federated baselines with and without DP, additional robustness analyses showed limited sensitivity to moderate mask noise and stable DP behavior around $C=1.0$, and the pilot real-scene transfer study preserved the same method ranking as the synthetic-shadow benchmarks. The current benchmark is nevertheless best interpreted as a controlled cross-silo study, with larger multi-season field validation, larger-federation scaling, and communication compression as important next deployment steps. Future work will extend the approach to multimodal sensing (e.g., NIR/thermal), integrate more accurate site-specific geometry estimation, and explore continual adaptation to seasonal shifts and evolving stress taxonomies under fixed privacy budgets.