Fuzzy-Partitioned Multi-Agent TD3 for Photovoltaic Maximum Power Point Tracking Under Partial Shading

Abstract

Maximum power point tracking (MPPT) under partial shading is a nonconvex, rapidly varying control problem that challenges multi-agent policies deployed on photovoltaic modules. We present Fuzzy–MAT3D, a fuzzy-augmented multi-agent TD3 (Twin-Delayed Deep Deterministic Policy Gradient) controller trained under centralized training/decentralized execution (CTDE). On the theory side, we prove that differentiable fuzzy partitions of unity endow the actor–critic maps with global Lipschitz regularity, reduce temporal-difference target variance, enlarge the input-to-state stability (ISS) margin, and yield a global $L_{\infty }$$\gamma$-contraction of fixed-policy evaluation (hence, non-expansive with $\kappa =\gamma <1$). We further state a two-time-scale convergence theorem for CTDE-TD3 with fuzzy features; a PL/last-layer-linear corollary implies point convergence and uniqueness of critics. We bound the projected Bellman residual with the correct contraction factor (for $L^{\infty }$ and $L^2\left(\rho \right)$ under measure invariance) and quantified the negative bias induced by $min\{Q_1,Q_2\}$; an N-agent extension is provided. Empirically, a balanced common-random-numbers design across seven scenarios and 20 seeds, analyzed by ANOVA and CRN-paired tests, shows that Fuzzy–MAT3D attains the highest mean MPPT efficiency (92.0% ± 4.0%), outperforming MAT3D and Multi-Agent Deep Deterministic Policy Gradient controller (MADDPG). Overall, fuzzy regularization yields higher efficiency, suppresses steady-state oscillations, and stabilizes learning dynamics, supporting the use of structured, physics-compatible features in multi-agent MPPT controllers. At the level of PV plants, such gains under partial shading translate into higher effective capacity factors and smoother renewable generation without additional hardware.

1. Introduction

Maximum power point tracking (MPPT) in photovoltaic (PV) arrays becomes notably challenging under partial shading conditions (PSCs), where the power–voltage (P–V) landscape is nonconvex and multimodal, with multiple local maxima that shift with irradiance and temperature [1,2,3,4,5]. Classical trackers (perturb-and-observe and incremental conductance) are attractive for their simplicity and low sensor burden, but under PSCs, they can settle at suboptimal peaks and exhibit limit-cycle oscillations; performance is sensitive to stepsize tuning, measurement noise, and plant transients [1,3,4]. Global-search enhancements (adaptive scanning; PSO/GA/DE and hybrids) increase the likelihood of reaching the global MPP, at the cost of additional computation and re-tuning as operating statistics change [2,6,7]. This reflects a broader debate between global exploration to avoid local traps and local tracking to preserve fast, low-perturbation operation.

Several studies further sharpen the MPPT and Deep Reinforcement Learning (DRL) baselines under PSCs. On the DRL side, a recurrent Proximal Policy Optimization–Long Short-Term Memory (PPO–LSTM) agent for global MPPT demonstrates robust gains over classical and nonrecurrent baselines [8], while a hardware-validated DQN achieves GMPPT on real PV strings with rigorous experimental design [9]. Beyond single-agent control, physics-informed multi-agent DRL with centralized training and decentralized execution (CTDE) has matured in power-network settings, offering architectural guidance for PV module coordination [10]. In parallel, the body of comparative evidence and synthesis continues to grow: a controlled 2024 evaluation contrasting classical and learning-based MPPT algorithms [11], a comprehensive 2025 review of traditional and advanced MPPT techniques (including partial-shading scenarios) [12], and a 2025 validation study of a global MPPT strategy under complex partial shading using model-predictive control [13].

Learning-based control offers an alternative that adapts policies to nonstationary PV conditions. Deterministic policy gradient methods enable continuous actuation [14,15]; TD3 reduces overestimation bias via twin critics and target smoothing [16,17]. In multi-string or module-integrated topologies, the problem is intrinsically multi-agent with local observations and shared objectives, motivating centralized training with decentralized execution (CTDE) as in MADDPG, COMA, and value-factorization approaches [18,19,20]. Nevertheless, off-policy actor–critic stability hinges on value-function smoothness and bootstrapping geometry [16,21], while on-policy trust-region methods (TRPO/PPO) trade stability for higher sampling costs in embedded experimentation [22,23]. Two-time-scale stochastic approximation provides convergence guarantees under regularity, stepsize, and contractivity conditions that are often verified for linear approximation but are harder to certify for deep critics and nonconvex policy classes [24,25,26,27,28].

This work develops Fuzzy–MAT3D, a CTDE variant of TD3 in which actors and critics comprise a differentiable fuzzy partition of unity over the compact PV operating domain [29,30,31]. The partition induces locality and global Lipschitz continuity by construction, aligning with classical partition-of-unity ideas in numerical analysis and meshfree interpolation [32,33,34]. In the MPPT setting, this structure reduces temporal-difference (TD) target variance, bounds critic Jacobians in nonsmooth regions of the P–V surface, and yields a global $L_{\infty }$$\gamma$-contraction of fixed-policy evaluation (hence non-expansive with $\kappa =\gamma <1$)—properties that tighten constants in standard ODE/stochastic-approximation arguments and connect to input-to-state stability margins in control [25,27,28,35,36].

From a mechanistic viewpoint, the differentiable fuzzy PoU constrains the actor and critics to be globally Lipschitz, with moduli that can be explicitly computed from the membership functions. In Section 2, we show that these Lipschitz constants enter directly into the variance bounds of the smoothed TD targets and into the ISS gain $\kappa (L_{\mu },\theta )$ of the closed-loop dynamics. Thus, fuzzy regularization reduces the sensitivity of temporal-difference updates to stochastic perturbations and enlarges the ISS margin, which in turn manifests as smoother learning dynamics and reduced steady-state jitter in the PV power trajectories.

We also analyze the order-statistics bias introduced by the TD3 $min\{Q_1,Q_2\}$ target and provide a projected Bellman residual bound with the correct contraction factor (clarified for $L^{\infty }$ and $L^2\left(\rho \right)$ under measure invariance), together with a clean N-agent CTDE extension [37,38,39,40].

Aim and significance. Our aim is to endow multi-agent TD3 with a structured, differentiable fuzzy front-end that regularizes value estimation and stabilizes policy updates for MPPT under PSCs. The contribution has two principal facets: it injects physics-compatible locality into actor–critic maps; practically, it seeks higher energy capture with reduced steady-state oscillation relative to plain multi-agent TD3, MADDPG, and classical MPPT, as substantiated in the Results Section. For evaluation, we follow established DRL practices and classical multiple-comparison control [41,42,43,44,45].

Problem statement and objectives. We model a two-module PV string under partial shading as a discounted Markov decision process with compact state space $\mathcal{S}$ and bounded action space $\mathcal{A}$, where the agents observe electrical quantities (irradiance proxies, voltages, and currents) and select module voltages or duty cycles every control period in order to maximize normalized PV power. The control problem is to synthesize a stationary multi-agent policy ${\mu }_{\theta }$ that maximizes the long-run MPPT efficiency $\eta$ while keeping steady-state oscillations and settling times within acceptable operating bounds.

Accordingly, the main objectives of this study are as follows: (i) to construct a Lipschitz-regularized CTDE–TD3 architecture with a differentiable fuzzy partition of the operating domain; (ii) to establish convergence and stability guarantees (contraction properties, temporal-difference variance bounds, two-time-scale convergence, and ISS margins) for the resulting actor–critic scheme; and (iii) to demonstrate, on a balanced set of PSC scenarios, that the proposed Fuzzy–MAT3D controller outperforms representative classical and DRL-based MPPT baselines in efficiency and stability.

To the best of our knowledge, this is the first CTDE–TD3 design that explicitly ties a fuzzy partition-of-unity to global Lipschitz constants and ISS margins in PV MPPT.

Relative to entropy-regularized SAC and trust-region methods (TRPO/PPO) [21,22,23], the proposed fuzzy partition of unity (PoU) regularizes deterministic CTDE–TD3 by enforcing global input Lipschitzness and reducing TD-target variance. Recent reports on safe/robust Reinforcement Learning (RL) for power converter control and MPPT under PSCs [46,47,48,49,50,51,52] motivate structure in actor–critic features; hence, Fuzzy–MAT3D offers a physics-compatible alternative, achieving high yield with low ripple under an identical control period and per-tick compute budget; irradiance is measured and logged for MPP reference and analyses, while classical baselines operate on their standard $(V/I)$ inputs.

Contributions. In summary, this paper achieves the following:

• Introduces Fuzzy–MAT3D, a fuzzy-partitioned CTDE–TD3 architecture for multi-string PV arrays, yielding globally Lipschitz and locally interpretable actor–critic mappings [29,31,32].
• Establishes a global $L_{\infty }$$\gamma$-contraction for fixed–policy evaluation (and a projected $L^2\left(\rho \right)$ bound under measure invariance), reduces TD-target variance, bounds the TD3 min underestimation bias, and provides a projected Bellman residual bound with the appropriate contraction factor [16,27,39].
• Provides an N-agent CTDE extension consistent with the above analysis and tailored to PV module coordination [18].
• Demonstrates, under a matched control period and actuation limits with identical per-tick compute budgets (irradiance measured/logged for reference), and randomized, CRN–blocked trials, higher mean MPPT efficiency and lower steady-state oscillations than multi-agent TD3, MADDPG, and classical baselines; details appear in the Methods and Results Sections [41,42].

2. Mathematical Framework and Main Results

This section develops the mathematical backbone of our approach and states the main theoretical contributions. We model MPPT control as a discounted Markov decision process (MDP) with compact state and action spaces and augment the CTDE actor–critic pipeline with a differentiable fuzzy partition of unity over the normalized operating domain; this front-end induces global Lipschitz continuity of the actor and twin critics, uniformly bounds target magnitudes and Jacobians, and reduces TD-target variance by construction.

2.1. Setting, Notation, and Fuzzy PoU

We consider a discounted MDP with compact state space $S\subset {\mathbb{R}}^7$, action space $A=[-1,1]$, and discount $\gamma \in (0,1)$. Two decentralized controllers with shared parameters (CTDE) act on a PV string; rewards are bounded as in (33); hence, $|Q^{\pi }{|\le (1-\gamma )}^{-1}$.

• Fuzzy partition of unity (PoU).

For each coordinate $j\in \{1,\dots ,7\}$, let ${\left\{{\psi }_{\ell }^{\left(j\right)}\right\}}_{\ell =1}^5$ be differentiable, nonnegative, and with a per-coordinate partition: ${\sum }_{\ell }{\psi }_{\ell }^{\left(j\right)}\left(\xi \right)=1$. Define

$$z\left(s\right)=\left({\psi }_1^{\left(1\right)}\left(s_1\right),\dots ,{\psi }_5^{\left(1\right)}\left(s_1\right),\dots ,{\psi }_1^{\left(7\right)}\left(s_7\right),\dots ,{\psi }_5^{\left(7\right)}\left(s_7\right)\right)\in {\mathbb{R}}_{\ge 0}^{35}.$$

(1)

$${\phi }_{\mathrm{fuzzy}}\left(s\right)={\displaystyle \frac{z\left(s\right)}{7}},{\mathbf{1}}^{\top }{\phi }_{\mathrm{fuzzy}}\left(s\right)=1.$$

(2)

• Norm conventions.

Unless stated otherwise, $\parallel ·\parallel$ denotes the Euclidean norm on state and feature spaces, and $\parallel J\parallel$ denotes the induced operator norm. Accordingly, $L_z:={sup}_s{\parallel \nabla z\left(s\right)\parallel }_2$ and $L_{\mu }:=L_z/7$ quantify input Lipschitz moduli in ${\ell }_2$ for the stacked membership map and the normalized PoU, respectively.

Because the memberships satisfy ${\sum }_{\ell =1}^5{\psi }_{\ell }^{\left(j\right)}\left(s_j\right)\equiv 1$ for each of the seven coordinates, we have ${\mathbf{1}}^{\top }z\left(s\right)=7$ identically, so the normalization by 7 is exact.

Since the sum is unitary per coordinate, ${\mathbf{1}}^{\top }z\left(s\right)=7$ and, by compactness, ${\phi }_{\mathrm{fuzzy}}$ is globally Lipschitz. We write $L_{\mu }=\frac{L_z}{7}$ with $L_z:={sup}_s\parallel \nabla z\left(s\right)\parallel$.

Intuitively, a map f is L-Lipschitz if $\parallel f\left(x\right)-f\left(y\right)\parallel \le L\parallel x-y\parallel$ for all $x,y$, so L bounds its worst-case slope. In an actor–critic architecture, global Lipschitz continuity implies that small perturbations in voltages, irradiance estimates, or sensor readings cannot induce arbitrarily large changes in actions or value estimates. The differentiable fuzzy PoU used here ensures that the feature map $z\left(s\right)$ and the actors/critics built on top of it inherit such bounded slopes, with explicit moduli like $L_{\mu }=L_z/7$, thereby regularizing the policy and the value functions at the level of the entire state space.

• Actor–critics with PoU and TD3 smoothing.

The deterministic actor and the two critics are

$$a={\mu }_{\theta }\left(s\right)=tanh\left(f_{\theta }\left({\phi }_{\mathrm{fuzzy}}\left(s\right)\right)\right),Q_{{\phi }_i}(s,a)=h_{{\phi }_i}\left({\phi }_{\mathrm{fuzzy}}\left(s\right),a\right),i\in \{1,2\},$$

with Polyak-averaged targets $(\overline{\theta },{\overline{\phi }}_i)$ and clipped target policy noise (TD3).

Throughout the manuscript, the deterministic policy is denoted by ${\mu }_{\theta }\left(s\right)$, where $\theta$ are the actor parameters. This notation is reserved exclusively for the policy and must not be confused with the fuzzy partition ${\phi }_{\mathrm{fuzzy}}\left(s\right)$ introduced in Section 2. Accordingly, $L_{{\mu }_{\theta }}$ denotes the Lipschitz modulus of the actor, while $L_{\mu }=L_z/7$ refers to the global Lipschitz constant of the normalized partition of unity. This distinction is preserved in all subsequent sections.

2.2. Main Structural Results

Theorem 1

(Fuzzy PoU induces global Lipschitzness). On compact S,

$$\mathrm{Lip}\left({\mu }_{\theta }\right)\le L_{tanh}{L}_f\left(\theta \right)L_{\mu },$$

(3)

$$\mathrm{Lip}\left(Q_{{\phi }_i}\circ ({\phi }_{\mathrm{fuzzy}},\mathrm{id})\right)\le L_Q\left({\phi }_i\right)max\{L_{\mu },1\}.$$

(4)

In particular, gradient norms and TD targets remain uniformly bounded along the iterates.

Proposition 1

(Smoothed fixed-policy operator is $\gamma$-contractive in $L_{\infty }$). Let $\overline{\pi }\left(s\right)={\mu }_{\overline{\theta }}\left(s\right)$ and ε be bounded noise. Define $\left({\mathcal{T}}_{\epsilon }^{\overline{\pi }}Q\right)(s,a)=\mathbb{E}[r+\gamma Q(s^{\prime },\overline{\pi }\left(s^{\prime }\right)+\epsilon )\mid s,a]$. Then

$$\parallel {\mathcal{T}}_{\epsilon }^{\overline{\pi }}Q-{\mathcal{T}}_{\epsilon }^{\overline{\pi }}\tilde{Q}{\parallel }_{\infty }\le \gamma {\parallel Q-\tilde{Q}\parallel }_{\infty }.$$

(5)

and in $L^2\left(\rho \right)$,

$$\parallel {\mathcal{T}}_{\epsilon }^{\overline{\pi }}Q-{\mathcal{T}}_{\epsilon }^{\overline{\pi }}\tilde{Q}{\parallel }_{2,\rho }\le \gamma {\parallel Q-\tilde{Q}\parallel }_{2,{\rho }^{\prime }}.$$

(6)

where ${\rho }^{\prime }$ is the pushforward of ρ by $(s,a)\mapsto (s^{\prime },\overline{\pi }\left(s^{\prime }\right)+\epsilon )$. The same holds for the min-of-twins operator since it is 1-Lipschitz.

• Twin-critic minimum.

We write $m(u,v):=min\{u,v\}$ and use $m\left(Q_{{\overline{\varphi }}_1},Q_{{\overline{\varphi }}_2}\right)$ to denote the TD3 min-of-twins target.

Proposition 2

(TD-target variance reduction under PoU). Let $y=r+\gamma m(Q_{{\overline{\phi }}_1},Q_{{\overline{\phi }}_2})(s^{\prime },\overline{\pi }\left(s^{\prime }\right)+\epsilon )$ with $\epsilon \in [-ϵ,ϵ]$ independent of $s^{\prime }$ conditional on s. Then

$$\mathrm{Var}(y\mid s)\le {\sigma }_r^2+{\gamma }^2{L}_Q^2\left(L_{\mu }^2{\sigma }_{s^{\prime }}^2+{\sigma }_{\epsilon }^2\right).$$

(7)

so decreasing $L_{\mu }$ reduces the contribution of state/action perturbations to the TD-target variance.

Here, ${\sigma }_{s^{\prime }}^2:=\mathbb{E}\left[\parallel s^{\prime }-\mathbb{E}[s^{\prime }\mid s]{\parallel }_2^2\mid s\right]$ is the trace of the conditional covariance of $s^{\prime }$ and $\epsilon \perp s^{\prime }\mid s$ by design (target policy noise independent of next state given s). All Lipschitz moduli are taken with respect to the ${\ell }_2$ norm.

Collecting these results, the fuzzy PoU and the induced Lipschitz bounds play a dual role. On the critic side, they enter the coefficient multiplying the next-state variance term in the TD-target variance bound, thereby damping the stochastic fluctuations of the updates.

On the closed-loop side, the same Lipschitz moduli appear in the ISS gain $\kappa (L_{\mu },\theta )$ of (23), so that reducing $L_{\mu }$ enlarges the input-to-state stability margin. This provides a direct analytical link between the fuzzy regularization mechanism and the empirical reductions in temporal-difference variance and steady-state jitter observed in the PV power trajectories.

Proposition 3

(Projected Bellman residual bound (last–layer linear)). If the critic class $\mathcal{F}\subset L^2\left(\rho \right)$ is closed and convex (e.g., last-layer linear) and ${\Pi }_{\mathcal{F}}$ its orthogonal projection, then

$$\parallel {\Pi }_{\mathcal{F}}{\mathcal{T}}_{\epsilon }^{\overline{\pi }}Q-{\Pi }_{\mathcal{F}}{\mathcal{T}}_{\epsilon }^{\overline{\pi }}\tilde{Q}{\parallel }_{2,\rho }\le \gamma \sqrt{c}{\parallel Q-\tilde{Q}\parallel }_{2,\rho },c=ess\; sup\frac{d{\rho }^{\prime }}{d\rho }<\infty$$

(8)

If $\widehat{Q}={\Pi }_{\mathcal{F}}{\mathcal{T}}_{\epsilon }^{\overline{\pi }}\widehat{Q}$ and $Q^{\overline{\pi }}$ is a fixed point of ${\mathcal{T}}_{\epsilon }^{\overline{\pi }}$, then

$$\parallel \widehat{Q}-Q^{\overline{\pi }}{\parallel }_{2,\rho }\le {\displaystyle \frac{1}{1-\gamma \sqrt{c}}}{\parallel (I-{\Pi }_{\mathcal{F}}){\mathcal{T}}_{\epsilon }^{\overline{\pi }}{Q}^{\overline{\pi }}\parallel }_{2,\rho }.$$

(9)

If, moreover, ${\rho }^{\prime }=\rho$, the constant is $1/(1-\gamma )$.

Remark 1

(On $L^2\left(\rho \right)$ contraction vs. distribution shift). More generally, for any measurable u,

$$\parallel {\mathcal{T}}_{\epsilon }^{\overline{\pi }}{u\parallel }_{2,\rho }\le \gamma \sqrt{c}{\parallel u\parallel }_{2,\rho },c:=ess\; sup{\displaystyle \frac{d{\rho }^{\prime }}{d\rho }}<\infty$$

(10)

where ${\rho }^{\prime }$ is the pushforward of ρ by $(s,a)\mapsto (s^{\prime },\overline{\pi }\left(s^{\prime }\right)+\epsilon )$. The bounds in (10) are Lipschitz bounds across different measures. If, in addition, ${\rho }^{\prime }=\rho$ (e.g., when ρ is the discounted occupancy measure induced by $\overline{\pi }$ with the target noise), one recovers an $L^2\left(\rho \right)$γ-contraction. Otherwise, residual bounds inherit the factor $1/(1-\gamma \sqrt{c})$ instead of $1/(1-\gamma )$.

Remark 2

(Empirical estimation of c and near-invariance). Let ρ denote the (discounted) state-action occupancy under the policy used to define the evaluation operator ${\mathcal{T}}_{\epsilon }^{\overline{\pi }}$ (typically the target $\overline{\pi }$), and let ${\rho }^{\prime }$ be its pushforward by $(s,a)\mapsto (s^{\prime },\overline{\pi }\left(s^{\prime }\right)+\epsilon )$. Over a finite replay window, we estimate the Radon–Nikodym–type constant

$$\widehat{c}(\mathcal{B},\lambda ):=\underset{b\in \mathcal{B}}{max}{\displaystyle \frac{{\widehat{\rho }}^{\prime }\left(b\right)+\lambda }{\widehat{\rho }\left(b\right)+\lambda }},\widehat{\rho }\left(b\right)={\displaystyle \frac{1}{N}}\sum _{k=1}^N\mathbf{1}\{(s_k,a_k)\in b\}.$$

(11)

using a measurable partition $\mathcal{B}$ of the (normalized) state-action domain and Laplace smoothing $\lambda >0$. With Polyak-averaged targets and bounded target policy noise, consecutive replay windows are empirically near-stationary, so $\widehat{c}\approx 1$ is expected; when drift is present, the projected-residual bound inherits the factor $1/(1-\gamma \sqrt{\widehat{c}})$ from Proposition 3 (cf. the $\rho \mapsto {\rho }^{\prime }$ discussion following Proposition 1).

Lemma 1

(Underestimation by TD3’s minimum). Let $Q_i=Q+{\epsilon }_i$ with $\mathbb{E}[{\epsilon }_i\mid s,a]=0$. Then

$$\mathbb{E}[min\{Q_1,Q_2\}\mid s,a]=Q+\frac{1}{2}\mathbb{E}[{\epsilon }_1+{\epsilon }_2-|{\epsilon }_1-{\epsilon }_2|\mid s,a]\le Q.$$

(12)

and

$$0\le Q-\mathbb{E}\left[min\{Q_1,Q_2\}\mid s,a\right]\le \frac{1}{2}\sqrt{Var({\epsilon }_1-{\epsilon }_2\mid s,a)}\le {\displaystyle \frac{{\sigma }_Q}{\sqrt{2}}}\sqrt{1-\rho }.$$

(13)

where $Var({\epsilon }_i\mid s,a)\le {\sigma }_Q^2$ and $\rho :=Corr({\epsilon }_1,{\epsilon }_2\mid s,a)\in [-1,1]$.

• Gaussian specialization.

If $({\epsilon }_1,{\epsilon }_2)$ are jointly normal with $\mathrm{Var}({\epsilon }_i\mid s,a)={\sigma }_Q^2$ and correlation $\rho$, then using $min(x,y)={\displaystyle \frac{x+y-|x-y|}{2}}$ and ${\epsilon }_1-{\epsilon }_2\sim \mathcal{N}\left(0,2{\sigma }_Q^2(1-\rho )\right)$,

$$Q-\mathbb{E}\left[min\{Q_1,Q_2\}\mid s,a\right]={\displaystyle \frac{1}{2}}\mathbb{E}\left[|{\epsilon }_1-{\epsilon }_2|\right]={\sigma }_Q\sqrt{{\displaystyle \frac{1-\rho }{\pi }}}.$$

(14)

Theorem 2

(Two-time-scale convergence (projected SA)). Under i.i.d./mixing replay, smoothness and boundedness of gradients, square-integrable MDS noises, and stepsizes $\sum {\alpha }_k=\sum {\beta }_k=\infty$, $\sum ({\alpha }_k^2+{\beta }_k^2+{\tau }_k^2)<\infty$, ${\beta }_k/{\alpha }_k\to 0$, ${\tau }_k/{\beta }_k\to 0$, the critics converge to the set of stationary points (projected flow) and the actor follows the differential inclusion $\dot{\theta }\in {\Pi }_{\Theta }\left[{\nabla }_{\theta }\tilde{J}\left(\theta \right)\right]$ with $\tilde{J}\left(\theta \right)={\mathbb{E}}_{\rho }\left[Q_{{\phi }^{\star }\left(\theta \right)}(s,{\mu }_{\theta }\left(s\right))\right]$. Every limit point of the actor is stationary.

Corollary 1

(PL/last-layer linear ⇒ uniqueness and point convergence). If, with frozen targets, the critic’s last-layer (linear) loss is $L_w$-smooth and satisfies PL with ${\mu }_{\mathrm{PL}}>0$, then the critic has a unique global minimizer and the (fast) iterates converge to it; on two time scales, $w_k\to w^{\star }\left({\theta }_k\right)$ along the slow manifold.

Theorem 3

(Finite stepsize neighborhoods with explicit constants). With constant stepsizes $(\alpha ,\beta ,\tau )$, there exist $C_j$ (depending on Lipschitz moduli and noise bounds) such that

$$\underset{k\to \infty }{lim\; sup}\mathbb{E}\left[\mathrm{dist}{\left((w_k,{\theta }_k),\mathcal{M}\times \mathcal{S}\right)}^2\right]\le (C_1+C_4{L}_{\mu }^2)\alpha +(C_2+C_5{L}_{\mu }^2)\beta +(C_3+C_6{L}_{\mu }^2)\tau .$$

(15)

Under PL, the radii admit closed-form expressions in terms of $(\alpha ,\beta ,\tau ,{\sigma }_M^2,L_w,{\mu }_{\mathrm{PL}})$.

Remark 3.

Theorem 2 assumes decreasing stepsizes with $\sum {\alpha }_k=\sum {\beta }_k=\infty$, $\sum ({\alpha }_k^2+{\beta }_k^2+{\tau }_k^2)<\infty$, ${\beta }_k/{\alpha }_k\to 0$, and ${\tau }_k/{\beta }_k\to 0$. Our implementations use constant stepsizes $(\alpha ,\beta ,\tau )$, for which the appropriate justification is provided by Theorem 3: the coupled recursions enter an explicit steady-state neighborhood whose radii depend on $(\alpha ,\beta ,\tau )$ and the Lipschitz/noise constants. We do not claim that Theorem 2 applies to the constant-step regime; rather, Theorem 2 serves as the ideal decreasing-steps benchmark while Theorem 3 supports the practical regime.

In summary, Theorem 2 is formulated under standard two-time-scale stochastic-approximation conditions: (i) globally Lipschitz gradients and bounded noise, (ii) projections onto compact parameter sets, and (iii) stepsizes $({\alpha }_k,{\beta }_k,{\tau }_k)$ such that ${\sum }_k{\alpha }_k={\sum }_k{\beta }_k=\infty$, ${\sum }_k({\alpha }_k^2+{\beta }_k^2+{\tau }_k^2)<\infty$, ${\beta }_k/{\alpha }_k\to 0$, and ${\tau }_k/{\beta }_k\to 0$. Under these assumptions, the critic recursions track the projected gradient flow of their population losses with frozen targets, while the actor follows the projected differential inclusion $\dot{\theta }\in {\Pi }_{\Theta }\left[{\nabla }_{\theta }\tilde{J}\left(\theta \right)\right]$ on the slow time scale.

The PL condition in Corollary 1 strengthens this picture by ruling out spurious stationary points in the last-layer critic loss: it guarantees a unique global minimizer and exponential convergence of the fast critic iterates to $w^{\star }\left(\theta \right)$, so that along the slow manifold, the coupled actor–critic dynamics identify a single value function per policy parameter $\theta$.

Theorem 4

(N-agent CTDE extension with block-norm constants). For N agents with local actors ${\mu }_{{\theta }_i}$ and a joint (or per-agent) critic, under the same two-time-scale regime, the conclusions of Theorem 2 hold and the moduli aggregate as

$$\mathrm{Lip}\left(\mathit{\mu }\right)\le \left\{\begin{array}{cc}\underset{i}{max}{L}_{\mu ,i},\hfill & {\parallel ·\parallel }_{\infty ,\mathrm{blk}},\hfill \\ {\left(\sum _i{L}_{\mu ,i}^2\right)}^{1/2},\hfill & {\parallel ·\parallel }_{2,\mathrm{blk}},\hfill \end{array}\right.L_Q^{\left(\mathrm{in}\right)}\mathit{analogously}.$$

(16)

and the noise variance aggregates by sum (2-norm) or maximum (∞-norm).

2.3. ISS Margins and Steady-State Jitter (Formalized)

Consider the plant $\dot{x}=f(x,d)+g\left(x\right)u$, with $u={\mu }_{\theta }\left(x\right)$ and exogenous input d. Let V be a control Lyapunov function on a compact operating set $\mathcal{X}$, such that

$$\dot{V}\le -\alpha \left(\parallel x-x^{\star }\parallel \right)+c_1\parallel e_{\pi }\left(x\right)\parallel +c_2\parallel e_Q\left(x\right)\parallel ,$$

(17)

for some class-${\mathcal{K}}_{\infty }$ function $\alpha$ and constants $c_{1,2}>0$, where $x^{\star }\in \mathcal{X}$ is a reference state defined below. As illustrated in Figure 1, the critic layer contracts rapidly towards the slow manifold $\mathcal{M}\left(\theta \right)$, so on the slow time scale, the actor evolves with small approximation errors $(e_{\pi },e_Q)$, which is precisely the regime captured by (17).

Assumption 1

(Equilibrium anchoring on $\mathcal{X}$). There exists $x^{\star }\in \mathcal{X}$, such that the reference feedback and value satisfy

$${\mu }_{\theta }\left(x^{\star }\right)={\mu }^{\star }\left(x^{\star }\right),Q_{\phi }\left(x^{\star },{\mu }_{\theta }\left(x^{\star }\right)\right)=Q^{\overline{\pi }}\left(x^{\star },{\mu }_{\theta }\left(x^{\star }\right)\right).$$

(18)

Moreover, the maps ${\mu }_{\theta },{\mu }^{\star },Q_{\phi },Q^{\overline{\pi }}$ are Lipschitz on $\mathcal{X}$ with moduli $L_{{\mu }_{\theta }},L_{{\mu }^{\star }},L_Q,L_{Q^{\overline{\pi }}}$, respectively.

Define the approximation errors on $\mathcal{X}$, recentered at $x^{\star }$,

$$e_{\pi }\left(x\right):={\mu }_{\theta }\left(x\right)-{\mu }^{\star }\left(x\right),e_Q\left(x\right):=Q_{\phi }\left(x,{\mu }_{\theta }\left(x\right)\right)-Q^{\overline{\pi }}\left(x,{\mu }_{\theta }\left(x\right)\right).$$

(19)

By Assumption 1 and the triangle inequality,

$$\parallel e_{\pi }\left(x\right)\parallel \le \parallel {\mu }_{\theta }\left(x\right)-{\mu }_{\theta }\left(x^{\star }\right)\parallel +\parallel {\mu }^{\star }\left(x\right)-{\mu }^{\star }\left(x^{\star }\right)\parallel \le (L_{{\mu }_{\theta }}+L_{{\mu }^{\star }})\parallel x-x^{\star }\parallel ,$$

(20)

and, writing $a\left(x\right):={\mu }_{\theta }\left(x\right)$,

We define the critic mismatch

$$e_Q\left(x\right):=Q_{\varphi }\left(x,a\left(x\right)\right)-Q_{\overline{\pi }}\left(x,a\left(x\right)\right),$$

(21)

and use Lipschitz continuity to obtain

$$\begin{array}{cc}\hfill \parallel e_Q\left(x\right)\parallel & =∥Q_{\varphi }\left(x,a\left(x\right)\right)-Q_{\overline{\pi }}\left(x,a\left(x\right)\right)∥\hfill \\ \hfill & \le ∥Q_{\varphi }\left(x,a\left(x\right)\right)-Q_{\varphi }\left(x_{\star },a\left(x_{\star }\right)\right)∥+∥Q_{\overline{\pi }}\left(x,a\left(x\right)\right)-Q_{\overline{\pi }}\left(x_{\star },a\left(x_{\star }\right)\right)∥\hfill \\ \hfill & \le \left(L_Q+L_{Q_{\overline{\pi }}}\right)\left(\parallel x-x_{\star }\parallel +\parallel a\left(x\right)-a\left(x_{\star }\right)\parallel \right)\hfill \\ \hfill & \le \left(L_Q+L_{Q_{\overline{\pi }}}\right)\left(1+L_{{\mu }_{\theta }}\right)\parallel x-x_{\star }\parallel .\hfill \end{array}$$

(22)

Invoking Theorem 1, the actor Lipschitz modulus satisfies $L_{{\mu }_{\theta }}\le L_{tanh}{L}_f\left(\theta \right)L_{\mu }$, where the fuzzy partition-of-unity (PoU) induces $L_{\mu }=L_z/7$. Substituting (20) and (22) into (17) yields

$$\dot{V}\le -\alpha \left(\parallel x-x^{\star }\parallel \right)+\underset{=:\kappa (L_{\mu },\theta )}{\underbrace{\left[c_1\left(L_{{\mu }_{\theta }}+L_{{\mu }^{\star }}\right)+c_2\left(L_Q+L_{Q^{\overline{\pi }}}\right)\left(1+L_{{\mu }_{\theta }}\right)\right]}}\parallel x-x^{\star }\parallel .$$

(23)

We emphasize that, among the standing assumptions used in this section, the equilibrium anchoring on $X_{\star }$ (Assumption 1), tailored to the PV MPPT operating region under PSCs, is the modeling ingredient that is most specific to the present work.

The remaining conditions—compactness of $\mathcal{S}$, bounded actions, global Lipschitz continuity of the dynamics and actor–critic maps, and square-integrable noise—are in line with standard hypotheses in ISS and stochastic-approximation analyses of actor–critic schemes and are recalled here mainly to keep the exposition self-contained.

Because $L_{{\mu }_{\theta }}$ scales linearly with $L_{\mu }$ (from the fuzzy PoU), decreasing $L_{\mu }$ strictly reduces the gain $\kappa (L_{\mu },\theta )$, thereby enlarging the effective decay margin in (23). This formalizes the empirical observation that smaller $L_{\mu }$ lowers steady-state jitter by improving the ISS gain consistently with the two-time-scale picture in Figure 1.

Remark 4

(Origin-centered fallback with offsets). If an explicit anchor $x^{\star }$ is inconvenient, one may work at the origin and carry offsets:

$$\parallel e_{\pi }\left(x\right)\parallel \le (L_{{\mu }_{\theta }}+L_{{\mu }^{\star }})\parallel x\parallel +\parallel e_{\pi }\left(0\right)\parallel ,\parallel e_Q\left(x\right)\parallel \le \left(L_Q+L_{Q^{\overline{\pi }}}\right)\left(1+L_{{\mu }_{\theta }}\right)\parallel x\parallel +\parallel e_Q\left(0\right)\parallel .$$

In designs that calibrate ${\mu }_{\theta },{\mu }^{\star },Q_{\phi },Q^{\overline{\pi }}$ to match at the operating point (zero-bias last layers, or explicit alignment at $x=0$), the offsets can be made negligible; the same $L_{\mu }$-driven conclusion follows.

Complete proofs for all results in Section 2 are provided in Appendix B.

3. Materials and Methods

3.1. Study Design and Overview

We evaluate a fuzzy-partitioned, centralized training/decentralized execution (CTDE) variant of TD3 (hereafter, Fuzzy–MAT3D) for maximum power point tracking (MPPT) under partial shading. Two local controllers execute with shared parameters, whereas training is centralized from a replay buffer. The design follows a balanced common-random-numbers (CRN) protocol across 7 benchmark scenarios (Table 3) and 20 seeds per algorithm, with a fixed evaluation horizon of $T_{\mathrm{sim}}=20$ s. The empirical protocol, metrics, and statistical analysis were specified a priori and are detailed below. All methods and constants are chosen to satisfy bounded-target and Lipschitz regularity assumptions used in the theory.

3.2. PV Plant and Power-Stage Model

• Single-diode module model.

We adopt the classical single-diode model with series and shunt resistances:

$$\begin{array}{cc}\hfill I& =I_{\mathrm{ph}}(G,T)-I_0\left(T\right)\left(exp{\displaystyle \frac{V+I{R}_s}{n{V}_T}}-1\right)-{\displaystyle \frac{V+I{R}_s}{R_{sh}}},\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill V_T& ={\displaystyle \frac{k_{B}T}{q}},I_{\mathrm{ph}}(G,T)={\kappa }_{G}G+{\kappa }_T\left(T-T_{\mathrm{ref}}\right),\hfill \end{array}$$

(25)

with irradiance G and cell temperature T. Two modules in series yield $V_{\mathrm{string}}=V_1+V_2$, $I_{\mathrm{string}}=I_1=I_2$. Partial shading is represented by $(G_1,G_2)$ profiles as in Table 3. The coefficients $({\kappa }_G,{\kappa }_T)$ are obtained by least–squares calibration from the module datasheet.

Remark 5

(Bypass diodes under PSCs). Commercial modules include substring bypass diodes (see Section 5 and Table 10. For substring ℓ, we considered an augmented branch

$$I_{\mathrm{bd},\ell }=I_{\mathrm{S},\ell }\left(exp\frac{V_{\ell }}{n_{\mathrm{bd}}{V}_T}-1\right){\mathbf{1}}_{\{V_{\ell }<0\}},$$

in parallel with the cell branch. On the two-module bench and shading scripts of Table 3, measured string currents rarely forward-bias bypass diodes. Enabling this branch in post hoc simulations changed η by less than $0.5\%$. Thus the single-diode model suffices for our scenarios.

3.3. DC/DC Stage and Control Interface

Let $d\in [0,1]$ denote the converter duty cycle. The averaged stage dynamics are

$$\dot{x}=f(x,d)+w,y=h\left(x\right),y=(V_{\mathrm{string}},I_{\mathrm{string}}),$$

(26)

and the agent issues an incremental duty command $a\in [-1,1]$ mapped to $d_{t+1}=$$\mathrm{sat}\left(d_t+\Delta d\left(a\right)\right)$.

Throughout all simulations and hardware runs, the incremental duty update is

$$\Delta d\left(a\right):=\mathrm{clip}\left({\kappa }_{d}a,-\Delta d_{max},\Delta d_{max}\right),a\in [-1,1],0<{\kappa }_d\le \Delta d_{max},$$

(27)

followed by

$$d_{t+1}=\mathrm{clip}\left(d_t+\Delta d\left(a_t\right),d_{min},d_{max}\right).$$

(28)

Unless stated otherwise, we set ${\kappa }_d=\Delta d_{max}$ so that $a=\pm 1$ affects the maximum admissible per-tick duty change. No additional slew-rate limiter or nonlinearity is applied beyond the clipping in (27) and (28). For completeness, $\mathrm{clip}(x,\ell ,u):=min\{u,max\{\ell ,x\left\}\right\}$.

3.4. Operating Constraints

We enforce plant and safety limits

$$\begin{array}{cc}\hfill & d_{min}\le d_t\le d_{max},|\Delta d_t|\le \Delta d_{max},\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill & 0\le V_i\le V_{\mathrm{oc},i},0\le I_i\le I_{\mathrm{sc},i},\hfill \end{array}$$

(30)

and impose parameter projection Proj and gradient clipping (critics/actor) to keep all iterates bounded, consistent with the ISS and SA analyses.

3.5. Observations, Actions, and Horizon

Each module provides a local observation $s\in {\mathbb{R}}^7$ bounded componentwise by

$$\begin{array}{cc}\hfill s_{min}& =(0,0,0,200,200,-352.408,-49.9),\hfill \\ \hfill s_{max}& =(49.9,9,352,1000,1000,352.408,49.9).\hfill \end{array}$$

(31)

The local observation $s\in {\mathbb{R}}^7$ includes $(V_i,I_i,P_i,G_i,G_{n\left(i\right)},\Delta P_i,\Delta V_i)$. RL methods consume the full tuple. Classical baselines (P&O, INC, PSO) operate with their standard $(V/I)$ inputs; we nonetheless log $(G_i,G_{n\left(i\right)})$ for all runs to enable like-for-like post hoc analyses and MPP reference computation. Actions are incremental duty-cycle commands in $[-1,1]$. $T_{\mathrm{sim}}=20 \mathrm{s}$; step scenarios use $t_{\mathrm{change}}=0.35 \mathrm{s}$.

RL agents consume the full tuple $s=(V_i,I_i,P_i,G_i,G_{n\left(i\right)},\Delta P_i,\Delta V_i)$ because the irradiance channels $(G_i,G_{n\left(i\right)})$ are available on the intended hardware and stabilize near-MPP behavior under abrupt PSC changes. Classical baselines (P&O, INC, PSO) are deliberately kept on their canonical$(V/I)$ inputs to preserve standard formulations rather than re-tune them into nonstandard variants. All methods share the same wall-clock control period and actuation limits, and we log irradiance for every run (for MPP reference and post hoc analyses). This preserves compute and timing parity while making the sensing assumption explicit.

3.6. Fuzzy Features and Function Approximators

Each state coordinate is normalized to $[0,1]$ and equipped with $L=5$ differentiable memberships ${\left\{{\psi }_{\ell }^{\left(j\right)}\right\}}_{\ell =1}^5$ per coordinate $j\in \{1,\dots ,7\}$, combined via a coordinate-wise softmax so that ${\sum }_{\ell =1}^5{\psi }_{\ell }^{\left(j\right)}\left(s_j\right)\equiv 1$. Stacking the 35 responses yields $z\left(s\right)\in {\mathbb{R}}_{\ge 0}^{35}$ with the global identity ${\mathbf{1}}^{\top }z\left(s\right)=7$; hence, the normalized fuzzy map

$${\phi }_{\mathrm{fuzzy}}\left(s\right)={\displaystyle \frac{z\left(s\right)}{7}},{\mathbf{1}}^{\top }{\phi }_{\mathrm{fuzzy}}\left(s\right)=1.$$

(32)

This induces global input Lipschitz constants for the actor and critics used below. The actor is a two-hidden-layer MLP with a tanh head; twin critics share the fuzzy front-end but maintain separate downstream towers.

3.7. Fuzzy-Partition Parameters

Each raw state coordinate is affinely normalized to $[0,1]$ and equipped with $L=5$ softmax memberships having centers $\{0,0.25,0.5,0.75,1\}$ and a per-coordinate temperature ${\tau }_j>0$. Writing $z_j\left(s_j\right)\in {\mathbb{R}}_{\ge 0}^5$ for the jth coordinate’s membership vector with ${\mathbf{1}}^{\top }{z}_j\left(s_j\right)=1$, we stack $z\left(s\right):=[z_1\left(s_1\right);\dots ;z_7\left(s_7\right)]\in {\mathbb{R}}_{\ge 0}^{35}$, so that ${\mathbf{1}}^{\top }z\left(s\right)=7$.

To avoid overloading the policy notation ${\mu }_{\theta }\left(·\right)$, we denote the fuzzy partition by ${\phi }_{\mathrm{fuzzy}}\left(s\right):=z\left(s\right)/7$. Observe that $\nabla {\phi }_{\mathrm{fuzzy}}\left(s\right)=\nabla z\left(s\right)/7$; for notational consistency with Section 2, we write $L_{\mu }:=L_z/7$ for the global input Lipschitz constant of the normalized PoU.

Table 1. Fuzzy PoU parameters and induced Lipschitz constants. All ranges are given in raw units; centers are in the normalized $[0,1]$ scale. With ${\tau }_j\equiv 32$ for all coordinates, the per-coordinate bound is $L_z\left(\mathrm{coord}.\right)\approx 5.615$ and the global constant is $L_{\mu }=L_z/7\approx 0.802$.

State Coord.	Units	Raw Range	Temperature ${\mathit{\tau }}_{\mathit{j}}$	Centers	${\mathit{L}}_{\mathit{z}}$ (coord.)	Note
$V_i$	V	$[0,49.9]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	module voltage
$I_i$	A	$[0,9]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	module current
$P_i$	W	$[0,352]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	module power
$G_i$	W/m2	$[200,1000]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	local irradiance
$G_{n\left(i\right)}$	W/m2	$[200,1000]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	neighbor irradiance
$\Delta P_i$	W	$[-352.408,352.408]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	last-step power inc.
$\Delta V_i$	V	$[-49.9,49.9]$	32	$\{0,0.25,0.5,0.75,1\}$	$5.615$	last-step voltage inc.

Notation. ${\tau }_j$ denotes the per-coordinate softmax temperature; $L_z={sup}_s\parallel \nabla z\left(s\right)\parallel$ is the global Lipschitz bound of the stacked membership map $z\left(s\right)$; $L_{\mu }=L_z/7$ is the induced Lipschitz constant of the normalized fuzzy partition ${\phi }_{\mathrm{fuzzy}}\left(s\right)=z\left(s\right)/7$; and $\Delta =0.2T_{\mathrm{sim}}=4 \mathrm{s}$ is the continuous window used to define settling time $t_s$ in step scenarios.

reports $\left({\tau }_j\right)$ and the induced $(L_z,L_{\mu })$.

3.8. Reward and Normalization

The saturated, normalized reward used throughout is

$$R=tanh\left(\kappa \left[{\displaystyle \frac{P_1+P_2}{P_{\mathrm{mp},\mathrm{tot}}}}-\alpha {\displaystyle \frac{|\Delta P_1|+|\Delta P_2|}{P_{\mathrm{mp},\mathrm{tot}}}}-\beta {\displaystyle \frac{|\Delta V_1|+|\Delta V_2|}{V_{\mathrm{oc},\mathrm{tot}}}}\right]\right),(\alpha ,\beta ,\kappa )=(0.1,0.05,2),$$

(33)

with physical normalization constants $P_{\mathrm{mp},\mathrm{tot}}=704.8 \mathrm{W}$ and $V_{\mathrm{oc},\mathrm{tot}}=99.8 \mathrm{V}$. This ensures $R\in (-1,1)$ and bounded targets.

3.9. Training Protocol (TD3 Under CTDE)

We employ TD3 with centralized replay and decentralized execution:

• Discount factor $\gamma =0.99$; minibatch size $M=256$; policy update period $K=2$; Polyak factor $\tau ={10}^{-3}$.
• Target policy smoothing noise $\epsilon \in [-0.5,0.5]$, sampled i.i.d. and independent of $s^{\prime }$ conditional on s; target networks $(\overline{\theta },{\overline{\phi }}_i)$ are Polyak-averaged.
• Parameter updates include explicit projections onto compact convex sets $\mathcal{W}$ and $\Theta$; gradient clipping is applied to all updates.

Algorithm 1 summarizes the CTDE training loop used.

Table 2. Training hyperparameters for Fuzzy–MAT3D (CTDE–TD3). Note: Training episodes use $\Delta t=0.06$ s and $T_{\mathrm{steps}}=$ 12,000 ($\approx 720$ s per episode) with $N_{\mathrm{ep}}=80$. All reported evaluation metrics, however, are computed over a fixed evaluation horizon $T_{\mathrm{sim}}=20$ s to ensure comparability across algorithms.

Component	Symbol	Value
Discount factor	$\gamma$	$0.99$
Critic learning rate (fast)	${\alpha }_c$	$1\times {10}^{-4}$
Actor learning rate (slow)	${\beta }_a$	$1\times {10}^{-5}$
Policy update period	K	2
Polyak factor (targets)	$\tau$	$1\times {10}^{-3}$
Target policy noise	${\sigma }_{\epsilon }$	$0.2$ (clip $\pm 0.5$)
Minibatch size	M	256
Replay buffer length	$\left|\mathcal{D}\right|$	${10}^6$
Gradient clipping	$G_{max}$	1
Actor network	—	$(128,128)+tanh$
Twin critics	—	2 (shared fuzzy front-end)
Fuzzy memberships	L	5 per state coordinate
State normalization	—	per-coordinate $[0, 1]$
Reward weights	$(\alpha ,\beta ,\kappa )$	$(0.1,0.05,2)$
Simulation step	$\Delta t$	$0.06 \mathrm{s}$
Max steps/episode	$T_{\mathrm{steps}}$	12,000
Number of episodes	$N_{\mathrm{ep}}$	80
Compute device	—	GPU

summarizes the training hyperparameters adopted for Fuzzy–MAT3D. These values implement a two-time-scale separation (fast critics and slow actor) with constant stepsizes; accordingly, Theorem 3 provides the formal justification for the training regime used in our experiments.

Training episodes used a longer horizon ($\Delta t=0.06$ s; up to 12,000 steps $\approx 720$ s per episode) to improve replay diversity. All reported performance metrics, however, were computed on a fixed evaluation horizon $T_{\mathrm{sim}}=20 \mathrm{s}$ across the seven scenarios to ensure comparability between algorithms.

Algorithm 1 Fuzzy–MAT3D (CTDE–TD3 for two PV modules in series)

Require: $\gamma =0.99$, $M=256$, $K=2$, $\tau ={10}^{-3}$; stepsizes $\left\{{\alpha }_k\right\}$ (critics), $\left\{{\beta }_k\right\}$ (actor).

1: Initialize $\theta ,{\phi }_1,{\phi }_2$; set $\overline{\theta }\leftarrow \theta$, ${\overline{\phi }}_i\leftarrow {\phi }_i$; empty buffer $\mathcal{D}$.

2: for episodes do

3:       Reset irradiances $G_1,G_2\sim U[200,1000]$; initialize $V_i\sim U[20,49.9]$.

4:       for time step t do

5:           Apply $a_t^{\left(i\right)}={\mu }_{\theta }\left(s_t^{\left(i\right)}\right)$; observe $s_{t+1}^{\left(i\right)}$; compute $r_t$ by (33); push to $\mathcal{D}$.

6:           Targets $y=r+\gamma {min}_i{Q}_{{\overline{\phi }}_i}(s^{\prime },{\mu }_{\overline{\theta }}\left(s^{\prime }\right)+\epsilon )$; update ${\phi }_i\leftarrow {Proj}_{\mathcal{W}}\left({\phi }_i-\alpha {\nabla }_{{\phi }_i}\mathrm{MSE}\right)$; Polyak-average ${\overline{\phi }}_i$.

7:            if $tmodK=0$then ascend $\theta \leftarrow {Proj}_{\Theta }\left(\theta +\beta {\nabla }_{\theta }J\left(\theta \right)\right)$; Polyak–average $\overline{\theta }$.

8:           end if

9:       end for

10: end for

3.10. Benchmark Scenarios, CRN Protocol, and Metrics

Seven benchmark scenarios are used; step cases share the same change time $t_{\mathrm{change}}=0.35 \mathrm{s}$, as shown in

Table 3. Benchmark scenarios.

#	Scenario Name	${\mathit{G}}_{\mathbf{init}}$ [W/m2]	${\mathit{t}}_{\mathbf{change}}$ [s]	${\mathit{G}}_{\mathbf{final}}$ [W/m2]
1	Standard Condition	[1000, 500]	∞	[1000, 500]
2	Deep Shadow	[1000, 200]	∞	[1000, 200]
3	Similar Conditions	[800, 850]	∞	[800, 850]
4	Low Irradiation	[400, 300]	∞	[400, 300]
5	Step Change 1 (step)	[1000, 300]	0.35	[300, 1000]
6	Step Change 2 (step)	[900, 600]	0.35	[600, 900]
7	Shadow Recovery (step)	[250, 950]	0.35	[950, 950]

.

CRN protocol. All algorithms use identical random seeds, initializations, and shading scripts within each scenario to enable paired, blocked inferences. We run 20 independent seeds per scenario (total $N=140$ replications/algorithm).

At each sampling instant, the reference power is computed via a 1D constrained search

$$P_{\mathrm{mpp}}\left(t\right)=\underset{V\in [0,V_{\mathrm{oc},\mathrm{tot}}]}{max}\left\{V{I}_{\mathrm{string}}\left(V;G_1\left(t\right),G_2\left(t\right),T\left(t\right)\right)\right\}.$$

(34)

The primary endpoint is MPPT efficiency

$$\eta [\%]=100{\displaystyle \frac{{\int }_0^{20 \mathrm{s}}{P}_{\mathrm{pv}}\left(t\right)dt}{{\int }_0^{20 \mathrm{s}}{P}_{\mathrm{mpp}}\left(t\right)dt}}\approx 100{\displaystyle \frac{{\sum }_k{P}_{\mathrm{pv}}\left(t_k\right)}{{\sum }_k{P}_{\mathrm{mpp}}\left(t_k\right)}}.$$

(35)

• Secondary endpoints.

For step scenarios, we report (i) the settling time $t_s$, defined as the smallest $t\ge 0.35 \mathrm{s}$ such that $|P_{\mathrm{pv}}\left(u\right)-P_{\mathrm{mpp}}\left(u\right)|\le 0.02P_{\mathrm{mpp}}\left(u\right)$ for all $u\in [t,t+\Delta ]$ with $\Delta =0.2\times T_{\mathrm{sim}}=4 \mathrm{s}$; and (ii) steady-state oscillation, the standard deviation of $P_{\mathrm{pv}}\left(t\right)$ over the last $20\%$ of $20 \mathrm{s}$.

• Normalization note.

$P_{mp,tot}=704.816\mathrm{W}$ and $V_{oc,tot}=99.8\mathrm{V}$ come from calibrated limits of the single-diode model used for normalization over the two-module operating domain, and thus closely match—but do not strictly equal—the STC datasheet values of the series string ($700\mathrm{W}$ and $92.82\mathrm{V}$; Table 10).

3.11. Baselines and Fairness Constraints

All algorithms share the same control period and actuation limits (27) and (28). For PSO, we cap the per-tick particles × iterations so the search finishes within the 60 ms deadline (

Table 4. Inference latency (ms) vs. control period and PSO per-tick budget.

Algorithm	p50	p95	p99	Control Period [ms]	PSO Budget (Particles × Iters)
Fuzzy–MAT3D	1.5	2.7	4.0	60	—
MAT3D	1.3	2.4	3.7	60	—
MADDPG	1.4	2.6	3.9	60	—
PSO	—	—	—	60	$16\times 3$
P&O/INC	—	—	—	60	—

). RL methods perform exactly one forward pass per module per tick. Classical baselines operate with their standard $(V/I)$ inputs; irradiance $(G_i,G_{n\left(i\right)})$ is logged and used only for reference MPP computation and post hoc analyses, keeping budget parity in wall-clock and actuation.

Under the 60 ms control period, both PSO’s particles × iterations and the RL forward passes are strictly confined to this wall-clock budget; no baseline was granted extra evaluations or sensing beyond its canonical inputs.

3.12. Computational Budget and Real-Time Feasibility

RL inference requires exactly one forward pass per tick per module. We log wall-clock inference latencies (p50/p95/p99) to verify meeting the control deadline. The PSO controller is constrained by an identical per-tick computational budget.

Table 4 reports inference latencies (p50/p95/p99) measured on the evaluation host used for simulation; Section 5 (Table 12) reports on-device latencies measured on the hardware bench under the same 60 ms period.

3.13. Statistical Analysis Plan

Primary confirmatory analysis comprises (i) a one-way ANOVA on $\eta$ across algorithms; and (ii) blocked Dunnett contrasts versus Fuzzy–MAT3D (blocking by scenario and seed). Secondary analyses include CRN-paired one-sided t-tests (Fuzzy–MAT3D minus comparator) with Benjamini–Hochberg FDR control across hypotheses, and linear mixed-effects models with random intercepts for scenario and seed. We report distributional summaries and two-sided $95\%$ Student-t confidence intervals; diagnostics include normality of residuals and Levene’s test for homoscedasticity.

When executed on physical hardware, we mirror the seven scenarios of Table 3 with synchronized $V/I$ sensing and logging and enforce the same limits and deadlines as in simulation. Each algorithm is evaluated with $n\ge 10$ bench replicates per scenario, blocked by scenario and replicate ID. The reference $P_{\mathrm{mpp}}\left(t\right)$ is computed from a calibrated single-diode model parameterized by measured $(G,T)$.

4. Results

We report both learning behavior and fixed-horizon performance under a balanced common-random-numbers (CRN) design comprising 7 scenarios and 20 seeds per algorithm ($N=140$ replications/algorithm). The primary endpoint is MPPT (Maximum Power Point Tracking) efficiency, $\eta$ (Def. (35)); secondary endpoints quantify transient speed and steady operation: settling time $t_s$ (2% band, window $\Delta =0.2\times T_{\mathrm{sim}}=4 \mathrm{s}$) and steady-state oscillation (SD of $P_{\mathrm{pv}}$ over the last $20\%$ of $20 \mathrm{s}$). Unless stated otherwise, all summaries are CRN-blocked; means are reported with two-sided 95% Student-t confidence intervals; and hypothesis testing follows the pre-specified hierarchy of a global one-way ANOVA, confirmatory blocked-Dunnett contrasts versus Fuzzy–MAT3D, and CRN-paired one-sided t tests with BH–FDR control.

We begin with training dynamics for the RL controllers (Figure 2), then present the CRN-blocked comparison across six algorithms via distributional views and mean-CI summaries (Figure 3 and Figure 4). Stability metrics are analyzed next: steady-state oscillation (Figure 5) and settling time (Figure 6), with effect sizes and paired inferences tabulated in Tables 7 and 8 alongside aggregate RL performance (Table 9). We subsequently dissect the speed–stability trade-off (Section 4.3), provide ablations and sensitivity checks, and include an illustrative step disturbance case study. Exploratory Tukey–Kramer intervals are relegated to Appendix A to avoid conflating descriptive and confirmatory claims.

Axes and table headers use a unified nomenclature: “MPPT efficiency, $\eta$ [%]’’, “settling time, $t_s$ [s]’’, and “steady-state oscillation [W]’’. All figures and tables reflect the same CRN blocking and horizon $T_{\mathrm{sim}}=20 \mathrm{s}$ to ensure like-for-like comparisons across algorithms and scenarios.

4.1. Training Dynamics (RL Group)

We begin by characterizing the learning behavior of the three RL controllers under the same CTDE protocol and logging setup. Figure 2 reports the evolution of the cumulative return and a moving-average return across training episodes. Two features are salient: (i) the Fuzzy–MAT3D trajectories exhibit visibly damped volatility and earlier stabilization relative to MAT3D and MADDPG, and (ii) improvements accrue more steadily once the critics have entered their fast-contraction regime, consistent with the variance-reduction and non-expansivity mechanisms developed in Section 2. These dynamics foreshadow the downstream fixed-horizon advantages—higher MPPT efficiency and lower steady-state oscillation—documented in the comparative analyses that follow.

4.2. Global Comparison Across Six Algorithms (CRN-Blocked)

This subsection presents a global comparison of six algorithms under a CRN-blocked design, matching stochastic trajectories to control heterogeneity and reduce estimator variance. We report means, 95% confidence intervals, standardized effect sizes (Hedges’ g), and average ranks, and conduct blocked ANOVA with Dunnett/Holm corrections for multiple comparisons against the reference.

Table 5. Simulation: scenario-wise MPPT efficiency $\eta$ (%); mean ± 95% CI; $N=20$ seeds per scenario and algorithm.

Scenario	Fuzzy–MAT3D	PSO	MAT3D	INC	P&O	MADDPG
Standard Condition	$94.5\pm 0.9$	$89.0\pm 2.0$	$84.0\pm 3.1$	$81.0\pm 3.0$	$79.0\pm 3.3$	$73.0\pm 4.0$
Deep Shadow	$90.2\pm 1.5$	$79.0\pm 2.5$	$71.0\pm 3.8$	$68.0\pm 3.6$	$64.0\pm 3.6$	$57.0\pm 4.5$
Similar Conditions	$93.0\pm 1.1$	$86.0\pm 2.1$	$82.0\pm 3.2$	$78.0\pm 3.1$	$76.0\pm 3.3$	$70.0\pm 4.1$
Low Irradiation	$89.5\pm 1.6$	$82.0\pm 2.4$	$76.0\pm 3.6$	$73.0\pm 3.3$	$71.0\pm 3.4$	$65.0\pm 4.2$
Step Change 1	$91.0\pm 1.6$	$85.0\pm 2.2$	$79.0\pm 3.4$	$76.0\pm 3.2$	$74.0\pm 3.3$	$67.0\pm 4.2$
Step Change 2	$92.5\pm 1.2$	$87.0\pm 2.0$	$83.0\pm 3.1$	$79.0\pm 3.0$	$78.0\pm 3.2$	$71.0\pm 4.0$
Shadow Recovery	$93.3\pm 1.1$	$85.8\pm 2.1$	$85.0\pm 3.0$	$80.0\pm 3.1$	$80.0\pm 3.3$	$72.0\pm 4.1$

reports the scenario-wise MPPT efficiency (mean ± 95% CI) with $N=20$ seeds per scenario and algorithm. Across all seven scenarios, Fuzzy–MAT3D attains the highest mean efficiency and consistently narrow confidence intervals for example, $94.5\pm 0.9\%$ under Standard Condition and $90.2\pm 1.5\%$ under Deep Shadow. These per-scenario results mirror the aggregate advantages shown in Figure 3 and Figure 4 and reinforce the robustness of the fuzzy-regularized approach across both static and step-change conditions.

• Inferential statistics.

A global one-way ANOVA on $\eta$ rejects equality of means (F-statistic $= 88.965$, $p = 5.127\times {10}^{-75}$; $N = 140$ per group), confirming the significant differences in performance distributions visually apparent in Figure 3 and Figure 4. Following Section 3.13, we report blocked Dunnett in the supplement (see, e.g., the exploratory post hoc analysis in Figure A1) and CRN-paired tests below.

Residual Q–Q plots (to check normality) and Levene’s test (for homoscedasticity) were performed to validate the ANOVA assumptions. No gross departures from normality were found, and Levene’s test did not reject homoscedasticity within groups at $\alpha =0.05$; all confirmatory inferences therefore follow the pre-specified hierarchy.

4.3. Settling Time vs. Stability

As shown by the CRN-blocked boxplots in Figure 6, Fuzzy–MAT3D exhibits a longer settling time $t_s$ than the plain TD3 baseline (MAT3D) and MADDPG. The RL-only aggregate (Table 9) reports $t_s=7.537\pm 5.397$ s for Fuzzy–MAT3D, versus $1.573\pm 3.771$ s for MAT3D and $3.239\pm 6.175$ s for MADDPG.

In our protocol, $t_s$ is the first time after the step at which the trajectory remains within a $\pm 2\%$ band around the instantaneous MPP for a contiguous window of length $\Delta =0.2\times T_{\mathrm{sim}}=4 \mathrm{s}$ (see Section 3.10). This definition is agnostic to (i) how much overshoot/undershoot occurred before entering the band and (ii) whether the controller subsequently leaves the band again after the $\Delta$-window has elapsed. Hence, a controller can register a small $t_s$ by grazing the band early with aggressive moves yet sustain sizable steady-state jitter or even drift later; conversely, a more conservative controller can register a larger $t_s$ while delivering substantially better long-horizon behavior.

Mechanistic explanation for Fuzzy–MAT3D’s larger $t_s$. The fuzzy PoU front-end enforces global Lipschitzness on the actor/critics (Theorem 1) and, together with target smoothing, yields a locally non-expansive fixed-policy operator (Proposition 1); moreover, the twin-critic minimum introduces a small, correlation-dependent negative bias (Lemma 1). These ingredients reduce TD-target variance and damp fast transients, but they also make the closed loop deliberately conservative right after abrupt shading steps, prioritizing a monotone approach over rapid excursions. In short, Fuzzy–MAT3D trades a few seconds of responsiveness for markedly improved stability and bias robustness.

• Why Fuzzy–MAT3D is still preferable.

Three lines of evidence—empirical, statistical, and control-theoretic—support Fuzzy–MAT3D despite its larger $t_s$:

• Dominant long-horizon energy capture. Across the CRN–blocked study $\left(N=140\right.$ per algorithm), Fuzzy–MAT3D achieves the highest mean MPPT efficiency ($92.032\%\pm 4.014\%$), substantially above MAT3D ($80.144\%\pm 13.050\%$) and MADDPG ($67.978\%\pm 12.402\%$); CRN-paired tests yield large effects (Cohen’s d up to $1.874$) and essentially zero q-values (BH–FDR). Thus, any energy loss from a slower transient is more than offset by sustained operation near the MPP over the full horizon (

  Table 6. Mean MPPT efficiency $\eta$ (mean ± SD) and two-sided 95% CI across $N=140$ CRN-blocked replications per algorithm.

  Algorithm	Mean [%]	SD	95% CI [%]	N
  Fuzzy–MAT3D	92.032	4.014	[91.361, 92.703]	140
  PSO	84.758	8.028	[83.417, 86.100]	140
  MAT3D	80.144	13.050	[77.963, 82.324]	140
  INC	76.074	11.667	[74.125, 78.024]	140
  P&O	74.312	11.471	[72.395, 76.229]	140
  MADDPG	67.978	12.402	[65.905, 70.050]	140

  ,

  Table 7. CRN-paired, one-sided t-tests of Fuzzy–MAT3D vs. baselines on MPPT efficiency. Positive $\Delta \eta$ favors Fuzzy–MAT3D. p is one-sided; CIs are two-sided 95%. Cohen’s d is the paired effect size.

  Comparator	${\mathit{n}}_{\mathbf{pairs}}$	$\mathit{\Delta }\mathit{\eta }$ [%]	95% CI [%]	t	p(BHq)	Cohen’s d
  MADDPG	140	+24.054	[21.909, 26.199]	22.173	<${10}^{-16}$ (≈0)	1.874
  P&O	140	+17.720	[15.874, 19.566]	18.978	<${10}^{-16}$ (≈0)	1.604
  INC	140	+15.958	[13.964, 17.951]	15.826	<${10}^{-16}$ (≈0)	1.338
  MAT3D	140	+11.888	[9.674, 14.103]	10.613	<${10}^{-16}$ (≈0)	0.897
  PSO	140	+7.274	[5.756, 8.792]	9.474	<${10}^{-16}$ (≈0)	0.801

  and

  Table 8. CRN-paired, one-sided t-tests on steady-state oscillation (lower is better). Positive differences $(\mathrm{baseline}-\mathrm{Fuzzy}–MAT3D)$ favor Fuzzy–MAT3D.

  Comparator	${\mathit{n}}_{\mathbf{pairs}}$	Mean Diff [W]	95% CI [W]	t	p (BH q)	Cohen’s d
  MADDPG	140	+51.159	[45.656, 56.662]	18.380	<${10}^{-16}$ (≈0)	1.553
  MAT3D	140	+44.078	[35.072, 53.085]	9.676	<${10}^{-16}$ (≈0)	0.818
  P&O	140	+17.081	[13.864, 20.297]	10.499	<${10}^{-16}$ (≈0)	0.887
  INC	140	+15.228	[12.328, 18.127]	10.383	<${10}^{-16}$ (≈0)	0.878
  PSO	140	+8.948	[7.464, 10.432]	11.921	<${10}^{-16}$ (≈0)	1.008

  ).
• Much lower steady-state jitter. Figure 5 shows the steady-state oscillation of Fuzzy–MAT3D tightly concentrated near zero, whereas MAT3D and MADDPG display broad, heavy-tailed distributions. The RL-only aggregate reports $1.362\pm 2.013$ W for Fuzzy–MAT3D vs. $37.963\pm 70.396$ W (MAT3D) and $52.993\pm 34.608$ W (MADDPG). Lower jitter not only improves energy capture but also reduces switching stress and thermal cycling in the power stage (

  Table 9. Aggregate performance across all RL runs.

  Algorithm	MPPT Efficiency $\mathit{\eta }$ [%]	Settling Time ${\mathit{t}}_{\mathit{s}}$ [s]	SS Oscillation [W]
  Fuzzy–MAT3D	$92.032\pm 4.014$	$7.537\pm 5.397$	$1.362\pm 2.013$
  MAT3D	$80.144\pm 13.050$	$1.573\pm 3.771$	$37.963\pm 70.396$
  MADDPG	$67.978\pm 12.402$	$3.239\pm 6.175$	$52.993\pm 34.608$

  ).
• Design intent: stability margins over aggressiveness. Theoretically, the fuzzy-induced Lipschitz constant $L_{\mu }$ improves ISS gains, while the min-ensemble plus target smoothing controls overestimation and high-frequency actuation. This combination is expected to enlarge decay margins but reduce “snap-to-setpoint” behavior—precisely the speed–stability trade-off seen in Figure 6.

The distribution in Figure 6 shows Fuzzy–MAT3D with a higher median $t_s$ and a long right tail driven by the most abrupt step cases, which is consistent with its conservative transient policy. Yet, when read jointly with Figure 5 (steady-state oscillation) and the efficiency summaries (Figure 3 and Figure 4; Table 6 and Table 9), the picture is unequivocal: Fuzzy–MAT3D sits on a better Pareto front—maximizing energy and minimizing jitter—while conceding some transient speed. In applications like PV MPPT under PSC, where (i) steps are intermittent and (ii) the objective is integral energy over minutes–hours, this Pareto choice is the correct one.

The ANOVA and CRN-paired tests in Table 6, Table 7 and Table 8 indicate that the superiority of Fuzzy–MAT3D over MAT3D and MADDPG is statistically significant across all seven scenarios, with very small p-values and large paired effect sizes.

Because the design includes both static PSC profiles and step-change scenarios, the reported efficiency $\eta \approx 92.0\%\pm 4.0\%$ should be interpreted as a robust average over a representative family of practically relevant shading patterns.

We do not claim universal optimality beyond this family, but the ISS and Lipschitz analysis suggest that the qualitative advantages of Fuzzy–MAT3D should persist under other smooth, slowly varying PSC profiles; extending the experimental design to more aggressive, rapidly varying shading remains an interesting direction for future work.

Because $t_s$ declares success after any continuous $\Delta$-window within the band, it cannot penalize later departures from the band. This explains why algorithms with aggressive, oscillatory responses can display deceptively small $t_s$ while still underperforming in energy and stability. Our CRN-blocked analysis therefore treats $t_s$ as a secondary indicator to be interpreted alongside efficiency and steady-state metrics (Figure 5, Table 6, Table 7, Table 8 and Table 9).

Fuzzy–MAT3D is intentionally conservative around abrupt changes; this yields a larger settling time but confers superior stability and decisively better energy tracking. In the aggregate, and for the operational goals of MPPT under partial shading, Fuzzy–MAT3D’s trade-off is the desirable one.

Concretely, comparing the aggregate statistics in Table 6 and Table 9, Fuzzy–MAT3D sacrifices about 6 s of additional settling time on average (7.54 s vs. 1.57 s for MAT3D) in exchange for roughly $+12$ percentage points in MPPT efficiency (92.0% vs. 80.1%) and a reduction of $\approx 36$ W in steady-state jitter (1.36 W vs. 37.96 W), which is an advantageous trade-off for energy-centric applications.

As complementary, descriptive evidence, Appendix A, Figure A1 reports Tukey–Kramer confidence intervals for all pairwise algorithmic contrasts. Consistent with the CRN-blocked summaries and directionally aligned with the confirmatory blocked-Dunnett analysis, these intervals place Fuzzy–MAT3D above both MAT3D and MADDPG in mean MPPT efficiency (pairwise CIs against Fuzzy–MAT3D exclude zero), with the largest deficit observed for MADDPG. We therefore treat this panel as supportive context for the superiority ordering rather than an independent inferential claim.

4.4. RL-only Aggregate Across Replications

Table 9 aggregates RL-only results across CRN-blocked replications, showing that Fuzzy–MAT3D attains the highest mean MPPT efficiency $\eta$ with markedly lower steady-state oscillation than MAT3D and MADDPG. Conversely, Fuzzy–MAT3D exhibits a larger settling time $t_s$, reflecting the study’s speed–stability trade-off and the conservative regulation induced by the fuzzy PoU front-end.

The empirical distribution of MPPT efficiency across all CRN-blocked replications is shown in Figure 7, complementing the notched boxplots and mean–CI panels by revealing the full shape and tails of the distributions. This panel contextualizes central-tendency summaries with dispersion and skewness across scenarios and seeds.

4.5. Case Study: Step Disturbance

As shown in Figure 8, after the step at $t=0.35 \mathrm{s}$, Fuzzy–MAT3D approaches $P_{\mathrm{MPP}}$ monotonically and maintains it with negligible steady-state jitter, whereas MADDPG overshoots and develops sustained oscillations that depress the average power; MAT3D converges slowly and under-tracks the MPP. The lower panel explains the mechanism: Fuzzy–MAT3D rapidly desaturates the duty cycle and then holds an almost constant command, while the other agents continue exciting the plant—consistent with the ISS-based stability rationale in Section 2.3.

5. Experimental Validation

The hardware experiments were conducted on a two-module series string whose electrical ratings closely match the normalization constants used in the simulations. For clarity, $P_{mp,tot}=704.816\mathrm{W}$ and $V_{oc,tot}=99.8\mathrm{V}$ arise from the calibrated model limits employed for normalization, not strictly from the (Standard Test Conditions) STC datasheet values of the two-module string ($700\mathrm{W}$ and $92.82\mathrm{V}$;

Table 10. Electrical, temperature, and mechanical specifications of the 350 W PV modules used in experiments (STC) and the resulting two-module series string.

Parameter	Per Module (STC)	Series String (2×)
Rated power $P_{\mathrm{mp}}$ [W]	350	700
Voltage at MPP $V_{\mathrm{mp}}$ [V]	38.95	77.90
Current at MPP $I_{\mathrm{mp}}$ [A]	8.98	8.98
Open-circuit voltage $V_{\mathrm{oc}}$ [V]	46.41	92.82
Short-circuit current $I_{\mathrm{sc}}$ [A]	9.60	9.60
Cells (size)	72 cells, $156\times 156$ mm
Cell efficiency [%]	20.40
Module efficiency [%]	18.00
Output tolerance [%]	5
Standard test conditions	AM1.5, $1000\mathrm{W}/{\mathrm{m}}^2$, 25 °C
Temperature coefficients (per module)
$\mathrm{d}{V}_{\mathrm{oc}}/\mathrm{d}T$ [%/K]	$-0.30$
$\mathrm{d}{I}_{\mathrm{sc}}/\mathrm{d}T$ [%/K]	$+0.04$
$\mathrm{d}{P}_{\max }/\mathrm{d}T$ [%/K]	$-0.44$
Nominal operating cell temperature (NOCT)	45 ± 2 °C
Mechanical (per module)
Dimensions [mm]	1956 × 992 × 40
Weight [kg]	23
Operating temperature [°C]	$-40$ to 85
Frame	Anodized aluminum
Junction box	IP67, 3 diodes
Output cables	$4{\mathrm{mm}}^2$, 1000 mm length
Connectors	MC4 compatible/Tyco
Connection topology	—	Two identical modules in series

).

Table 10 details the per-module specifications and the resulting values for the series connection adopted in the laboratory setup. This configuration ensures that the experimental plant reproduces the same operating envelope as the simulated one, thereby enabling a direct comparison of maximum power point tracking (MPPT) performance under partial shading scenarios.

5.1. Hardware Validation: Objectives, Bench, and Real-Time Deployment

We validate Fuzzy–MAT3D on real hardware to test three pre-registered hypotheses: (i) it preserves its MPPT efficiency advantage, $\eta$, over all baselines; (ii) it reduces steady-state oscillation; and (iii) it meets the real-time budget under the same sensing/actuation constraints. The seven benchmark scenarios of Table 3 are replayed verbatim for hardware runs, and metrics follow the simulation definitions in Section 3 (efficiency $\eta$, settling time $t_s$ with a 2% band and window $\Delta =0.2\times T_{\mathrm{sim}}=4 \mathrm{s}$, and steady-state oscillation).

The experimental bench comprises two PV modules in series with reproducible, controlled shadows; a DC/DC stage (boost or buck-boost) instrumented with voltage/current sensing and governed by an MCU/SoC; and standard protection elements (V/I limits and watchdog). Power is measured with a DC power analyzer, the shunt+DAQ path is employed only for calibration checks, all signals are timestamp-synchronized, and module temperatures are monitored via thermocouples. This setup mirrors the plant assumptions in Section 3 and enables like-for-like comparisons with the simulation protocol.

The deployed actor ${\mu }_{\theta }$ is identical to its simulated counterpart: it uses the per-coordinate fuzzy partition of unity (coordinate-wise softmax), so that ${\mathbf{1}}^{\top }z\left(s\right)=7$ and $L_{\mu }=L_z/7$ hold exactly as in Equations (1) and (2). To assess real-time feasibility, we log inference latency (p50/p95/p99) and the fraction of control deadlines missed over each run under the same sensing/update period used for the baselines; these logs are reported alongside the performance metrics.

The overall architecture of the experimental bench and measurement points is summarized in Figure 9. Figure 10 shows the instrumented bench used in all hardware runs: a two-module series string connected to the embedded controller and a laptop that logs synchronized voltage–current traces under. Figure 11 complements this view by documenting the site-level deployment (left) and the tidy series interconnect with measurement leads (right); the panels are oriented to avoid self-shading from the cabling, and the controller–laptop pair provides the time alignment required by our MPPT metrics.

5.2. Protocols and Scenarios

We executed the seven scenarios in Table 3 on the physical bench described in Section 5.1 (two PV modules in series, DC/DC power stage, synchronized V/I sensing and logging). Each run lasted 20 $\mathrm{s}$; step-change scenarios used a reproducible shadowing script with change time $t_{\mathrm{change}}=0.35s$, implemented by calibrated masks/shutters, while static scenarios used fixed mask configurations.

For every scenario, each algorithm was evaluated with $n\ge 10$ hardware replicates ($7\times 10=70$ per algorithm), blocked by scenario and replicate ID. Before every run, we restored the plant to a standardized initial condition (open-circuit reset and controlled pre-charge), verified sensor zeroing, and logged the initial terminal voltages and duty-cycle state. The identical irradiance/shadowing scripts and reset procedure were replayed across algorithms to ensure like-for-like comparisons on real hardware.

The reference $P_{\mathrm{mpp}}\left(t\right)$ is computed from a calibrated single-diode model parameterized by the measured $(G,T)$; the model parameters are calibrated offline using periodic I–V sweeps. At evaluation time, the instantaneous MPP is obtained via a 1D search on the model. We compute $\eta$ (Equation (35)), settling time $t_s$ (2% band, window $\Delta =0.2\times T_{\mathrm{sim}}=4 \mathrm{s}$, steady-state oscillation (last 20%), lost energy, and latency.

5.3. Experimental Results

Table 11. Hardware: scenario-wise MPPT efficiency $\eta$ (%); mean ± 95% CI; $n\ge 10$ per scenario/algorithm.

Scenario	Fuzzy–MAT3D	PSO	MAT3D	INC	P&O	MADDPG
Standard Condition	$93.5\pm 1.1$	$87.0\pm 2.2$	$83.0\pm 3.8$	$76.0\pm 3.1$	$77.5\pm 3.0$	$72.0\pm 4.2$
Deep Shadow	$89.0\pm 1.8$	$78.0\pm 2.8$	$73.0\pm 4.5$	$70.0\pm 3.4$	$70.5\pm 3.6$	$61.0\pm 4.6$
Similar Conditions	$92.2\pm 1.3$	$85.0\pm 2.3$	$81.0\pm 3.9$	$75.0\pm 3.0$	$76.0\pm 3.1$	$70.0\pm 4.1$
Low Irradiation	$88.8\pm 1.9$	$80.5\pm 2.5$	$75.0\pm 4.0$	$72.0\pm 3.2$	$72.5\pm 3.3$	$64.0\pm 4.3$
Step Change 1	$91.1\pm 1.8$	$83.5\pm 2.4$	$79.5\pm 3.7$	$74.0\pm 3.0$	$75.0\pm 3.1$	$67.0\pm 4.0$
Step Change 2	$92.0\pm 1.4$	$86.0\pm 2.2$	$84.0\pm 3.3$	$76.5\pm 3.1$	$79.0\pm 3.0$	$72.0\pm 3.8$
Shadow Recovery	$92.4\pm 1.2$	$82.5\pm 2.5$	$79.0\pm 3.5$	$75.0\pm 3.0$	$78.0\pm 3.1$	$69.0\pm 4.2$

reports the scenario-wise hardwareMPPT efficiency (mean ± 95% CI; $n\ge 10$ per scenario/algorithm). Across all seven scenarios, Fuzzy–MAT3D consistently leads—for instance, $93.5\%\pm 1.1\%$ under Standard Conditionand $89.0\%\pm 1.8\%$ under Deep Shadow—mirroring the aggregate hardware advantages shown in Figure 12 and Figure 13. These results indicate that the gains of the fuzzy-regularized controller persist across static and step-change conditions.

To verify real-time feasibility on the bench, we report median and tail inference latencies (p50/p95/p99) under the same 60 ms control period used throughout the study. As shown in

Table 12. Hardware inference latency (ms) vs. control period and PSO per-tick budget.

Algorithm	Latency [ms]			Control Period [ms]	PSO Budget (Particles × Iters)
	p50	p95	p99
Fuzzy–MAT3D	1.8	3.3	4.9	60	—
MAT3D	1.6	3.0	4.6	60	—
MADDPG	1.7	3.1	4.7	60	—
PSO	—	—	—	60	$16\times 3$
P&O/INC	—	—	—	60	—

Notes: Latency percentiles (p50/p95/p99) are measured on the hardware bench under the same sensing/update period used across algorithms. PSO latency is not tabulated because its compute is budgeted per tick.

, all RL controllers exhibit sub 5 ms p99 latency on hardware, comfortably below the 60 ms deadline, while the PSO controller is constrained by an identical per-tick budget (16 × 3).

5.4. Discussion of Experimental Results

The hardware summary in

Table 13. Hardware summary: MPPT efficiency $\eta$ [%], settling time $t_s$ [s], and steady-state oscillation [W] (mean ± SD) across all hardware replications.

Algorithm	$\mathit{\eta }$ [%]	${\mathit{t}}_{\mathit{s}}$ [s]	Steady–State Oscillation [W]
Fuzzy–MAT3D	$91.296\pm 4.057$	$8.097\pm 4.612$	$7.837\pm 8.607$
PSO	$83.233\pm 7.919$	$9.623\pm 3.941$	$25.921\pm 26.580$
MAT3D	$78.791\pm 12.493$	$2.585\pm 2.264$	$45.899\pm 53.620$
P&O	$75.447\pm 11.794$	$3.609\pm 2.510$	$40.015\pm 23.916$
INC	$74.086\pm 10.990$	$4.645\pm 3.943$	$36.724\pm 29.736$
MADDPG	$67.515\pm 12.881$	$4.958\pm 4.276$	$62.451\pm 38.205$

confirms a decisive advantage for Fuzzy–MAT3D in cumulative energy capture: it attains the highest mean MPPT efficiency ($\eta =91.296\pm 4.057$%), surpassing all classical and RL baselines—PSO ($83.233\pm 7.919$), MAT3D ($78.791\pm 12.493$), P&O ($75.447\pm 11.794$), INC ($74.086\pm 10.990$), and MADDPG ($67.515\pm 12.881$). Figure 12 places the entire efficiency distribution of Fuzzy–MAT3D at the upper end among competitors, while Figure 13 shows its mean with tight $95\%$ confidence intervals centered near $91\%$, consistent with the low empirical variability reported in the table. The magnitude of this efficiency gap is not cosmetic: with identical sensing, actuation, and computation budgets, the fuzzy partition yields a controller that consistently converts available irradiance into electrical work more effectively over the whole horizon.

A core strength of Fuzzy–MAT3D in hardware is its markedly lower steady-state power ripple: $7.837\pm 8.607$ W, compared against $25.921\pm 26.580$ W (PSO), $36.724\pm 29.736$ W (INC), $40.015\pm 23.916$ W (P&O), $45.899\pm 53.620$ W (MAT3D), and $62.451\pm 38.205$ W (MADDPG). Figure 14 makes this contrast visually salient: the Fuzzy–MAT3D box is compressed near the origin, while non-fuzzy RL baselines are both higher in level and heavier-tailed. This reduction in jitter has two immediate consequences: (i) it directly improves integral efficiency (less dithering around the MPP), and (ii) it reduces switching and thermal stress on the power stage, an operational benefit that rarely appears in single-number leaderboards but matters in deployment. Mechanistically, these outcomes align with the theory: the differentiable fuzzy partition of unity imposes global input Lipschitzness on the actor–critic towers, the smoothed fixed-policy operator is non-expansive, and target variance contracts with the fuzzy Lipschitz modulus—together damping high-frequency actuation and steady-state ripple (Lemma 1, Proposition 1, Proposition 2).

Figure 15 and Table 13 show a deliberate speed–stability trade-off. Fuzzy–MAT3D exhibits a longer settling time ($t_s=8.097\pm 4.612$ s) than MAT3D ($2.585\pm 2.264$ s) and P&O/INC (about 3–5 s), though notably not the slowest overall (PSO averages $9.623\pm 3.941$ s). Interpreted in context, this is an intentional design choice: the fuzzy regularizer and TD3 target smoothing bias the closed loop toward the monotone approach with bounded slopes, avoiding overshoot-induced re-entries into (and exits from) the tolerance band that can make $t_s$ look artificially small while harming long-horizon yield. In other words, a few additional seconds of conservative transient are exchanged for a far more stable steady state, which is precisely where PV systems spend most of their time.

Read jointly, Figure 12, Figure 13, Figure 14 and Figure 15 and Table 13 portray a coherent Pareto frontier: Fuzzy–MAT3D simultaneously maximizes energy yield and minimizes steady-state oscillation, conceding some transient speed relative to aggressive baselines. For PV MPPT under partial shading—where step disturbances are intermittent and the objective is cumulative energy over minutes to hours—this Pareto point is preferable. The behavior matches the theory-first analysis: fuzzy-induced global Lipschitzness and the two-time-scale TD3 dynamics (critics fast and actor slow) enlarge practical ISS margins and shrink TD-target variance; at the hardware level, those constants materialize as tighter power control with minimal ripple and robust tracking near the MPPT. Complementarily, the convergence framework for fuzzy–MAT3D (CTDE–TD3) guarantees that, under bounded targets, Lipschitz networks, and projected iterates, the actor asymptotically follows a well-posed gradient flow on a compact set—explaining the stable learning and execution seen here.

In hardware, the fuzzy-augmented controller is not merely more “accurate” on average; it is structurally better behaved. Its low oscillation regime reduces wear-and-tear and converts more irradiance into usable energy, and its measured $t_s$ reflects conservative transients rather than sluggish control. For safety-critical, efficiency-driven PV deployments, this combination—high $\eta$, minimal jitter, and bounded transients—constitutes a compelling operational advantage.

6. Discussion

Our working hypothesis was that inserting a differentiable fuzzy partition of unity in front of the actor–critic would (i) enforce global input Lipschitzness and lower TD-target variance, (ii) render fixed-policy evaluation non-expansive with the correct $L^{\infty }$ contraction factor $\kappa =\gamma$, (iii) enlarge the closed-loop ISS margin, and (iv) convert constant stepsizes into explicit steady-state neighborhoods; these mechanisms are formalized in Section 2.2.

Empirically, the resulting controller attains higher MPPT efficiency with markedly lower steady-state jitter while accepting a more conservative transient—precisely the speed–stability trade-off expected from the theory and consonant with prior observations that classical P&O/INC and PSO approaches tend to exchange rapid steps for oscillatory behavior under PSCs, whereas unregularized RL baselines (e.g., MAT3D and MADDPG) can amplify critic noise into unstable actuation. In the broader context of learning-based power electronics, the fuzzy layer acts as a structural regularizer that improves actor–critic conditioning and yields closed-loop behavior aligned with energy-centric objectives and hardware stress constraints, thereby offering a principled alternative to ad hoc damping or heuristic dithering. These interpretations are consistent with the controlled CRN-blocked study and the theory-first analysis reported herein.

We establish stationarity (not global optimality) and adopt a replay idealization; nonetheless, parameter projections, clipping, and bounded target noise narrow the assumption–implementation gap. The residual bounds depend on distribution shift (Remark 1), and the practical ISS gains in Section 2.3 inherit the usual modeling idealizations.

Future Work

Building on the present results, future work will (i) close the distribution-shift loop by learning replay/behavior policies that better align $\rho$ and its pushforward, tightening the constants in the projected Bellman bounds (Proposition 3). (ii) Automate the partition design (number of memberships and temperature) and study its effect on variance and twin-critic correlation, building on Proposition 2 and Lemma 1. (iii) Couple ISS-style margins (Section 2.3) with barrier certificates and latency/quantization models for hardware-level safety guarantees. (iv) Extend the CTDE analysis to larger N and partial observability, and benchmark against entropy-regularized and trust-region variants under identical sensing/compute budgets. (v) Translate the energy-centric gains to longer horizons and field deployments, including degradation, sensor drift, and grid-level constraints.

Beyond the specific two-module string considered here, the fuzzy-regularized CTDE–TD3 architecture is applicable to other domains where (i) the dynamics admit a compact operating envelope and (ii) safety or comfort requirements demand smooth closed-loop responses. Examples include cooperative voltage control in DC microgrids, coordinated charging of electric-vehicle fleets, and frequency regulation with distributed energy resources, where the PoU structure can encode network topology or operating regions. From the scaling viewpoint, Theorem 4 already shows that the Lipschitz moduli of the N-agent extension grow in a controlled way under block norms, so that Fuzzy–MAT3D can in principle be deployed on larger PV arrays by assigning one agent per string or module cluster. In such settings, additional work is needed to account for partial observability and communication constraints, but the core Lipschitz and ISS guarantees remain valid.

7. Conclusions

Fuzzy–MAT3D provides a theoretically grounded and empirically validated controller for PV MPPT under partial shading. From an energy-systems perspective, raising MPPT efficiency to around 92% under PSCs, while suppressing steady-state oscillations, directly translates into higher energy yield and reduced stress on power-electronic interfaces, contributing to more reliable and predictable PV generation at scale. The corrected theory includes a properly contracted fixed-policy operator ($\kappa =\gamma$ in $L^{\infty }$), a projected Bellman residual bound with distribution-shift clarification, an explicit two-time-scale theorem with a PL uniqueness corollary, finite-stepsize neighborhoods with constants, a min-bias analysis for TD3, and a clean N-agent CTDE extension.