ST-CDF: A Generative AI Framework for Physics-Consistent Imputation and Simulation in Precision Agriculture

Abstract

Incomplete spatio-temporal (ST) data from sensor networks in precision agriculture often limits environmental modeling and decision-making accuracy. To address this, we propose the Spatio-Temporal Conditional Diffusion Framework (ST-CDF), a generative approach for high-fidelity data reconstruction. The framework’s core is a deep denoising network that integrates a Graph Attention Network (GAT) to explicitly model non-Euclidean spatial correlations, a Differential Attention Transformer to capture abrupt temporal dynamics, and an Inverse Discrete Wavelet Transform (IDWT) module to preserve multi-scale signal details. The generative process is constrained by a physics-informed training objective, which injects known physical laws (i.e., the Penman–Monteith equation for reference evapotranspiration, $E{T}_0$) as an inductive bias, ensuring the imputed data maintains physical consistency. For privacy-preserving deployment on resource-constrained IoT devices, we extend the framework with a Federated Cluster-Guided Distillation (Fed-CGD) strategy. We conducted extensive experiments against established methods on two real-world agricultural datasets. ST-CDF demonstrated improved imputation accuracy across evaluated metrics. Its efficacy was most pronounced in the physically-demanding $E{T}_0$ calculation task, where data imputed by ST-CDF at an 80% missing rate achieved a Root Mean Square Error (RMSE) of 0.3485 and a Coefficient of Determination ($R^2$) of 0.7558, outperforming the baseline models. Furthermore, we explore ST-CDF as an explainable (XAI) framework for active agricultural decision support, demonstrating its utility in performing counterfactual simulations of “what-if” interventions, such as irrigation. The findings highlight ST-CDF as an effective, physically-grounded, and interpretable tool for data-driven scientific computation and precision agriculture.

1. Introduction

Multivariate time series prediction and classification have been widely applied across modern data-driven domains [1,2,3]. In precision agriculture, meteorological forecasting, and hydrological research, accurate and continuous environmental monitoring data form the foundation for resource optimization, disaster warning, and scientific decision-making. For example, sensor networks in precision agriculture collect real-time soil moisture and temperature data to support irrigation management and crop growth analysis [4,5,6], while observations from weather stations serve as essential inputs for climate modeling and environmental assessment [7,8,9].

However, smart agriculture deployments in remote rural areas often suffer from unreliable connectivity and limited bandwidth. Agricultural IoT systems typically consist of geographically dispersed devices, where continuous data transmission is frequently disrupted by network fluctuations or energy constraints. Under such conditions, long contiguous periods of missing values, referred to as data chasms, inevitably emerge when communication with cloud servers is interrupted. These data gaps render traditional cloud-centric imputation strategies ineffective and necessitate robust, edge-resident data reconstruction mechanisms. Incomplete spatio-temporal (ST) data from these sensor networks significantly limits environmental modeling and decision-making accuracy. Furthermore, as smart agriculture rapidly evolves toward the paradigm of Agriculture 5.0 [10], there is an increasingly pressing need for Human-Centered AI (HCAI) solutions. In this context, merely improving raw predictive accuracy is insufficient; imputation frameworks must also prioritize model transparency, operational trust, and human-in-the-loop decision support to be practically viable for end-users.

In addition, agricultural sensing data frequently contain sensitive information related to crop yields and land locations. Centralized training paradigms require uploading raw data to a central server, raising significant privacy concerns and incurring prohibitive communication costs over narrow-bandwidth rural networks. Federated Learning (FL), pioneered by FedAvg [11], enables collaborative model training without sharing raw data [12]. Nevertheless, directly applying FL to large-scale generative models remains challenging, as iterative transmission of full model parameters places an excessive burden on communication-constrained agricultural IoT systems. Recent advances in Federated Distillation [13] alleviate this limitation by exchanging compact student models instead of full parameters. Notably, data chasms spanning long time intervals can significantly distort intrinsic spatio-temporal dynamics, leading to biased analyses and, in extreme cases, erroneous conclusions [14,15,16]. Therefore, developing imputation methods capable of high-fidelity reconstruction on the edge remains of both theoretical importance and practical urgency [17,18,19].

To address these challenges, considerable progress has been made in recent years, which can be broadly categorized into deterministic paradigms, generative approaches, and edge-deployment techniques. Traditional statistical approaches like ARIMA [20] and MICE [21] heavily rely on stationarity assumptions. To capture more complex dynamics, deep learning variants have been introduced. RNN-based models like BRITS [22,23,24] have enhanced step-wise temporal modeling. Concurrently, Transformer architectures have dominated the field. While recent advanced models like iTransformer [25,26] have revolutionized time series analysis by inverting the dimensional treatment, their functional characteristic is inherently optimized for predicting future horizons. For data imputation, architectures equipped with specialized masked attention mechanisms or cross-variable fusion, such as SAITS [27,28,29] and the highly efficient linear-complexity ImputeFormer [17], provide a more technically appropriate functional baseline. Meanwhile, generative models like GANs [30,31,32] and evolutionary variants [33] have been utilized. More recently, diffusion models [34,35,36] have emerged as a robust alternative. Following TimeGrad [37], structured models such as SSSD [38], CSDI [39], and PriSTI [40] serve as foundational representatives, while cutting-edge frameworks like CoFILL [41] represent the latest state-of-the-art in collaborative spatio-temporal diffusion. Furthermore, recent advancements seek to integrate physical constraints into the diffusion process (e.g., PIDM [42], PhyDA [43], PISD [44], PIDDM [45], and SG-Diff [46]). However, effectively integrating diffusion-based generative modeling with intrinsic physical laws and differential spatio-temporal dynamics remains an open challenge [47].

Despite these advancements, existing solutions exhibit specific limitations when applied to the complex spatio-temporal dynamics of agricultural environments. First, deterministic models often exhibit significant degradation during block missing events due to the lack of an explicit generative prior. As purely data-driven estimators, they are prone to generating values that are statistically probable yet physically contradictory. Second, current diffusion architectures predominantly utilize standard scaled dot-product attention [41,48]. These mechanisms act as low-pass filters and are poorly suited for isolating transient anomalies or abrupt environmental changes (e.g., sudden rainfall). Moreover, while some models capture spatio-temporal dependencies, they remain “physics-blind,” treating variables as independent statistical tokens rather than thermodynamically coupled entities. Finally, deploying heavy diffusion models on IoT devices presents a fundamental paradox: the iterative reverse generation process requires massive computational footprints. Traditional distillation often fails to capture the complex, multi-step generative trajectories of diffusion models, making direct deployment on solar-powered rural sensors infeasible.

To address these critical limitations, this work proposes the Spatio-Temporal Conditional Diffusion Framework (ST-CDF) for the high-fidelity imputation of multivariate sensor time series data. ST-CDF integrates conditional diffusion modeling with structured spatio-temporal feature extraction and physics-informed constraints, enabling physically consistent reconstruction even in the presence of long contiguous data chasms. A deep spatio-temporal denoising network jointly extracts spatial topological information and temporal dynamics to guide the reverse diffusion process, ensuring that generated data are both statistically accurate and physically plausible.

The main contributions of this study are summarized as follows:

• We propose a diffusion-based framework specifically designed for spatio-temporal data imputation, leveraging structured spatio-temporal conditions to guide the generative process.
• We design a deep denoising network that integrates graph-based spatial modeling and differential temporal attention to effectively capture coupled spatio-temporal dependencies, with an IDWT-based module preserving multi-scale signal characteristics.
• We introduce a physics-informed training objective to enforce physical consistency and demonstrate the framework’s utility for explainable analysis and counterfactual simulation in agricultural decision support.
• We propose a cluster-guided distillation strategy and extend it to a federated setting (Fed-CGD). This approach reduces communication overhead by over 80% and computational complexity by 15-fold, enabling real-time, privacy-preserving imputation on resource-constrained edge devices.

Extensive experiments on two large-scale real-world datasets—a long-term soil moisture dataset and a regional meteorological dataset—demonstrate that ST-CDF provides a competitive and robust alternative to state-of-the-art methods in both imputation accuracy and downstream task performance. The results further validate its robustness in handling long missing intervals and extreme environmental events, highlighting its strong potential as an interpretable tool for data-driven scientific computation in precision agriculture.

2. Materials and Methods

To comprehensively evaluate the performance of the model proposed in this paper on the data imputation task, this study selected two real-world datasets with distinctly different physical characteristics.In the experimental workflow, the raw sensor data first undergoes preprocessing steps, including normalization and outlier handling. Subsequently, the data is fed into a module composed of a Multi-head Differentiable Attention Transformer and a Graph Attention Network for Spatio-Temporal feature extraction. The extracted features are then used to train the core diffusion model. To enhance inference efficiency, we further apply knowledge distillation to the trained model, and the final data imputation is completed using the resulting lightweight student model.

2.1. System Overview

To address the prevalent challenges of missing and anomalous data in large-scale sensor networks, this paper proposes an innovative data imputation framework that integrates deep spatio-temporal feature extraction, a conditional diffusion model, and knowledge distillation. The framework aims to achieve high-fidelity reconstruction of missing data by deeply mining the intrinsic spatio-temporal dependencies within the data and leveraging the powerful distribution-fitting capabilities of generative models. The workflow of the entire system is illustrated in the figure, with its core composed of five interconnected stages: data preprocessing, spatio-temporal feature extraction, diffusion model training, model distillation, and data imputation. Each module is tightly linked to the next, collectively ensuring the accuracy, robustness, and efficiency of the imputation task. The architecture of ST-CDF is shown in Figure 1.

2.2. Dataset Construction and Preprocessing

To evaluate the model’s capability for spatio-temporal data imputation, this study utilizes two long-term, real-world sensor datasets collected from operational agricultural IoT networks.

Soil Moisture Dataset: Deployed in a standardized experimental field in Fukang, Xinjiang, this network records multi-layer volumetric soil water content from 18 June 2024, to 1 August 2025 (hourly frequency). Its dense spatial deployment provides an ideal platform for validating the model’s ability to leverage cross-site spatial correlations.

Meteorological Dataset: Collected from an automatic weather station in Xinxiang, Henan Province, this dataset records hourly meteorological information (wind speed, temperature, relative humidity, solar radiation, and precipitation) from 13 June 2020, to 28 May 2025. The strong physical coupling among its variables makes it suitable for evaluating physical consistency via downstream tasks (e.g., $E{T}_0$ calculation).

To simulate real-world forecasting scenarios and strictly prevent data leakage, both datasets were chronologically partitioned into training (70%), validation (15%), and test (15%) sets, rather than using random splitting.

2.3. Data Preprocessing and Task Formulation

2.3.1. Anomaly Detection and Normalization

Operational edge sensing often suffers from sensor drift and environmental noise. To establish a standardized evaluation protocol, the raw data underwent a rigorous two-stage cleaning pipeline. First, univariate outliers violating physical plausibility were removed using the $3\sigma$ rule. Second, to identify subtle multivariate anomalies arising from complex variable interactions, we utilized the non-parametric Isolation Forest (iForest) algorithm [49,50,51]. All identified anomalous data points were replaced with Not-a-Number (NaN) values to serve as the ground truth for the imputation task.

Subsequently, to eliminate dimensional differences and stabilize model convergence, all sensor variables were scaled to the interval $[0,1]$ using Min–Max normalization. Crucially, to prevent target leakage, all normalization statistics and anomaly detection thresholds were fitted exclusively on the training set and then applied to the validation and test sets.

2.3.2. Imputation Task Formulation

To rigorously evaluate the model’s reconstruction fidelity, we artificially applied binary masks M to the complete time series to simulate distinct IoT failure modes:

• Random Point Masking (Main Evaluation): Independent Bernoulli sampling was applied with a masking probability $p\in [30\%,80\%]$ to simulate intermittent packet loss and random sensor noise.
• Continuous Block Masking (Case Study): Continuous sequences of time steps (e.g., 120 h gaps) were removed to simulate prolonged hardware failures or power outages.

Given the complete data x, the model is tasked with reconstructing the masked unknown values $x_{miss}=(1-M)\odot x$ relying solely on the observed context $x_{obs}=M\odot x$.

2.3.3. Generative Framework Based on Spatio-Temporal Conditional Diffusion

To effectively mitigate the problem of missing data in agricultural sensor networks, we applied the ST-CDF framework, as illustrated in Figure 2. Rather than relying on simple interpolation, this approach utilizes a conditional diffusion process to reconstruct missing records while adhering to the physical constraints and spatio-temporal couplings inherent in the environmental data. The core mechanism involves systematically adding noise to the data distribution and subsequently training a reverse process to denoise and reconstruct the continuous time series.

The procedure begins with a forward diffusion process, modeled as a fixed Markov chain that incrementally injects Gaussian noise into the complete historical data $x_0$ over T steps [52,53]. The noisy state $x_t$ at any timestep t can be directly computed as:

$$q\left(x_t\mid x_0\right)=\mathcal{N}\left(x_t;\sqrt{{\overline{\alpha }}_t}{x}_0,\left(1-{\overline{\alpha }}_t\right)\mathbf{I}\right)$$

(1)

where $ϵ\sim \mathcal{N}(0,\mathbf{I})$ is standard Gaussian noise, and ${\overline{\alpha }}_t={\prod }_{i=1}^t{\alpha }_i$ is a pre-defined variance schedule. This transforms the complex observational data into a tractable Gaussian distribution.

Subsequently, a neural network ${ϵ}_{\theta }$ is trained to reverse this process [54,55]. To ensure the imputed values are consistent with the actual agricultural context, we incorporate known sensor observations as a condition mask C, concatenated with the noisy input $x_t$. The model is optimized using a Mean Squared Error (L2) objective:

$$\mathcal{L}={\mathbb{E}}_{t,x_0,ϵ,C}{∥ϵ-{ϵ}_{\theta }\left(\sqrt{{\overline{\alpha }}_t}{x}_0+\sqrt{1-{\overline{\alpha }}_t}ϵ,t,C\right)∥}^2$$

(2)

2.3.4. Spatio-Temporal Feature Extraction Module

To process the structural complexity of the sensor data, the denoising network utilizes specific modules for spatial and temporal dependencies.

For spatial correlations, we employ a Graph Attention Network (GAT). The agricultural variables form a graph structure $\mathcal{G}=(\mathcal{V},\mathcal{E})$. For the soil moisture dataset, we utilize a static geo-spatial graph where node adjacency is determined by physical Euclidean distance, reflecting local topological continuity. For the meteorological dataset, we employ a dynamic graph where edge weights are continuously updated via self-attention to capture transient interactions (e.g., the strong coupling between humidity and temperature during precipitation). The attention coefficient ${\alpha }_{ij}$ for node i receiving information from neighbor j is calculated as:

$${\alpha }_{ij}={\displaystyle \frac{exp\left(LeakyReLU\left({\mathbf{a}}^T\left[\mathbf{W}{h}_i\parallel \mathbf{W}{h}_j\right]\right)\right)}{{\sum }_{k\in {\mathcal{N}}_i}exp\left(LeakyReLU\left({\mathbf{a}}^T\left[\mathbf{W}{h}_i\parallel \mathbf{W}{h}_k\right]\right)\right)}}$$

(3)

where $h_i$ represents the node features and W is the weight matrix. The updated representation $h_i^{\prime }$ is obtained by $\sigma \left({\sum }_{j\in {\mathcal{N}}_i}{\alpha }_{ij}\mathbf{W}{h}_j\right)$.

To capture temporal dynamics, we introduce a modified Multi-Head Differential Attention mechanism. In agricultural environments, relative variations (such as sudden soil moisture increases following irrigation) are often more informative than absolute values. A single-head differential attention score combines standard similarity with a difference function:

$$score\left({\mathbf{Q}}_i,{\mathbf{K}}_j\right)={\displaystyle \frac{\left({\mathbf{Q}}_i{\mathbf{W}}_Q\right){\left({\mathbf{K}}_j{\mathbf{W}}_K\right)}^T}{\sqrt{d_k}}}+f_{\mathrm{diff}}\left({\mathbf{Q}}_i{\mathbf{W}}_Q,{\mathbf{K}}_j{\mathbf{W}}_K\right)$$

(4)

The function $f_{\mathrm{diff}}$ integrates the difference vector between the query and key:

$$f_{\mathrm{diff}}(\mathbf{A},\mathbf{B})={\displaystyle \frac{1}{\sqrt{d_k}}}\sum \left(ReLU\left((\mathbf{A}-\mathbf{B}){\mathbf{W}}_{diff}\right)\right)$$

(5)

This adjustment allows the model to respond more sensitively to abrupt environmental transitions.

Finally, to separate meaningful environmental changes from high-frequency sensor noise, the framework processes features in the frequency domain using the Discrete Wavelet Transform (DWT). The signals are decomposed into low-frequency approximation coefficients $\mathbf{c}$ and high-frequency detail coefficients $\mathbf{d}$ via a Haar wavelet. The Inverse Discrete Wavelet Transform (IDWT) is then used to reconstruct the time series $\widehat{x}$:

$$\widehat{x}=IDWT(\widehat{\mathbf{c}},\widehat{\mathbf{d}})=\sum _k{\widehat{\mathbf{c}}}_k{\varphi }_k\left(t\right)+\sum _j\sum _k{\widehat{\mathbf{d}}}_{j,k}{\psi }_{j,k}\left(t\right)$$

(6)

where $\varphi$ and $\psi$ are the scaling and wavelet functions. This ensures the output maintains global phenological trends while preserving essential local fluctuations.

2.3.5. Physics-Informed Conditional Generation and Training

While the standard diffusion process is unconditional, agricultural data imputation requires the generated sequences to be strictly constrained by known sensor observations. We reframe the reverse denoising process into an iterative conditional generation process. At each timestep t, the model predicts the unconstrained denoised result $x_{t-1}^{\prime }$. To enforce consistency with the known observations $x_o$, we compute the true noisy state $x_{o,t-1}$ according to the forward process, and inject it into the known regions defined by the binary mask m:

$$x_{t-1}=m\odot x_{o,t-1}+(1-m)\odot x_{t-1}^{\prime }$$

(7)

By repeatedly executing this calibration mechanism, the sampling trajectory is guided towards a valid environmental data manifold that strictly aligns with the observed context.

During the training phase, the mask m is concatenated with the noisy data $x_t$ as an additional input condition. The base training objective is the standard diffusion loss ${\mathcal{L}}_{diff}$:

$${\mathcal{L}}_{diff}={\mathbb{E}}_{t,x_0,ϵ,m}{∥ϵ-{ϵ}_{\theta }\left(x_t,t,m\right)∥}^2$$

(8)

To ensure the imputed meteorological data adheres to natural thermodynamic laws, we introduce a physics-informed loss term ${\mathcal{L}}_{phy}$. Let $\mathcal{F}(·)$ denote the standard FAO-56 Penman–Monteith equation for calculating hourly reference evapotranspiration ($E{T}_0$). The model’s estimated clean outputs ${\widehat{x}}_0$ are de-normalized to their original physical units prior to the calculation:

$${\mathcal{L}}_{phy}={∥\mathcal{F}(DeNorm\left({\widehat{x}}_0\right))-\mathcal{F}(DeNorm\left(x_0\right))∥}^2$$

(9)

The final objective is ${\mathcal{L}}_{total}={\mathcal{L}}_{diff}+\lambda {\mathcal{L}}_{phy}$, where $\lambda$ controls the strength of the physical constraint (set to 0.1 for the meteorological dataset). For the soil moisture dataset, where exact hydro-geological parameters for the Richards equation are unavailable, we intentionally set $\lambda =0$ to prevent parameter-induced bias. Instead, the framework relies on the GAT and differential attention modules to implicitly learn and maintain the underlying spatio-temporal physical consistency of water propagation. Post-processing clipping is additionally applied during inference to ensure all generated values fall within valid physical domains (e.g., relative humidity bounded to [0, 100]).

To rigorously clarify the operational mechanism of the proposed objective function, it is imperative to distinguish between the training and inference phases, particularly regarding the absence of ground truth data.

The total loss function (${\mathcal{L}}_{total}$), encompassing both the data-driven diffusion loss (${\mathcal{L}}_{diff}$) and the physics-informed penalty (${\mathcal{L}}_{phy}$), is strictly computed only during the training phase. During training, the model is optimized using a completely observed historical dataset. To simulate real-world missingness, artificial masks are applied to the input; however, the original complete data (Ground Truth) remains accessible to the objective function to calculate the gradients and update the model parameters ($\theta$). It is through this training process that the thermodynamic constraints of the Penman–Monteith equation are deeply encoded into the neural network’s weights.

Conversely, during the inference (testing or deployment) phase, where the ground truth of the masked data is genuinely missing, no loss is calculated, and no weights are updated. The model operates entirely in a forward-pass generative mode. It reconstructs the missing values by executing the reverse diffusion process using the frozen, optimized parameters. Because the physical laws have been parameterized into the network during training, the generated imputations inherently adhere to the physical constraints without requiring real-time loss computation or ground truth validation.

2.4. Model Compression and Federated Edge Deployment

The iterative sampling of diffusion models imposes a heavy computational burden for deployment on resource-constrained agricultural edge devices (e.g., solar-powered weather stations or remote soil nodes). To resolve this, we introduce a Cluster-Guided Distillation (CGD) strategy to transfer the spatio-temporal knowledge from the complex Teacher model (${\theta }_T$) into a lightweight Transformer-based Student model (${\theta }_S$) capable of single-step inference.

To identify the most representative knowledge, we applied the standard K-Means algorithm to cluster the hidden feature representations of the Teacher model’s intermediate layers. One layer closest to each cluster centroid was selected as a “Hint Layer.” The Student model is then optimized using a tripartite hybrid loss:

$${\mathcal{L}}_{\mathrm{total}}={\mathcal{L}}_{\mathrm{MSE}}+\alpha {\mathcal{L}}_{\mathrm{soft}}+\beta {\mathcal{L}}_{\mathrm{hint}}$$

(10)

where ${\mathcal{L}}_{\mathrm{MSE}}$ calculates the error against the ground truth, ${\mathcal{L}}_{\mathrm{soft}}$ aligns the Student’s final prediction with the Teacher’s output, and ${\mathcal{L}}_{\mathrm{hint}}$ forces the Student’s internal features to mimic the Teacher’s extracted representations via a linear projection layer:

$${\mathcal{L}}_{\mathrm{hint}}={∥F_{\mathrm{teacher}}-(W_{proj}·F_{\mathrm{student}})∥}^2$$

(11)

Furthermore, to address the limited cellular bandwidth typical of remote agricultural monitoring networks, we extend this distillation process into a Federated Learning paradigm (Fed-CGD). Instead of transmitting the parameter-heavy Teacher model, each edge client k utilizes its private local dataset ${\mathcal{D}}_k$ to optimize a local Student model ${\theta }_S^k$ by mimicking a pre-trained local Teacher. Clients then transmit only the parameters of the lightweight Student model (${\theta }_S$) to the central server for aggregation:

$${\theta }_S^{global}={\displaystyle \sum _{k=1}^K}{\displaystyle \frac{|{\mathcal{D}}_k|}{\left|\mathcal{D}\right|}}{\theta }_S^k$$

(12)

This strategy reduces the communication overhead from $\mathcal{O}\left(\right|{\theta }_T\left|\right)$ to $\mathcal{O}\left(\right|{\theta }_S\left|\right)$, enabling efficient, privacy-preserving collaborative learning across the distributed agricultural IoT framework.

3. Results

3.1. Evaluation Metrics

To comprehensively and quantitatively evaluate the performance of our proposed Spatio-Temporal Conditional Diffusion Framework (ST-CDF) on the data imputation task, this study selected four core evaluation metrics: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Coefficient of Determination ($R^2$), and Continuous Ranked Probability Score (CRPS). While MAE, RMSE, and $R^2$ focus on the deterministic accuracy and goodness of fit, CRPS is specifically introduced to evaluate the quality of uncertainty quantification provided by the generative models.

In the following formulas, n represents the total number of imputed sample points, $y_i$ represents the ground truth value, ${\widehat{y}}_i$ represents the model’s predicted value (mean of samples), and $\overline{y}$ represents the average of all ground truth values.

$$MAE={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\left|y_i-{\widehat{y}}_i\right|$$

(13)

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}$$

(14)

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}{{\sum }_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}$$

(15)

To assess the probabilistic calibration of the diffusion model, the CRPS measures the difference between the predicted cumulative distribution function (CDF) $F_i$ and the empirical observation indicator function. It is defined as:

$$CRPS={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\int }_{-\infty }^{\infty }{\left(F_i\left(z\right)-\mathbb{I}\left(z\ge y_i\right)\right)}^{2}dz$$

(16)

where $\mathbb{I}(·)$ is the indicator function which equals 1 if the condition is true and 0 otherwise. A lower CRPS indicates that the model’s generated distribution is both accurate (centered on the truth) and sharp (low uncertainty).

3.2. Experimental Setup and Baseline Models

3.2.1. Hardware and Software

The computational environment for this study was configured as follows: The core deep learning model training and inference tasks were executed on a high-performance computing node equipped with an NVIDIA RTX 4090 GPU (24 GB VRAM). Data preprocessing and subsequent analysis were handled by a server equipped with an Intel Xeon Platinum 8352V processor and 64 GB of RAM. On the software level, the entire experimental environment was built on Python 3.10, using PyTorch 2.4.0 as the deep learning framework and the CUDA 11.8 toolkit for GPU acceleration. To ensure the stability and reproducibility of all experimental results, we set a fixed random seed for all processes. To enhance model training efficiency, we incorporated the Flash_Attention algorithm, an advanced technique that accelerates the attention mechanism by optimizing memory access and computation patterns. The application of this algorithm, combined with data augmentation strategies, not only significantly reduced the training cycle of our Transformer models but also guaranteed high data utilization efficiency, thereby effectively enhancing the model’s scalability.

3.2.2. Hyperparameter Settings and Validation Strategy

The specific numerical values of the hyperparameters for the Spatio-Temporal Conditional Diffusion Framework were meticulously selected relying on a combination of hardware constraints, empirical tuning, and systematic grid search analysis.

First, regarding the validation strategy, it is critical to clarify that traditional K-fold cross-validation was explicitly avoided in this study. Given the strong temporal autocorrelation inherent in meteorological and soil moisture time series, standard random cross-validation would introduce severe data leakage by using future data to predict past events. Instead, we employed a strict chronological hold-out validation strategy. The hyperparameter tuning and model selection were strictly guided by the model’s performance on the 15% chronological validation set, ensuring objective evaluation.

For the core architectural hyperparameters, specifically the number of Transformer encoding layers N and the total diffusion steps T, we conducted a comprehensive grid search to assess their impact on the model’s output quality. Guided by the performance plateau observed in the validation RMSE, we selected $N=6$ and $T=500$ as the optimal configuration to balance high-fidelity physical imputation with computational efficiency.

Other training hyperparameters were empirically tuned to ensure stable convergence while respecting hardware limitations. To maximize the utilization of the 24 GB VRAM on the RTX 4090 without encountering out-of-memory errors, the batch_size was set to 32. To ensure sufficient representational capacity for the complex multivariate features, the embedding dimension was set to 256 with 8 attention_heads, accompanied by a dropout rate of 0.25 to prevent overfitting.

For the optimization process, the initial learning_rate was configured to $0.0002$. The model was optimized using the Adam optimizer with its default momentum parameters of ${\beta }_1=0.9$ and ${\beta }_2=0.99$. A ReduceLROnPlateau scheduler was employed for dynamic adjustment, halving the learning rate when the validation loss did not improve for 10 consecutive epochs. Training was conducted for a maximum of 500 epochs, with a strict early stopping policy applied: if the validation loss showed no improvement for a patience of 200 consecutive epochs, the training was terminated early to save computational resources and lock in the best generalized weights.

3.2.3. Baseline Models

For model evaluation, we selected several mainstream and cutting-edge time series imputation baseline models for comparison, including MICE [21], GAIN [56], Transformer, SAITS [57], CSDI [39], PriSTI [40], ImputeFormer [17], and CoFILL [41]. Each baseline model demonstrates different conceptual paradigms and performance characteristics in handling missing data tasks. MICE is a representative non-deep learning method: it is a powerful multivariate iterative approach that builds a predictive model for each variable with missing values and fills them in cyclically. GAIN represents the adversarial generative approach, which learns the intrinsic distribution of the data through a minimax game between a generator and a discriminator to produce realistic imputed values. Among the attention-based deep learning models, we included the standard Transformer and its advanced variant SAITS. The standard Transformer models global correlations between all time points, while SAITS builds upon this by introducing a special masked attention mechanism, allowing it to more efficiently learn the complex relationships between observed and missing values. For early diffusion-based methods, CSDI utilizes conditional score-based diffusion to explicitly model the conditional distribution of missing values given observations, whereas PriSTI further specializes this approach for spatio-temporal data by employing a noise matching network with conditioned attention. Finally, to rigorously validate our framework against the latest advancements, we incorporated two recent state-of-the-art models: ImputeFormer, a highly efficient linear-complexity Transformer that achieves top-tier performance by intricately fusing temporal dynamics with cross-variable dependencies, and CoFILL, an advanced diffusion framework utilizing a collaborative noise-matching strategy to achieve highly accurate spatio-temporal imputation under extreme sparsity.

3.3. Comparison with Different Mainstream Models

To rigorously ensure statistical robustness and preclude the possibility of cherry-picking favorable outcomes, all experimental models—including our proposed ST-CDF and the baselines—were evaluated across five independent runs using different random initialization seeds. Consequently, all quantitative metrics reported in this section are presented as the mean value ± the standard deviation ($\sigma$). This rigorous evaluation protocol confirms that the performance advantages of our framework are statistically significant and highly stable.

To thoroughly investigate the ability of each model to recover variables with different physical characteristics, we systematically evaluated their imputation accuracy on five core meteorological variables: wind speed (m/s), air temperature (C), relative humidity (%), solar radiation (MJ/m²), and precipitation (mm). Figure 3 shows the Root Mean Squared Error (RMSE) curves for each model on the five variables as the data missingness rate was progressively increased from 30% to 80%.

As shown in Figure 3, the experimental results delineate two distinct performance tiers. Traditional statistical methods (e.g., MICE) and standard deep learning models (including GAIN, Transformer, and SAITS) are generally outperformed by the advanced diffusion-based frameworks, specifically CSDI, PriSTI, and our proposed model. As the data missingness rate increases, all models exhibit a decline in performance; however, the non-diffusion baselines show a particularly steep degradation, underscoring their limitations in capturing complex distributions under sparse data conditions. In contrast, the diffusion-based methods, particularly CSDI, PriSTI, and our ST-CDF, maintain significantly flatter error growth curves. Crucially, our model further surpasses these strong diffusion baselines, demonstrating superior performance stability due to its unique physics-informed differential attention mechanism.

This performance gap is further nuanced by variable characteristics. While classical models yield acceptable results for variables with clear periodicity and smooth trends (e.g., air temperature), the imputation task becomes substantially more challenging for all models when dealing with variables that are inherently sparse and stochastic, such as precipitation. Notably, our model’s superiority is most pronounced on these difficult variables, which strongly validates its advanced capability to capture complex, non-linear data distributions. In virtually all test scenarios, our model’s RMSE curve is consistently lower than those of all competitors. We attribute this comprehensive and sustained advantage to its unique Spatio-Temporal information fusion mechanism. This mechanism enables the model to look beyond the temporal dynamics of a single variable, leveraging contextual information from physically related variables to achieve more accurate and robust estimations.

After analyzing the imputation accuracy for individual variables, we further evaluated the comprehensive performance of each model on a key downstream task: the calculation of daily reference evapotranspiration ($E{T}_0$). The calculation of $E{T}_0$ depends on a precise combination of multiple meteorological variables; therefore, this task can more comprehensively reflect the overall quality and practical value of the imputed data.

Table 1. Imputation performance of ST-CDF against classical and recent SOTA baseline models (including ImputeFormer and CoFILL) on the meteorological dataset. Results are reported as the mean ± standard deviation over 5 independent random seeds. Deterministic methods such as MICE yield identical outputs across repeated runs; therefore, the standard deviation is reported as 0. Note: ’-’ indicates the metric is not applicable to deterministic models. Bold values highlight the best performance for each specific metric across all evaluated models. Upward arrows (↑) denote that higher values indicate better performance, while downward arrows (↓) denote that lower values indicate better performance.

Missing Rate	Metric	MICE	GAIN	Transformer	SAITS	CSDI	PRISTI	ImputeFormer	CoFILL	ST-CDF
30%	$R^2\uparrow$	$0.796\pm 0.000$	$0.825\pm 0.014$	$0.832\pm 0.010$	$0.887\pm 0.009$	$0.899\pm 0.007$	$0.908\pm 0.006$	$0.915\pm 0.005$	$\mathbf{0}.\mathbf{928}\pm \mathbf{0}.\mathbf{006}$	$0.929\pm 0.005$
	RMSE ↓	$0.368\pm 0.000$	$0.339\pm 0.013$	$0.228\pm 0.009$	$0.260\pm 0.008$	$0.248\pm 0.007$	$0.231\pm 0.006$	$0.222\pm 0.005$	$0.209\pm 0.005$	$\mathbf{0}.\mathbf{205}\pm \mathbf{0}.\mathbf{004}$
	MAE ↓	$0.305\pm 0.000$	$0.270\pm 0.011$	$0.165\pm 0.007$	$0.199\pm 0.006$	$0.185\pm 0.005$	$0.176\pm 0.005$	$0.164\pm 0.004$	$0.151\pm 0.004$	$\mathbf{0}.\mathbf{142}\pm \mathbf{0}.\mathbf{003}$
	CRPS ↓	-	$0.152\pm 0.010$	-	-	$0.109\pm 0.005$	$0.100\pm 0.004$	-	$\mathbf{0}.\mathbf{083}\pm \mathbf{0}.\mathbf{003}$	$0.084\pm 0.003$
40%	$R^2\uparrow$	$0.729\pm 0.000$	$0.789\pm 0.017$	$0.825\pm 0.013$	$0.865\pm 0.011$	$0.871\pm 0.010$	$0.881\pm 0.008$	$0.896\pm 0.007$	$0.909\pm 0.006$	$\mathbf{0}.\mathbf{919}\pm \mathbf{0}.\mathbf{006}$
	RMSE ↓	$0.444\pm 0.000$	$0.389\pm 0.016$	$0.341\pm 0.012$	$0.290\pm 0.010$	$0.232\pm 0.009$	$0.260\pm 0.008$	$0.235\pm 0.007$	$0.219\pm 0.006$	$\mathbf{0}.\mathbf{208}\pm \mathbf{0}.\mathbf{005}$
	MAE ↓	$0.371\pm 0.000$	$0.312\pm 0.014$	$0.279\pm 0.010$	$0.226\pm 0.009$	$0.158\pm 0.008$	$0.201\pm 0.007$	$0.180\pm 0.006$	$0.164\pm 0.005$	$\mathbf{0}.\mathbf{149}\pm \mathbf{0}.\mathbf{004}$
	CRPS ↓	-	$0.186\pm 0.012$	-	-	$0.112\pm 0.006$	$0.115\pm 0.005$	-	$0.098\pm 0.004$	$\mathbf{0}.\mathbf{090}\pm \mathbf{0}.\mathbf{003}$
50%	$R^2\uparrow$	$0.640\pm 0.000$	$0.695\pm 0.023$	$0.819\pm 0.017$	$0.825\pm 0.015$	$0.810\pm 0.013$	$0.869\pm 0.011$	$0.875\pm 0.009$	$0.878\pm 0.008$	$\mathbf{0}.\mathbf{880}\pm \mathbf{0}.\mathbf{007}$
	RMSE ↓	$0.543\pm 0.000$	$0.489\pm 0.020$	$0.329\pm 0.015$	$0.341\pm 0.013$	$0.339\pm 0.012$	$0.261\pm 0.010$	$0.285\pm 0.009$	$0.256\pm 0.007$	$\mathbf{0}.\mathbf{250}\pm \mathbf{0}.\mathbf{006}$
	MAE ↓	$0.471\pm 0.000$	$0.419\pm 0.017$	$0.261\pm 0.012$	$0.270\pm 0.011$	$0.274\pm 0.010$	$0.190\pm 0.008$	$0.210\pm 0.007$	$0.181\pm 0.006$	$\mathbf{0}.\mathbf{173}\pm \mathbf{0}.\mathbf{005}$
	CRPS ↓	-	$0.241\pm 0.015$	-	-	$0.154\pm 0.008$	$0.133\pm 0.006$	-	$0.116\pm 0.005$	$\mathbf{0}.\mathbf{106}\pm \mathbf{0}.\mathbf{004}$
60%	$R^2\uparrow$	$0.533\pm 0.000$	$0.601\pm 0.029$	$0.730\pm 0.023$	$0.746\pm 0.020$	$0.816\pm 0.018$	$0.819\pm 0.015$	$0.825\pm 0.013$	$0.836\pm 0.011$	$\mathbf{0}.\mathbf{842}\pm \mathbf{0}.\mathbf{009}$
	RMSE ↓	$0.639\pm 0.000$	$0.580\pm 0.026$	$0.416\pm 0.019$	$0.399\pm 0.017$	$0.316\pm 0.015$	$0.309\pm 0.013$	$0.301\pm 0.011$	$0.290\pm 0.009$	$\mathbf{0}.\mathbf{282}\pm \mathbf{0}.\mathbf{008}$
	MAE ↓	$0.573\pm 0.000$	$0.511\pm 0.022$	$0.334\pm 0.016$	$0.320\pm 0.014$	$0.228\pm 0.012$	$0.218\pm 0.010$	$0.211\pm 0.009$	$0.203\pm 0.008$	$\mathbf{0}.\mathbf{196}\pm \mathbf{0}.\mathbf{006}$
	CRPS ↓	-	$0.298\pm 0.018$	-	-	$0.175\pm 0.010$	$0.154\pm 0.008$	-	$0.139\pm 0.006$	$\mathbf{0}.\mathbf{124}\pm \mathbf{0}.\mathbf{005}$
70%	$R^2\uparrow$	$0.421\pm 0.000$	$0.495\pm 0.037$	$0.660\pm 0.030$	$0.644\pm 0.027$	$0.671\pm 0.024$	$0.765\pm 0.020$	$0.772\pm 0.017$	$0.790\pm 0.014$	$\mathbf{0}.\mathbf{802}\pm \mathbf{0}.\mathbf{012}$
	RMSE ↓	$0.731\pm 0.000$	$0.674\pm 0.033$	$0.474\pm 0.025$	$0.492\pm 0.022$	$0.464\pm 0.020$	$0.356\pm 0.017$	$0.342\pm 0.014$	$0.329\pm 0.012$	$\mathbf{0}.\mathbf{316}\pm \mathbf{0}.\mathbf{010}$
	MAE ↓	$0.669\pm 0.000$	$0.611\pm 0.029$	$0.391\pm 0.021$	$0.411\pm 0.018$	$0.379\pm 0.016$	$0.259\pm 0.014$	$0.245\pm 0.012$	$0.236\pm 0.010$	$\mathbf{0}.\mathbf{225}\pm \mathbf{0}.\mathbf{008}$
	CRPS ↓	-	$0.352\pm 0.024$	-	-	$0.215\pm 0.013$	$0.185\pm 0.011$	-	$0.160\pm 0.009$	$\mathbf{0}.\mathbf{146}\pm \mathbf{0}.\mathbf{007}$
80%	$R^2\uparrow$	$0.313\pm 0.000$	$0.382\pm 0.048$	$0.551\pm 0.039$	$0.540\pm 0.035$	$0.691\pm 0.031$	$0.698\pm 0.026$	$0.715\pm 0.022$	$0.742\pm 0.019$	$\mathbf{0}.\mathbf{756}\pm \mathbf{0}.\mathbf{015}$
	RMSE ↓	$0.826\pm 0.000$	$0.777\pm 0.042$	$0.563\pm 0.032$	$0.578\pm 0.029$	$0.416\pm 0.026$	$0.418\pm 0.022$	$0.395\pm 0.018$	$0.368\pm 0.015$	$\mathbf{0}.\mathbf{349}\pm \mathbf{0}.\mathbf{012}$
	MAE ↓	$0.765\pm 0.000$	$0.708\pm 0.036$	$0.487\pm 0.027$	$0.492\pm 0.024$	$0.326\pm 0.021$	$0.314\pm 0.018$	$0.290\pm 0.015$	$0.272\pm 0.012$	$\mathbf{0}.\mathbf{255}\pm \mathbf{0}.\mathbf{010}$
	CRPS ↓	-	$0.415\pm 0.030$	-	-	$0.245\pm 0.017$	$0.221\pm 0.014$	-	$0.194\pm 0.012$	$\mathbf{0}.\mathbf{179}\pm \mathbf{0}.\mathbf{009}$

summarizes the Coefficient of Determination ($R^2$), Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE) between the $E{T}_0$ calculated from data completed by each model and the $E{T}_0$ calculated from the ground truth data, under three representative missingness rates: 30% to 80%.

It is important to clarify the dual role that the Penman–Monteith $E{T}_0$ equation plays within our framework. $E{T}_0$ is actively utilized as a physics-informed loss function (${\mathcal{L}}_{phy}$) exclusively during the training phase to enforce thermodynamic regularization on the generative process. However, in this evaluation section, $E{T}_0$ serves a distinct and independent purpose: it acts as a downstream validation metric. Here, we calculate $E{T}_0$ using the final, fully imputed meteorological data generated during the inference phase. Because the calculation of $E{T}_0$ depends on a precise, non-linear combination of multiple meteorological variables, this downstream task comprehensively reflects whether the imputed data has successfully retained its physical consistency and practical value for real-world agricultural applications.

3.3.1. Ablation Study

To systematically validate the distinct contributions of the innovative components within our proposed ST-CDF framework, we designed a rigorous set of ablation experiments. The core of our denoising network architecture lies in its parallel spatio-temporal feature extraction, which relies on two key modules. These are Graph Attention Network (GAT), used to explicitly capture instantaneous spatial correlations, and Multi-head Differential Attention Transformer, used to keenly perceive dynamic temporal changes. To quantify the individual impact of each component on imputation accuracy, we compared the following four model variants on the meteorological dataset. All variants were trained using identical configurations and evaluated on the same metrics.

ST-CDF (Full Model): The complete proposed architecture (GAT + Differential Attention Transformer), serving as the performance benchmark. Variant 1 (Diff-Attn only): This variant has the GAT module removed. The model cannot explicitly model the spatial topology between sensors and must rely solely on the Differential Attention Transformer to learn temporal dependencies from each channel independently. Variant 2 (GAT + Standard-Attn): This variant retains the GAT module but replaces the “Multi-head Differential Attention Mechanism” in the Transformer with a standard, value-based multi-head self-attention mechanism. Variant 3 (ST-CDF w/o Physics-Loss): The full ST-CDF architecture, but trained only with the standard $L_{noise}$ (i.e., $\lambda =0$), serving to isolate the impact of the physics-informed loss.

As shown in Figure 4, the results of the ablation study clearly confirm the importance of our two core design choices: Value of Spatial Modeling (GAT): When comparing Variant 1 (Diff-Attn only) to the ST-CDF (Full Model), the removal of the GAT module results in a significant performance degradation (higher RMSE/MAE) across all missing rates. This strongly validates that for multivariate time series data with strong physical coupling, such as meteorological data, explicitly modeling spatial dependencies is crucial for accurate imputation. Value of Differential Attention: Similarly, the comparison between Variant 2 (GAT + Standard-Attn) and the ST-CDF (Full Model) reveals that replacing differential attention with standard self-attention also incurs a distinct performance loss. This gap underscores our central hypothesis: in dynamically changing time series, capturing the relative changes or differences between elements provides a more robust and effective signal for temporal modeling than capturing static similarity alone, especially in high-sparsity scenarios.

Value of Physics-Informed Loss: A new, critical insight is gained by comparing the ST-CDF (Full Model) against Variant 3. While both use the identical, powerful spatio-temporal architecture, the Full Model trained with $L_{hybrid}$ consistently outperforms Variant 3 (ST-CDF w/o Physics-Loss) across all metrics and missing rates. This performance gap demonstrates that injecting physical constraints acts as a powerful regularizer, guiding the model to learn a data manifold that is not only statistically likely but also physically valid. This directly contributes to the model’s superior performance in the downstream $E{T}_0$ calculation task, as it was explicitly trained for this consistency.

In summary, our component-wise ablation analysis (Figure 4) explicitly quantifies the contributions: removing the GAT module (Variant 1) causes the most significant performance drop, with RMSE increasing by approximately 18% at high missing rates, confirming the dominance of spatial correlations. Replacing Differential Attention with standard attention (Variant 2) degrades RMSE by 8-10%, validating its role in capturing dynamic changes. Finally, omitting the Physics Loss (Variant 3) leads to a 5% degradation in general metrics but a substantial 12% error increase in the downstream $E{T}_0$ calculation, verifying its necessity for physical consistency.

3.3.2. Spatial Feature Extraction Capability

To validate the model’s practical capabilities, we selected a soil moisture sensor dataset containing three geographically adjacent monitoring sites (Field A, Field B, and Field C). Due to their proximity, the data from these monitoring points exhibit highly synergistic patterns of change in response to external environmental factors (such as synchronous rainfall).

To simulate a severe real-world sensor failure, we artificially selected a period from Field A’s sensor data that included this significant rainfall event and removed a continuous 120 h segment of all its observations. This created a massive information chasm, while the complete data from Field B and Field C served as crucial spatial corroborating evidence.

We compared the imputation results of our ST-CDF framework against two purely time series models:

• ST-CDF (Spatio-Temporal): Our proposed method, which utilizes its GAT module to process spatial correlations from Fields A, B, and C, and its differential attention Transformer for temporal context.
• Sparse Transformer (Temporal Optimization): An advanced pure time series model, serving as a strong temporal baseline.
• Standard Transformer (Temporal Only): A classic pure time series model for basic comparison.

Figure 5 visually demonstrates the imputation effectiveness. The core challenge was to capture the sharp rise and subsequent slow decline in soil moisture caused by the rainfall. As shown, the temporal-only models completely failed this task. The Standard Transformer (blue dash-dotted line) produced an overly smooth curve, failing to capture the peak. The Sparse Transformer (purple dotted line) performed slightly better but still showed significant lag and insufficient magnitude, as it could not utilize the corroborating evidence from Fields B and C.

In contrast, our proposed ST-CDF (Teacher) model (red dashed line) demonstrated robust performance. Its curve shows a high-degree of overlap with the ground truth data, accurately restoring the magnitude and timing of the rainfall peak.

To understand how ST-CDF achieves this superior performance, we analyzed its internal attention mechanisms during this critical event. This Explainable AI (XAI) analysis, visualized in Figure 6, provides two clear insights:

• Spatial (GAT) Analysis: When imputing the moisture peak at Field A, the GAT module assigned dominant attention weights to the concurrent data from the adjacent Fields B and C. This confirms the model learned to use spatial context from neighboring sensors to reconstruct an event that was missing from the target sensor’s own history.
• Temporal (Differential Attention) Analysis: Simultaneously, the Differential Attention Transformer focused its highest weights on the time steps immediately preceding the peak. This demonstrates its sensitivity to abrupt changes (the onset of rainfall), which standard attention mechanisms might smooth over.

Figure 6. Explainable AI (XAI) visualization of ST-CDF during a peak rainfall event: (a) GAT spatial attention weights, and (b) Differential Attention heatmap.

Figure 6. Explainable AI (XAI) visualization of ST-CDF during a peak rainfall event: (a) GAT spatial attention weights, and (b) Differential Attention heatmap.

This XAI analysis demonstrates that ST-CDF’s accuracy is derived from a learned, explainable understanding of how external events propagate through space and time, rather than purely data-driven pattern matching.

Meanwhile, the lightweight ST-CDF (Student) model also performed remarkably well. While slightly less precise than the teacher, it still successfully captured the event’s critical dynamics, vastly outperforming both pure time series baselines.

This experiment, supported by the XAI analysis, strongly demonstrates that when dealing with continuous data loss driven by external events with strong spatial correlations, the fusion of Spatio-Temporal information is the key. Purely time series models, regardless of their architectural sophistication, perform poorly due to the limitations of their information dimension. By virtue of its unique and explainable Spatio-Temporal feature extraction and fusion architecture, our method provides the most accurate and reliable recovery in extreme data loss scenarios, significantly surpassing all compared purely temporal models.

3.3.3. Counterfactual Simulation of Precision Irrigation

Building on the XAI-verified spatio-temporal learning, we conducted a counterfactual simulation to test ST-CDF’s utility as a decision-support tool. We simulated a 3 h irrigation event at Field A and tasked ST-CDF and the Standard Transformer baseline to predict the impact on the adjacent, un-irrigated Field B.

As shown in Figure 7, the Standard Transformer (blue line) fails to capture the intervention at Field A due to the absence of a spatio-temporal fusion mechanism. Consequently, its prediction for Field B aligns entirely with the no-intervention baseline, limiting its utility for this counterfactual simulation.

In contrast, ST-CDF successfully simulates the physical consequence of the irrigation. It predicts a physically plausible response in Field B: a delayed rise in soil moisture, which is attenuated (has a lower peak) compared to the direct intervention at Field A. This demonstrates that ST-CDF has learned the implicit physical rules of water propagation between sensor locations from the training data.

This simulation capability extends the utility of our framework beyond data imputation, offering a potential tool for scenario-based prediction in precision agriculture.

3.3.4. Sensitivity Analysis and Hyperparameter Evaluation

To systematically validate our architectural choices and address the trade-offs between physical consistency, imputation accuracy, and computational efficiency, we conducted comprehensive sensitivity analyses on two critical parameters: the physics-informed loss weight ($\lambda$) and the number of diffusion steps (T).

The hyperparameter $\lambda$ controls the injection strength of the thermodynamic physical constraint. To assess its impact, we evaluated ST-CDF on the meteorological dataset (at a 50% missing rate) across a range of values: $\lambda \in \{0.0,0.01,0.05,0.1,0.2,0.5\}$. We measured both standard imputation accuracy (via RMSE) and physical consistency (via the $R^2$ of the downstream $E{T}_0$ calculation).

As expected, when $\lambda$ is strictly $0.0$ or extremely small, the model acts predominantly as a purely data-driven generative model; while the intrinsic data imputation RMSE remains acceptable, the physical consistency metric ($E{T}_0$$R^2$) drops significantly. Conversely, when $\lambda$ is excessively large (e.g., $0.5$), the objective function becomes overly dominated by the physical penalty, forcing the model to deviate from the natural observational data distribution, which degrades the general imputation RMSE. Our empirical results confirm that setting $\lambda =0.1$ provides the optimal regularization balance, yielding both minimal imputation error and maximal physical alignment.

To preclude local optima and rigorously investigate the interaction effects between the model’s critical generative hyperparameters, we performed a comprehensive grid search over the Transformer layer count (N) and the number of diffusion steps (T). A $5\times 5$ parameter grid was defined, with $N\in \{2,4,6,8,10\}$ and $T\in \{25,50,100,200,500\}$. All 25 hyperparameter combinations were fully trained and evaluated on the validation split of the meteorological dataset (at a 50% missing rate). The resulting validation RMSE for each combination is presented in the heatmap in Figure 8.

The heatmap in Figure 8, where darker shades represent lower RMSE and superior performance, yields several key insights:

• Main Effects: A clear performance gradient is visible along both axes. Holding N constant (i.e., moving horizontally along any row), RMSE consistently decreases as T increases. This confirms the necessity of sufficient diffusion steps for high-fidelity generation. Similarly, holding T constant (moving vertically along any column), performance improves as N increases, demonstrating the value of model depth for capturing complex spatio-temporal dependencies.
• Interaction Effect and Performance Plateau: The heatmap clearly delineates a “performance plateau”—a region of dark, low-RMSE values—in the bottom-right quadrant. This region, where $N\ge 6$ and $T\ge 200$, represents a set of configurations that achieve robust, high-quality imputation. Critically, within this plateau, the marginal performance gain from adding more layers (e.g., from $N=8$ to $N=10$) or steps becomes minimal.

This grid search provides strong empirical validation for our chosen configuration. The selection of $N=6$ and $T=500$ is justified as it resides near the “elbow” of this performance plateau. This configuration strikes an optimal balance between achieving near-maximal imputation accuracy and maintaining computational efficiency, thereby avoiding the unnecessary overhead of excessively deep or slow models.

3.3.5. Model Distillation Capability

Although our proposed ST-CDF framework demonstrates high imputation accuracy, its iterative inference process based on the diffusion model incurs considerable computational overhead. This poses a challenge for practical deployment in resource-constrained scenarios such as the Agricultural Internet of Things (AIoT). To address this bottleneck, we conducted an efficiency evaluation to assess our model’s computational requirements relative to established baseline methods.

The core of this evaluation is a comparison between the teacher model (ST-CDF-Teacher), the student model (ST-CDF-Student), and two representative baseline methods: a standard Transformer and the classic MICE algorithm. We primarily conducted the comparison along two dimensions: model complexity (total number of trainable parameters) and inference speed (average per-sample imputation time). To ensure the fairness and reproducibility of the results, all tests were performed on a unified NVIDIA RTX 4090 GPU platform (with MICE running on a CPU). The standard test sample was defined as a 96 h meteorological time series with 40% random missingness.

The experimental results, as presented in

Table 2. Comparison of model complexity and empirical inference latency across high-performance GPU (RTX 4090) and edge CPU (Raspberry Pi 4B) environments. Note: ’N/A’ stands for ’Not Applicable’.

Model	Parameters (M)	FLOPs (G)	RTX 4090 Latency (ms)	Edge Device Latency (ms)
ST-CDF-Teacher	23.5	150.5	595	OOM/N/A
Transformer	12.5	30.2	95	841
ST-CDF-Student	3.8	10.1	52	182
MICE	N/A	N/A	N/A	670 (CPU only)

and Figure 9, clearly reveal the significant advantages brought by our proposed CGD framework, particularly for real-world IoT deployment.

First, the ST-CDF-Student achieves a substantial reduction in complexity compared to its teacher model. The parameter count drops from 23.5 M to 3.8 M, achieving a 6.2-fold model compression. More importantly, the theoretical computational load (FLOPs) required for a full imputation drops from 150.5 GFLOPs to 10.1 GFLOPs, a reduction of over 15-fold. On the high-performance RTX 4090 GPU, this theoretical gain is directly reflected in an empirical speedup of over 11 times (from 595 ms to 52 ms).

However, the most critical evaluation of an edge-computing methodology lies in its empirical performance on hardware with strictly limited resources. To address this and objectively validate our performance claims under practical conditions, we deployed the models on a representative agricultural IoT edge device profile (a Raspberry Pi 4B equipped with an ARM Cortex-A72 CPU).

As detailed in the “Edge Device Latency” column of Table 2, the heavy ST-CDF-Teacher model failed to execute efficiently (Out of Memory/prohibitive latency) due to the iterative nature of the 500-step diffusion reverse process, confirming that deploying raw diffusion models to rural IoT nodes is physically infeasible. In stark contrast, our single-step distilled ST-CDF-Student model successfully completed the multivariate data imputation task in just 182 milliseconds on the edge CPU. This is over 4.5 times faster than the standard Transformer baseline (841 ms) and significantly outpaces the classic MICE algorithm (670 ms) on the exact same hardware.

Given that precision agriculture sensor networks typically capture and transmit data at hourly or half-hourly intervals, an edge inference latency of 182 ms consumes a negligible fraction of the device’s operational duty cycle and power budget. In summary, this empirical edge-hardware test firmly substantiates the objectivity of our performance claims. It proves that our proposed CGD framework successfully transforms a large and accurate teacher model into a lightweight student model fully capable of real-time, robust deployment on resource-constrained agricultural endpoints.

3.3.6. Federated Learning Efficiency Analysis

To validate the practical utility of ST-CDF in resource-constrained IoT networks, we conducted a simulated federated learning experiment with $N=10$ edge clients under non-IID data settings (Dirichlet $\alpha =0.5$). We compared our proposed Fed-CGD strategy against standard FedAvg approaches using either the full Teacher model or the standalone Student model.

As shown in

Table 3. Comparison of communication cost, convergence rate, and imputation accuracy across three federated learning benchmarks. The Test RMSE is reported as the mean ± standard deviation over 5 independent random seeds.

Method	Params (M)	Comm. Cost/Rnd (MB)	Convergence (Rounds)	Total Comm. (GB)	Test RMSE
FedAvg (Teacher)	23.5	∼90.0	50	4.50	$0.248\pm 0.004$
FedAvg (Student)	3.8	∼14.5	120	1.74	$0.285\pm 0.008$
Fed-CGD	3.8	∼14.5	60	0.87	$0.252\pm 0.005$

, our proposed method effectively balances the trade-off between imputation accuracy and communication efficiency. While FedAvg (Teacher) achieves the lowest RMSE (0.248), it requires a substantial total communication overhead of 4.50 GB. Conversely, although FedAvg (Student) reduces per-round traffic, its limited network capacity leads to decreased convergence stability and lower accuracy (RMSE 0.285).

Figure 10 illustrates the convergence curves of Test RMSE across communication rounds. As observed, our Fed-CGD method (Green line) achieves a convergence rate significantly faster than the standalone Student model and comparable to the parameter-heavy Teacher model. This indicates that the proposed distillation strategy effectively stabilizes the training process in non-IID environments, allowing the model to reach high fidelity with fewer communication rounds.

Our Fed-CGD strategy, by leveraging local distillation, empowers the lightweight Student model to learn complex spatio-temporal representations similar to the Teacher. It achieves a highly competitive final RMSE of $0.252\pm 0.005$ while drastically reducing the total communication cost by 80.6% (from 4.50 GB to 0.87 GB). Combined with the empirically verified 182 ms edge inference latency on the Raspberry Pi established in the previous section (Table 2), this experiment conclusively proves that ST-CDF is not merely a theoretical imputation algorithm, but a highly viable, computationally feasible, and bandwidth-efficient system solution tailored for real-world agricultural IoT deployments.

4. Discussion

4.1. Advantages over Previously Known Models

The experimental results demonstrate that ST-CDF addresses key limitations of existing imputation paradigms. Traditional statistical methods (e.g., MICE) and standard deep learning models (e.g., GAIN, SAITS) often experience performance degradation under extreme sparsity because they rely primarily on local interpolation or unconstrained data-driven mapping. In contrast, ST-CDF leverages a conditional diffusion backbone enhanced by our Multi-head Differential Attention and Graph Attention Network (GAT). This architecture enables the model to look beyond the temporal dynamics of a single variable, effectively capturing abrupt environmental transitions and complex cross-sensor physical couplings. Furthermore, unlike generic diffusion baselines (e.g., CSDI, PriSTI), our framework integrates a physics-informed loss (${\mathcal{L}}_{phy}$) during training. This integration ensures that the generated imputations are not just statistically plausible, but better align with thermodynamic laws (as evidenced by the downstream $E{T}_0$ accuracy), thereby providing an effective approach for physically consistent data recovery.

4.2. Alignment with Agriculture 5.0 and Human-Centered AI

While Agriculture 4.0 primarily focused on automation and data-driven IoT systems, the paradigm is rapidly evolving toward Agriculture 5.0. This new framework emphasizes Human-Centered AI (HCAI), where artificial intelligence does not replace human expertise but rather augments it through explainability, trust, and human-in-the-loop decision support. Our ST-CDF framework inherently aligns with the core tenets of Agriculture 5.0 in the following dimensions:

• Transparency and Trust via Explainable AI (XAI): A major barrier to AI adoption in agriculture is the “black-box” nature of deep learning. As demonstrated in our XAI analysis, ST-CDF explicitly visualizes its internal decision-making processes. By providing spatial attention weights and differential attention heatmaps, the model offers complete transparency, enabling agronomists to cross-validate the model’s behavior against their domain knowledge.
• Human-in-the-Loop Decision Support and Practical Usability: By utilizing the Fed-CGD distillation strategy, the lightweight Student model can be directly deployed on resource-constrained edge devices. This ensures that farmers receive real-time, uninterrupted data on their local dashboards, even during network outages. The farmer remains firmly “in the loop,” utilizing this reliable data to make final, context-aware decisions regarding precision irrigation.
• Sustainability and Eco-efficiency: By recovering missing data with strict physical consistency, the ST-CDF framework prevents over-irrigation caused by sensor failures, directly supporting sustainable water resource management.

4.3. Limitations and Future Work

Despite these promising results, some limitations warrant further research. First, the model’s excellent performance in spatial imputation partially relies on a relatively dense sensor network to provide effective corroborating information, which could increase deployment costs in wide-area or resource-constrained environments. Future work will explore fusion techniques with multi-source remote sensing data, using low-cost areal data to enhance the imputation accuracy for sparse point-based sensors. Second, the framework’s generalization ability to other types of Spatio-Temporal data has not been fully validated, as this study focused primarily on agricultural and meteorological environmental sensors. Future research should adapt the framework using transfer learning techniques to broaden its applicability. Finally, although the Cluster-Guided Distillation (CGD) has significantly improved inference efficiency, the model’s computational complexity may still pose a deployment challenge for micro-edge devices with extremely limited power. Future research will prioritize exploring more extreme model lightweighting methods, such as model pruning and quantization, to ensure practical usability on various IoT endpoints.

5. Conclusions

In this work, we introduced ST-CDF, an innovative Spatio-Temporal Conditional Diffusion Framework designed for the high-fidelity and physically consistent imputation of multivariate agricultural sensor networks. By synergistically integrating advanced generative AI with thermodynamic regularizations, and deploying it via a Federated Cluster-Guided Distillation (Fed-CGD) strategy, this study successfully bridged the gap between complex deep learning and resource-constrained edge deployment.

Extensive evaluations confirm that ST-CDF achieves state-of-the-art performance. Quantitatively, ST-CDF consistently outperformed all baseline models across varying missing rates. For instance, under a challenging 80% data missing rate, ST-CDF maintained a robust Root Mean Squared Error (RMSE) of $0.349\pm 0.012$ and an $R^2$ of $0.756\pm 0.015$, exhibiting significantly lower error margins and higher stability than the Transformer baseline ($R^2=0.551$). Furthermore, the downstream validation explicitly proved the value of our physics-informed design: the reference evapotranspiration ($E{T}_0$) calculated from data imputed by ST-CDF achieved the lowest absolute error, confirming that the framework successfully reconstructs the integrity of underlying physical relationships rather than merely filling numerical gaps. Finally, the Fed-CGD strategy effectively compressed the parameter space, significantly reducing the communication overhead to enable privacy-preserving, edge-based deployment.

By providing a highly accurate, trustworthy, and physically reliable data foundation, ST-CDF strongly aligns with the human-centered goals of Agriculture 5.0, offering a powerful tool to bolster data-driven decision-making in precision agriculture and sustainable environmental management.