PIV-FlowDiffuser: Transfer-Learning-Based Denoising Diffusion Models for Particle Image Velocimetry

Abstract

Deep learning algorithms have significantly reduced the computational time and improved the spatial resolution of particle image velocimetry (PIV). However, the models trained on synthetic datasets might have degraded performances on practical particle images due to domain gaps. As a result, special residual patterns are often observed for the vector fields of deep learning-based estimators. To reduce the special noise step by step, we employ a denoising diffusion model (FlowDiffuser) for PIV analysis. And a data-hungry iterative denoising diffusion model is trained via a transfer learning strategy, resulting in our PIV-FlowDiffuser method. Specifically, we carry out the following: (1) pre-training a FlowDiffuser model with multiple optical flow datasets of the computer vision community, such as Sintel and KITTI; (2) fine-tuning the pre-trained model on synthetic PIV datasets. Note that the PIV images are upsampled by a factor of two to resolve small-scale turbulent flow structures. The visualized results indicate that our PIV-FlowDiffuser effectively suppresses the noise patterns. Therefore, the denoising diffusion model reduces the average endpoint error ($AEE$) by $59.4\%$ over the RAFT256-PIV baseline on the classic Cai’s dataset. In addition, PIV-FlowDiffuser exhibits enhanced generalization performance on unseen particle images due to transfer learning. Overall, this study highlights transfer-learning-based denoising diffusion models for PIV.

1. Introduction

Particle image velocimetry (PIV) provides instantaneous, quantitative, and whole-field velocity data and has become a fundamental non-intrusive optical measurement technique in fluid mechanics [1,2,3,4,5]. PIV works by introducing tracer particles into a flowing medium, illuminating them with a laser light sheet, and capturing two or more consecutive images of the particles with a camera. The instantaneous velocity vector field is estimated by analyzing the sequential particle recordings. However, practical PIV images are often susceptible to degradation in the course of measurements due to factors such as nonuniform light illumination, light reflections, background noise sources, and camera dark noise [6,7]. Consequently, these effects impose stringent demands on both analytical accuracy and generalization [8,9].

Over the past 40 years, cross-correlation and optical flow algorithms are the two primary algorithms employed for PIV processing [10,11,12,13,14,15,16]. For example, the window deformation iterative method (WIDIM) with central difference interrogation is verified to significantly improve the accuracy of estimation [17,18]. An optimized surrogate image, replacing one raw image, can generate a more accurate and robust correlation signal when addressing image background interference [7]. Despite their widespread use, cross-correlation methods extract velocity vectors from sparse interrogation windows, often resulting in low-resolution velocity fields. Optical flow methods estimate particle displacement based on the brightness preservation principle [11], which assumes that the particle image’s brightness remains unchanged after movement. However, when the optical flow principle is violated—often due to image noise in practical measurements—the risk of failure significantly increases [19,20].

To improve velocity estimation, deep learning techniques have gained popularity in PIV applications in recent years. The basic idea is that neural networks are trained to predict vectors from two input images [21,22,23,24,25,26,27,28,29,30,31,32,33]. PIV-DCNN is the first regression deep convolutional neural network for PIV estimation. It employs a four-level structure and offers compromising results compared to conventional cross-correlation methods [21]. Since then, a series of deep learning models (PIV-NetS [34], PIV-LiteFlowNet [22], LightPIVNet [35], RAFT-PIV [25], PIV-PWCNet [36], etc.) have been proposed and evaluated with synthetic particle images. Ji et al. proposed four physically constrained neural networks based on FlowNetS, which exhibited superior performance in capturing large-scale vortices [37]. Note that Cai et al. released a synthetic PIV dataset, which has been widely used by state-of-art methods for training deep learning models [34]. Similarly to WIDIM, neural networks equipped with iterative updates often achieve better results. The iterative implementation can take various forms depending on the researchers, such as cascaded refinement [21], coarse-to-fine pyramids [36], and convolutional gated recurrent units (Conv-GRU) [25]. Note that these models typically employ only a few iterations, with recurrent all-pairs field transforms (RAFTs)—the most iterative among them—usually set to 16 iterations. Nevertheless, these deep learning-based PIV algorithms estimate dense velocity fields from particle images in an end-to-end manner, which improves computational efficiency, accuracy, and spatial resolution compared to conventional cross-correlation or optical flow methods.

Despite advancements in neural networks, their performance in practical measurements is not always satisfactory, as illustrated in Figure 1. Specifically, noticeable error patterns can be observed in the residual of estimations. The underlying reasons might be twofold. First, the architecture of existing PIV neural networks lacks an explicit correction mechanism, meaning that errors accumulated during iterative updates cannot be effectively mitigated. That being said, the flow correction $\Delta \mathit{v}$ should be explicit modeled as a function with respect to current estimation $\mathit{v}$, i.e., $\Delta \mathit{v}={\mathit{f}}_{update}(\mathit{v},·)$. We argue that this explicit correction mechanism has the potential to rectify error patterns in the estimated flow field, thereby improving overall accuracy. Second, the neural models mentioned above are trained on synthetic datasets, which may differ significantly from real-world test cases, a discrepancy commonly referred to as the domain gap [38]. Currently available open-access synthetic datasets often lack sufficient diversity, limiting deep models’ ability to generalize to practical applications [22,36]. Thus, in this work, we focus on the challenges of designing an explicit error correction network architecture and developing effective training strategies.

Denoising diffusion models are known for their ability to generate high-quality images through a sequence of iterative denoising operations, i.e., $p_{\theta }\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t)$ [39,40]. Our insight is that the denoising diffusion model meets the explicit correction requirement for more accurate PIV measurement. Meanwhile, a denoising diffusion model (FlowDiffuser [41]) has established a new benchmark in the task of optical flow estimation, which further motivates us to use denoising diffusion models for PIV analysis. Unfortunately, the FlowDiffuser trained from scratch does not yield satisfactory results for PIV applications. One potential cause is that the synthetic dataset is insufficient for this data-hungry iterative model. However, transfer learning could effectively address data scarcity and improve generalization abilities [42,43,44]. As a transfer learning method, fine-tuning pre-trained models has already become popular for complex models [45]. The substantial requirement for labeled data in training accurate and robust PIV models has led us to investigate transfer learning (pre-trained models) as a potential solution.

As a result, we introduce PIV-FlowDiffuser, a denoising diffusion model trained using transfer learning, for particle image velocimetry. Our PIV-FlowDiffuser is designed to reduce measurement errors through iterative denoising, with the expectation of improving the accuracy of velocity estimation. Additionally, we transfer the initial weights from a pre-trained optical flow model and fine-tune it on PIV datasets. This fine-tuning approach, leveraging pre-trained parameters, enhances generalization and promotes efficient training. The main contributions are as follows:

• The denoising diffusion model (PIV-FlowDiffuser) is applied to PIV for the first time. PIV-FlowDiffuser employs an explicit correction mechanism that refines the estimated flow field iteratively, thereby reducing overall measurement errors.
• Transfer learning technique is employed to train deep models for PIV prediction, i.e., fine-tuning a pre-trained FlowDiffuser model from the computer vision domain. This approach leverages the robust feature extraction and flow reconstruction capabilities obtained through pre-training, potentially reducing the need for extensive labeled PIV data. Consequently, this enables accurate PIV estimation with reduced training time and enhances the model’s generalization performance.
• The feasibility of our PIV-FlowDiffuser model was validated through extensive synthetic and practical PIV images. Compared to the existing RAFT256-PIV model, our PIV-FlowDiffuser demonstrates decreased measurement errors, as evidenced by lower residuals in visualizations. Specifically, it achieves a $59.4\%$ reduction in the average endpoint error (AEE). Furthermore, the model exhibits superior accuracy on previously unseen data, indicating excellent generalization performance.

The rest of this article is organized as follows. Section 2 details the PIV-FlowDiffuser framework and transfer-learning-based training. Section 3 outlines the experimental settings of our evaluation. Section 4 presents the experimental results, including a thorough comparison with other baseline methods on both synthetic and practical PIV recordings. Section 5 gives a short conclusion of this work.

2. Transfer-Learning-Based Denoising Diffusion Models for PIV

In this section, we revisit the FlowDiffuser model [41] and present a straightforward adaptation method for PIV analysis. Additionally, we detail the transfer learning approaches employed to train a PIV model, including the selection of loss functions and training data.

2.1. The FlowDiffuser Model for PIV

Given the image pair $({\mathit{I}}_1,{\mathit{I}}_2)$, the velocity estimation procedure can be formulated as $\mathit{v}=f({\mathit{I}}_1,{\mathit{I}}_2)$, where $f(·)$ denotes the mapping from image pairs to velocity vectors [21]. Iterative updating is a common trick for effectively improving the performance of conventional PIV processing, i.e., ${\mathit{v}}^{new}={\mathit{v}}^{old}+\Delta \mathit{v}$, where $\Delta \mathit{v}$ is the estimated corrector [15,18,25]. Our insight is that, this deterministic approach can be extended to estimate the conditional probability distribution $p\left(\mathit{v}\right|{\mathit{I}}_1,{\mathit{I}}_2)$, providing a probabilistic generative perspective of the velocity field given the observed images. Similarly, the iterative update also has the corresponding probabilistic formulation $p\left({\mathit{v}}^{new}\right|{\mathit{v}}^{old},{\mathit{I}}_1,{\mathit{I}}_2)$. While different formulations remain fundamentally unchanged, adopting a probabilistic perspective enables the integration of conditional generative models into PIV measurements.

As a powerful generative method, denoising diffusion models [40] are capable of synthesizing highly realistic images ${\mathit{x}}_0$ via iterative backward diffusion (denoising), i.e., $p_{\theta }\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t)$. The key idea is that a neural network—parameterized by $\theta$—takes in two arguments ${\mathit{x}}_t,t$ and outputs a vector ${\mu }_{\theta }({\mathit{x}}_t,t)$ and a matrix ${\Sigma }_{\theta }({\mathit{x}}_t,t)$. This says the following:

$$p_{\theta }\left({\mathit{x}}_{t-1}\right|{\mathit{x}}_t)=\mathcal{N}\left({\mathit{x}}_{t-1}\right|{\mu }_{\theta }({\mathit{x}}_t,t),{\Sigma }_{\theta }({\mathit{x}}_t,t))$$

(1)

where the initial $p_{\theta }\left({\mathit{x}}_T\right)=\mathcal{N}(\mathbf{0},\mathit{I})$ is a diagonal Gaussian distribution. Note that the time step t is crucial for encoding the denoising process and is explicitly incorporated into the model. Inspired by the proven effectiveness of denoising diffusion probabilistic models, FlowDiffuser [41] considers the conditional influence of the input images, i.e., $p_{\theta }\left({\mathit{v}}_{t-1}\right|{\mathit{v}}_t,{\mathit{I}}_1,{\mathit{I}}_2)$, to redefine optical flow estimation as a generative process. In addition, the FlowDiffuser model demonstrates strong alignment with the probabilistic formulation of PIV estimation.

To this end, we have established an overview of the FlowDiffuser concept. We now explain our interpretation and implementation of this approach, focusing on three key components: a dual encoder for enhanced image representation, embedding enhancement techniques for encoding iterative step information, and a conditional recurrent denoising decoder (Conditional-RDD) for iterative updates, as illustrated in Figure 2. (1) Dual encoders pre-process the input image pair (${\mathit{I}}_1$, ${\mathit{I}}_2$), generating a context feature ${\mathit{x}}_c$ and a $4D$ correlation volume ${\mathit{x}}_{cv}$, as given by the RAFT [46]. The 4D correlation volumes can capture correlations of particle motion. In this case, this means that the condition on particle images can be replaced with more representative features, i.e., $p_{\theta }\left({\mathit{v}}_{t-1}\right|{\mathit{v}}_t,{\mathit{x}}_c,{\mathit{x}}_{cv})$. (2) The embedding enhancement module [41] explicitly synchronizes the time step t to the flow decoder. This being said, ${\mathit{x}}_o=EE(t,{\mathit{v}}_t,{\mathit{x}}_c,{\mathit{x}}_{cv})$ enhances the temporal embedding features and is fed into the GRU and flow head modules. (3) The conditional recurrent denoising decoder (conditional-RDD) takes the initial noisy flow field ${\mathit{v}}_t$ as input and optimizes it to ${\mathit{v}}_{t-1}$ through a denoising step (recurrent Conv-GRU neural network). In addition to the flow field ${\mathit{v}}_t$ being explicitly conditioned, the hidden feature ${\mathit{x}}_h$ is also considered as an alternative representation of the flow field. Hence, we define the implemented denoising decoder as $p_{\theta }\left({\mathit{v}}_{t-1}\right|{\mathit{v}}_t,{\mathit{x}}_c,{\mathit{x}}_{cv},{\mathit{x}}_h)$, where $p_{\theta }$ is the symbolic representation of the model and $\theta$ is the learnable weight. During inference, noise is progressively removed from the initial noisy estimation ${\mathit{v}}_T$, which follows a standard Gaussian distribution. These three components collectively contribute to the robustness and accuracy of the FlowDiffuser model in optical flow tasks [41].

To further enhance the model’s ability to measure small-scale turbulence, we employ a straightforward trick: upsampling the input images by a factor of 2 using bilinear interpolation (Figure 2). By increasing the image resolution through upsampling, the neural network can better capture fine details and intricate patterns associated with small-scale turbulence. Note that a corresponding downsampling process is performed at the output stage to maintain the same shape for the output flow field. This adaptation helps improve the measurement accuracy of multiscale flow fields. Moreover, using fractional scaling factors leads to pixel loss, while higher scaling factors cause a sharp increase in memory. This issue not only prevents our laboratory from proceeding with the experiment but also significantly prolongs the model’s inference time.

2.2. Transfer-Learning-Based Training

In this section, we present the training method for the FlowDiffuser model by employing a transfer learning method (fine-tuning) [47,48]. It leverages the robust features and latent flow representation learned during pre-training on large-scale optical flow datasets, which provides a good initialization for our PIV analysis. By fine-tuning the pre-trained model, we adapt it to estimate both global and local flow characteristics with minimal domain-specific data. This strategy significantly reduces training time and computational resources.

To adapt the FlowDiffuser model for PIV applications, structural adaptations are used to bridge the source domain (color sequence) and our target PIV data, as shown in Figure 2. First, the input PIV images, originally in grayscale, are transformed into a pseudo-color format by replicating the single channel across three channels. This conversion allows the model to leverage pre-trained features from color image datasets. Additionally, the output flow fields are downsampled to match the desired resolution, along with using the upsampling trick for images. With these adaptations, the PIV application of transfer learning to the FlowDiffuser model becomes feasible.To facilitate the reproduction of our research results, key experimental materials including training functions, evaluation functions, and hyperparameters have been made fully open source. These resources can be accessed via the link provided in the Abstract.

During the fine-tuning process, all model parameters are updated (none are frozen) since the input images and output velocity fields belong to different domains in the transfer learning framework. We also utilize advanced loss functions to guide the training process, which are given as

$$\mathcal{L}=\underset{{\mathit{v}}_0\sim q\left({\mathit{v}}_0\right|c),t\sim [1,T]}{\mathbb{E}}{\parallel {\mathit{v}}_{gt}-{\mathit{v}}_0\parallel }_1$$

(2)

where ${\parallel ·\parallel }_1$ indicates the $l_1$ distance between flow ground truth ${\mathit{v}}_{gt}$ and conditional denoising result ${\mathit{v}}_0$, and c is the abbreviation of the aforementioned probabilistic conditions. A one-cycle learning rate scheduler [49] is used to refine the network, facilitating precise adaptation to the PIV domain. Early stopping was adopted during the training process to control overfitting. The experiments showed that the optimal model typically emerged at the initial stage of training when the learning rate was of the order of ${10}^{-6}$. An excessively high learning rate not only caused the optimal model to be missed but also led to severe overfitting in subsequent training.

2.3. Datasets

The transfer learning framework involves a source domain dataset for initial model training and a target domain dataset for fine-tuning the pre-trained model. Additionally, test datasets are also required for performance evaluation. Here, we briefly introduce these three datasets used in this work. Regarding the source domain dataset, FlowDiffuser models [41] are pre-trained on a combination of Sintel, KITTI-2015 [50], and HD1K. Specifically, the officially released FlowDiffuser model (https://github.com/LA30/FlowDiffuser (accessed on 22 July 2025)) is employed as the pre-trained model for PIV analysis.

Regarding the target domain, two synthetic datasets are considered: Problem Class 1 (https://github.com/shengzesnail/PIV_dataset (accessed on 5 January 2025)) and Problem Class 2 (https://codeocean.com/capsule/7226151/tree/v1 (accessed on 20 January 2025)) [25]. Problem Class 1, introduced by Cai et al. [34], consists of synthetic particle image pairs with corresponding ground-truth fluid motions, designed to resemble ideal experimental conditions with high particle density and intensity. The Problem Class 1 dataset contains 14,150 particle image pairs at a resolution of $256\times 256$. It includes six fluid cases (uniform channel flow, direct numerical simulation of isotropic turbulence, flows around a backward-facing step, two-dimensional flows past a cylinder, DNS of turbulent channel flow, and simulations of a sea surface flow driven by an SQG model). This dataset contains over 10 different operating conditions, with an average of about 1000 image samples per condition. The Problem Class 1 dataset is divided into training, validation, and test sets in an 8:1:1 ratio. Problem Class 2 is based on the same ground-truth flow fields as Problem Class 1 but has reduced particle densities, intensities, and overall signal-to-noise ratios to better model real experimental data conditions. The Problem Class 2 dataset consists of 19,000 training image pairs and 1000 validation image pairs. Among these 1000 cases, we classify 224 as Backstep, 300 as JHTDB, 92 as DNS, 128 as Cylinder, 99 as SQG, 57 as Uniform, and the remaining 100 unrecognized cases as Other.

To further evaluate performance in complex flow fields, we test the PIV models on a turbulent wavy channel flow (TWCF) case [25,51]. This test case consists of experimental PIV measurements in an Eiffel-type wind tunnel and features a high spatial resolution of $2160\times 2560$ pixels. Moreover, due to the waviness of the sidewall (sinusoidal shape), both locally adverse and pressure gradient flow conditions occur in the channel, making this case well-suited for evaluating model performance on real PIV images.

3. Experimental Arrangement

In this section, we outline the experimental setup to evaluate performance through a series of experiments on both synthetic and real PIV cases. Specifically, three scenarios are considered: (1) accuracy assessment on synthetic PIV datasets where training and testing data share the same distribution, (2) generalization assessment on synthetic PIV datasets where training and testing data belong to different domains, and (3) practical performance on experimental PIV recordings with unknown distributions. In addition, the training and inference costs of multiple deep learning models are also tested. Before presenting the results of these experiments, the performance metrics of PIV models are introduced, along with the baseline methods used for comparison and other experimental details.

3.1. Evaluation Metrics

To visually assess performance, we provide the error residual map, calculated as $\parallel {\mathit{v}}_0-{\mathit{v}}_{gt}{\parallel }_2$, for intuitive evaluation. In addition, given the availability of ground-truth data, we also employ several classic quantitative metrics to evaluate performance: average endpoint error (AEE) [25], root mean square error (RMSE) [2,21], and average angular error (AAE) [52,53]:

$$\begin{array}{cc}\hfill & AEE={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\parallel {\mathit{v}}_0^{\left(i\right)}-{\mathit{v}}_{gt}^{\left(i\right)}\parallel }_2\hfill \\ \hfill & RMSE=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\parallel {\mathit{v}}_{pred}^{\left(i\right)}-{\mathit{v}}_{gt}^{\left(i\right)}\parallel }_2^2}\hfill \\ \hfill & AAE={\displaystyle \frac{1}{N}}\sum _{i=1}^N\arccos \left({\displaystyle \frac{{\mathit{v}}_0^{\left(i\right)}·{\mathit{v}}_{gt}^{\left(i\right)}}{\parallel {\mathit{v}}_0^{\left(i\right)}{\parallel }_2·{\parallel {\mathit{v}}_{gt}^{\left(i\right)}\parallel }_2}}\right)\hfill \end{array}$$

(3)

where ${\mathit{v}}_{gt}^{\left(i\right)}$ and ${\mathit{v}}_0^{\left(i\right)}$ denote the $ith$ ground truth and the estimated velocity vector, respectively. N is the total number of pixels, and ${\parallel ·\parallel }_2$ refers to the Euclidean distance.

3.2. Baseline Methods

The baseline methods include several classical PIV analysis approaches based on cross-correlation and deep neural networks, such as WIDIM [18], PIV-DCNN [21], PIV-LiteFlowNet [54], RAFT256-PIV [25], RAFT32-PIV [25], FlowFormer-PIV [55], and Twins-PIVNet [32]. Specifically, Twins-PIVNet is included as a recent, high-performance baseline, representing the pivotal trend of applying vision transformer architectures to PIV analysis.

Since two datasets (Problem Class 1 and Problem Class 2, detailed in Section 2.3) are used for training different FlowDiffuser models, we incorporate the datasets’ names into our model’s abbreviation. As a result, we define two primary PIV-FlowDiffuser models: PIV-FlowDiffuser-class1 and PIV-FlowDiffuser-class2. This suffix-based naming rule is also applied to other algorithms (RAFT256-PIV and FlowFormer-PIV), resulting in RAFT256-PIV-class1, RAFT256-PIV-class2, FlowFormer-PIV-class1, and FlowFormer-PIV-class2.

Additionally, to conduct ablation studies, we introduce two variants: (1) To evaluate the contribution of our adaptation module, we remove the upsampling/downsampling adaptation module, resulting in a model denoted as PIV-FlowDiffuser-class1(*). (2) To assess the effect of transfer learning, we also employ a pre-trained model (FlowDiffuser) without any scale adaptation nor fine-tuning.

4. Results and Discussions

4.1. Accuracy Performance on Synthetic Images

Figure 3 presents the vector fields and corresponding error distributions of two cases from the test set of Problem Class 1. In Figure 3a, we compare our proposed PIV-FlowDiffuser-class1 model with RAFT256-PIV-class1 on turbulent image data. The results show that the Endpoint Error (EPE) of PIV-FlowDiffuser-class1 is significantly lower than that of RAFT256-PIV-class1, particularly in regions characterized by small-scale turbulence. This qualitative outcome strongly demonstrates the superior performance of diffusion models in handling small-scale turbulent structures. The RAFT256-PIV-class1 method successfully captures the primary flow structures. However, it exhibits measurement errors, particularly in regions of small-scale turbulent flow. In Figure 3b, the baseline pre-trained model (FlowDiffuser) fails to predict the flow accurately, whereas fine-tuning (PIV-FlowDiffuser-class1(*)) significantly reduces the measurement error. Furthermore, by integrating an adaptation module (upsampling and downsampling), PIV-FlowDiffuser-class1 surpasses PIV-FlowDiffuser-class1(*). This means that the upsampling trick helps resolve the small-scale turbulence structure, as expected.

Table 1. The AEE results evaluated on the Problem Class 1 dataset. The best results are highlighted in bold, and the second is underlined. The data of WIDIM, PIV-DCNN, PIV-LiteFlowNet-en, RAFT256-PIV, and Twins-PIV are sourced from [25,32] (unit: pixels/frame).

Method	Backstep	JHTDB	DNS	Cylinder	SQG
WIDIM	0.034	0.084	0.304	0.083	0.457
PIV-DCNN	0.049	0.117	0.334	0.100	0.479
PIV-LiteFlowNet-en	0.033	0.075	0.122	0.049	0.126
RAFT256-PIV-class1	0.016	0.137	0.093	0.014	0.117
Twins-PIV-class1	0.013	0.092	0.056	0.012	0.091
PIV-FlowDiffuser-class1(*)	0.041	0.111	0.148	0.091	0.208
PIV-FlowDiffuser-class1	0.007	0.029	0.039	0.019	0.052

presents similar AEE results on the test set of Problem Class 1. Among the five test subsets, PIV-FlowDiffuser-class1 achieved the lowest measurement error in four subsets, consistent with the findings presented in Figure 3. Note that the PIV-FlowDiffuser-class1(*) algorithm also demonstrates acceptable performance compared to the traditional WIDIM baseline. The comparison between PIV-FlowDiffuser-class1 and PIV-FlowDiffuser-class1(*) reveals that the simple upsampling trick effectively enhances the accuracy of PIV measurements. Regarding the baseline methods, the recent Twins-PIV [32] method demonstrates the most competitive overall performance, achieving one top-ranking and three second-best results. In the entire test dataset, our PIV-FlowDiffuser-class1 exhibits a 59.4% error reduction (with an error of 0.0352) when compared to RAFT256-PIV-class1 (average error: 0.0866), which clearly demonstrates enhanced accuracy.

Table 2. The AEE results on the Problem Class 2 dataset. The best results are in bold, and the second-best results are underlined (unit: pixels/frame).

Method	Backstep	JHTDB	DNS	Cylinder	SQG	Uniform	Other
RAFT256-PIV-class2	0.131	0.476	0.646	0.124	0.593	0.174	0.380
FlowFormer-PIV-class2	0.377	0.680	1.316	0.297	1.042	0.850	1.406
PIV-FlowDiffuser-class2	0.155	0.296	0.565	0.138	0.466	0.328	0.587

presents the AEE results on the test set of Problem Class 2. Since Problem Class 2 corresponds to a large displacement dataset, all methods exhibit larger errors compared to the data in Table 1. This implies that measurements with large particle displacements are associated with increased measurement errors. The errors might originate from the linear motion approximation of actual curved particle trajectories with large displacements—a systematic error beyond the current research scope [15,56]. Unsurprisingly, our PIV-FlowDiffuser-class2 reduces the RAFT256-PIV-class2 error by half, achieving state-of-the-art performance.

4.2. Generalization to Out-of-Domain Dataset

In addition to in-domain testing (Section 4.1), out-of-domain generalization testing [57] is crucial for assessing robustness and reliability, and it might represent a promising avenue for developing widely deployable PIV models across diverse experimental conditions. By evaluating the deep neural models on datasets that differ significantly from the training data, researchers can identify potential weaknesses in handling diverse flow conditions, varying illumination, and different imaging noise levels.

Figure 4 illustrates the predicted vector fields (JHTDB flow) in scenarios where the test data exhibit significant distributional divergence from the training images. When evaluated on Problem Class 2 samples, the RAFT256-PIV-class1 model trained exclusively on Class 1 data demonstrates unsatisfactory performance. Trained from scratch, RAFT256-PIV-class1 fails to produce accurate predictions for out-of-domain datasets, clearly manifesting the domain gap challenge as evidenced by the prediction errors. By contrast, the transfer learning-enhanced FlowFormer-PIV-class1 model achieves improved accuracy when handling Problem Class 2 cases, demonstrating that fine-tuning a pre-trained model substantially enhances cross-domain generalization capabilities. More significantly, our proposed PIV-FlowDiffuser framework outperforms all counterparts in cross-domain scenarios, exhibiting exceptional robustness (as evidenced by quantitative metrics in

Table 3. The AEE, RMSE and AAE results evaluated on out-of-domain dataset. The best results are highlighted in bold. Note that the in-domain results are also provided (with the best results shown in parentheses) (unit: pixels/frame).

Method		Problem Class 1			Problem Class 2
	AEE	RMSE	AAE	AEE	RMSE	AAE
RAFT256-PIV-class1	0.0866	0.0741	0.1631	4.7564	5.4709	1.6131
FlowFormer-PIV-class1	0.3608	0.2956	0.3412	1.2518	1.4561	0.4236
PIV-FlowDiffuser-class1	(0.0352)	(0.0273)	(0.1283)	0.5537	0.7279	0.2683
RAFT256-PIV-class2	0.2107	0.1726	0.2644	0.3540	0.4502	0.2380
FlowFormer-PIV-class2	0.4205	0.3483	0.3890	0.7662	0.9180	0.3365
PIV-FlowDiffuser-class2	0.1021	0.0848	0.1667	(0.3124)	(0.3921)	(0.1795)

) and statistically reliable predictions across all test conditions.

As expected, Table 3 statistically reveals significant performance disparities between in-domain and out-of-domain configurations. Note that our PIV-FlowDiffuser framework maintains acceptable performance despite being exclusively trained on single-domain datasets. These experimental statistics confirm the enhanced generalization capacity obtained by transfer learning (fine-tuning).

4.3. Performance on Practical PIV Images

To further validate our method’s practical utility, we performed an evaluation on a real-world dataset from our custom-built platform (Figure 5a), which provides an absolute ground truth for precise error assessments. The predicted vector field (Figure 5b) exhibits high fidelity relative to the ground truth (Figure 5a), with minimal residual errors (Figure 5c), achieving an average Endpoint Error (EPE) of 0.158 pixels. This strong performance, demonstrating robust sub-pixel accuracy, confirms the model’s effectiveness and readiness for real-world PIV applications. However, it can be observed that there are some square-shaped artifacts in the central region of the image predicted by PIV-FlowDiffuser, which deviate slightly from the circular shape in the ground truth. This phenomenon is related to the fact that diffusion models utilize square matrices for tensor predictions.

Figure 6 presents different results for two cases from the practical turbulent wavy channel flow (TWCF) dataset. Given the absence of ground velocity truth, the PascalPIV method implements a multigrid cross-correlation algorithm to produce satisfactory velocity field estimates, thus making its outputs suitable as the reference method [25]. The flow fields exhibit multiscale fluctuations that span an order of magnitude in amplitude variation, providing benchmark cases for evaluating different estimation algorithms on practical PIV images. RAFT32-PIV-class1 fails in two TWCF cases (with maximum velocities of 12 pixels/frame), primarily attributable to the substantial domain shift between the training dataset and testing cases. This being said, the maximum displacement of the training set (Problem Class 1) is less than 10 pixels/frame. To mitigate this problem, domain expansion (data augmentation) is a straightforward strategy that addresses the observed inter-domain discrepancy. Trained on an expanded dataset (Problem Class 2), both RAFT256-PIV-class2 and RAFT32-PIV-class2 have acceptable performances despite the presence of some outliers.

As an alternative solution, transfer learning can use relevant optical flow knowledge learned from natural images to generalize PIV analysis. With an initialized pre-trained model, PIV-FlowDiffuser-class1(*) also produces reasonable measurements, demonstrating the generalization ability of transfer learning. Meanwhile, its results exhibit some non-negligible errors in the area with small, complex turbulent structures. By further incorporating the adaptation module (upsampling trick), PIV-FlowDiffuser-class1 reduces prediction noise significantly, resulting in comparable performances to PascalPIV and outperforming other PIV analysis baselines.

4.4. Experiments on Computational Cost

Table 4. The training time and average inference time for different methods. The data of RAFT256-PIV is sourced from [25], and Twins-PIV is sourced from [32]. The GPUs used in the table are all produced by NVIDIA Corporation, whose headquarters is located in Santa Clara, CA, USA.

Method	Device	Training-Time (h)	Average Inference Time (s)
WIDIM	-	-	0.86
PIV-DCNN	-	-	2.23
PIV-LiteFlowNet-en	-	-	0.13
RAFT256-PIV	Two NVIDIA Quadro RTX 6000 GPUs	∼18 (from scratch)	0.08
Twins-PIV-class1	Two NVIDIA Quadro RTX 8000 GPUs	∼21 (from scratch)	0.08
Twins-PIV-class2	Two NVIDIA Quadro RTX 8000 GPUs	∼32 (from scratch)	0.08
FlowFormer-PIV-class1	One NVIDIA GeForce RTX 3090 GPU	∼0.5 (fine-tuning)	0.05
FlowFormer-PIV-class2	One NVIDIA GeForce RTX 3090 GPU	∼1.5 (fine-tuning)	0.05
PIV-FlowDiffuser-class1	One NVIDIA GeForce RTX 3090 GPU	∼2.0 (fine-tuning)	0.27
PIV-FlowDiffuser-class2	One NVIDIA GeForce RTX 3090 GPU	∼5.0 (fine-tuning)	0.27

presents the training time and inference time for different neural networks of the PIV application. Compared to the recent Twins-PIV models trained from scratch, the training time (fine-tuning) of PIV-FlowDiffuser is reduced by 84.4∼90.5%. This verified that transfer learning can significantly save training time in building an outstanding PIV estimation neural network. Due to the increased complexity of the dataset (Problem Class2), PIV-FlowDiffuser-class2 exhibits a slower convergence rate and requires more training time (∼5 h). Nevertheless, this is still more efficient than training from scratch. In terms of inference time, PIV-FlowDiffuser is much slower than Twins-PIV and RAFT256 because it integrates an adaptation module, which upscales the image by a factor of two. Considering the accuracy improvement, we argue that the cost of PIV-FlowDiffuser is affordable given the current computing hardware.

5. Conclusions

In this work, we propose PIV-FlowDiffuser, a novel framework that introduces denoising diffusion models into PIV analysis. By iteratively performing conditional denoising, PIV-FlowDiffuser progressively refines the predicted flow fields, effectively suppressing noise and capturing small-scale turbulent structures. This leads to accurate velocity estimation and fewer outliers, outperforming conventional deep learning-based baselines. Secondly, we use transfer learning through a fine-tuning strategy, adapting a pre-trained optical flow model to the PIV domain. This significantly reduces the training time while enhancing the model’s generalization ability, even under domain shifts. The fine-tuned model benefits from generic motion features learned from large-scale datasets, enabling efficient adaptation with limited PIV-specific data. In addition, extensive experiments on both synthetic datasets and real-world PIV images confirm the outstanding accuracy and robustness of our PIV-FlowDiffuser. Notably, the method demonstrates strong generalization to out-of-domain scenarios, highlighting its practical potential for diverse PIV applications beyond the training distribution. Moving forward, we believe that the integration of denoising diffusion models into PIV will pave the way for new directions in accurate measurement, particularly under challenging turbulent and unsteady flow conditions.