Evolve Then Filter Regularization for Stochastic Reduced Order Modeling

Evolve Then Filter Regularization for Stochastic Reduced Order Modeling Xuping Xie 1,†*, Feng Bao 2,‡ and Clayton Webster 3,† 1 Oak Ridge National Lab; xiex@ornl.gov 2 Florida State University; bao@math.fsu.edu 3 Oak Ridge National Lab; webstercg@ornl.gov * Correspondence: xiex@ornl.gov † Address: One Beth Valley Road, Oak Ridge National Lab, Oak Ridge, TN, 37831 ‡ Address: 1017 Academic Way, Tallahassee, FL 32306


Introduction
Many important scientfic and engineering applications require repeated numerical simulations of large and complex dynamical systems with high computational cost [25,26,40,45].The traditional full order model simulation is limited due to the extremely demanding on computational resources.
Reduced order models (ROMs), therefore, have been successfully introduced to reduce the expensiveness of the numerical simulations.ROMs aim at finding a reduced space that approximate the original model (full order model) with orders of magnitude reduction in computational cost while maintaining high accuracy.The reduced space is constructed by truncating the reduced basis, often the proper orthogonal decomposition (POD) basis.The classical Galerkin projection based reduced order model (G-ROM) is one of the most popular model reduction method.It is obtained by projecting the full order model to the reduced space.G-ROM is sucessful across a range of disciplines, however, its' use in convection-dominated flows has been hampered by the numerical instability.This instability, usually in the form of unphysical numerical oscillations, yielding inaccurate results for nonlinear systems.To mitigate the spurious numerical oscillations, various stabilized reduced order models (ROMs) have been introduced see [2,5,13,24,31,32,42,44,54].One popular strategy is the ROM closure modeling which models the lost information in the truncation of the POD basis, many ROMs can be found in [3,4,13,46,48,50,54,57].Another approach is the regularization, which uses explicit spatial filtering to regularize the standard G-ROM and increase the numerical stability of the ROM approximation.Recent development of regularized ROMs method for deterministic systems have been introduced in [47,55].
Reduced order models (ROMs) for systems involving stochastic process have gained increasing attention recently [8,20,36].The development of ROMs for partial differential equations (PDEs) subject to random inputs acting on the boundary and PDEs with random coefficients have been considered intensely in various contexts [16,27,53].Some works have been done for ROMs for evolutionary PDEs driven by stochastic processes [11,14].[29] introduced the Leray-regularization reduced order model (L-ROM) for the stochastic system with Brownian motions.
In this paper, we address the instability issue of standard G-ROM for nonlinear stochastic PDEs by using regularization.Motiviated by [29], we introduce another regularized ROM, evolve-then-filter (EF-ROM), for SPDEs that are of relevance to fluid dynamics as used in [29].The main purpose is to numerically investigate the evolve-then-filter regularization ROM (EF-ROM) for the stabilization of the G-ROM within a simple setting, a stochastic Burgers equation (SBE) driven by linear multiplicative noise.In [29], it has been shown that the spurious oscillations in G-ROM persist as the noise is turned on, and the oscillations worsen as the noise amplitude increases.The numerical test of EF-ROM shows that it gives more accurate modeling of the SBE dynamics by reducing the oscillations of the G-ROM with a low dimensional approximation.
The rest of the paper is organized as follows.In Section 2, we briefly describe the SBE to be used in our numerical experiment.In Section 3, we provide details about the derivation of the corresponding G-ROM and EF-ROM based on proper orthogonal decomposition.In Section 5, we present our numerical investigation of the EF-ROM.Finally, we outline conclusions and potential future research directions in Sectioin 6

Stochastic Burgers equation (SBE)
The deterministic viscous Burgers equation and its stochastic version have been widely used in reduced order modeling, see [14,15,34,39,49].In this paper, we will focus on the following stochastic Burgers equation (SBE) driven by linear multiplicative noise: where u 0 is some appropriate initial datum to be specified, W t is a two-sided one-dimensional Wiener process, σ is a positive constant that measures the "amplitude" of the noise, and ν is a positive diffusion coefficient.Similar to [29], the multiplicative noise term σu • dW t is understood in the sense of Stratonovich [41].
SPDEs driven by linear multiplicative noise such as the SBE (1) arise in various contexts, including turbulence theory, non-equilibrium phase transitions, or simply the modeling of parameter disturbance [6,12,18,38].

Numerical Discretization of SBE
In our numerical experiment at Section 5, we collect the snapshots data from the direct numerical simulation of the SBE.The SBE (1) is solved by a semi-implicit Euler scheme as given in [14,Section 6.1].
The nonlinearity uu x = (u 2 ) x /2 and the noise term σu • dW t are discretized explicitly for each time step, while the other terms are treated implicitly.The Laplacian operator is discretized using the standard second-order central difference approximation.Thus, we can get the following semi-implicit discretization scheme: where u n i is the discrete approximation of u(i∆x, n∆t), ∆x and ∆t are the mesh size of the spatial discretization and the time step, respectively.The discretized Laplacian ∆ and the discretized spatial derivative ∇ in (2) are given by where the boundary conditions are u n 0 = u n N x −1 = 0, N x is the total number of grid points of the spatial discretization in [0, 1].The ζ n are random variables drawn independently from a normal distribution N (0, 1).The additional drift term σ 2 u n j /2 in (2) is the conversion of the Stratonovich noise term σu • dW t into Itô form.

Initial Condition
The initial condition is defined as following Where ξ is the step function defined by ξ(x) = 1 if x ∈ (0.05, 0.55) and ξ(x) = 0 otherwise.The φ is given by φ The parameter in φ is set to be = 0.01 in our numerical experiment.
The initial condition is slightly modified from the one used in [34].The modification is mainly intended to enforce the compatibility of the initial and boundary condition at the left boundary point (x = 0) and to avoid any potential regularity issues that may arise in the numerical discretization of the SBE in (2) due to the discontinuity in the step function.

Proper Orthogonal Decomposition
POD is one of the most popular data-driven reduced order modeling method, which we exclusively use to generate the ROM basis in this paper.We briefly describe the POD in this section.
We note, however, that other ROM bases (e.g., the dynamic mode decomposition (DMD)) could be used.For more details, the reader is referred to [9,10,40,51].The POD starts with the snapshots {u 0 , . . ., u N s }, which are numerical approximations of the SBE at N s different time instances.The POD seeks a low-dimensional space X r := span{ϕ 1 , . . ., ϕ r } that approximates the snapshots optimally with respect to L 2 -norm.
Consider an ensemble of snapshots R := span u 0 , . . ., u N s , which is a collection of velocity data from either numerical simulation results or experimental observations at time t i = i ∆t, i = 0, . . ., N s .The POD basis {ϕ} i come from the minimization problem: where (z j ) is the -th component of the eigenvector z j .Also the following error formula holds from [26,34]: Note that in many ROMs of fluid dynamics, snapshots matrix always assembled by subtracting the centering trajectory when generating the POD basis.That is, the fluctuations u = u − U, where U is the centering trajectory, are considered in the data matrix.For our numerical investigation, however, we do not use the centering trajetory approach for the simple one dimension SBE case.

Galerkin Projection ROM (G-ROM)
The classic Galerkin projection based reduced order model has been introduced for fluids for many years.The derivation of the POD Galerkin ROM (G-ROM) follows the standard Galerkin approximation procedure.We present the derivation of G-ROM for the SBE (1) below.For a fixed number of basis r ∼ O(10), the r-dimensional POD Galerkin approximation u r of the SBE solution u takes the following form: where the time-varying coefficients (ROM coefficients) {a j (t, ω)} r j=1 are determined by solving: Following the expansion of u r given in ( 6) and the orthogonality property of POD basis functions, we can get the more explicit form of the above equation: where j = 1, • • • , r.The above low dimensional dynamic system (8) is the called Galerkin ROM equation of the stochastic Burgers equation (SBE).The ROM online computation involves time integration of system (8), which carried out by using a standard Euler-Maruyanma scheme [33].
The fully discretized G-ROM of SBE is as follows: where ζ n are random variables drawn independently from a normal distribution N (0, 1).

Evolve-Then-Filter Regularized ROM
The G-ROM is efficient and relatively accurate for many fluid flows.As mentioned before, however, G-ROM is inccurate for convection-dominated flows because of the numerical instability.In this Section, we introduce and present details of the EF-ROM regularization for the SBE to investigate potential improvement for numerical instability.This EF-ROM regularization based on POD spatial filtering to smooth the flow variables and increase numerical stability of the model, see Sec.We present details of the ROM spatial filtering (Differential Filter) in this Section.The POD differential filter (DF) is defined as follows: Let δ be the radius of the DF.For a given u r ∈ X r , find u r ∈ X r such that Differential filters have been used in the simulation of convection-dominated flows with standard numerical methods [21,22].The DF (10) uses an explicit length scale δ (i.e., the radius of the filter) to eliminate the small scales (i.e., high frequencies) from the input.Indeed, the DF (10) uses an elliptic operator to smooth the input variable.The DF also has a low computational overhead as it solves a linear system with a very small r × r matrix that is precomputed.Another advantage is ROM DF preserve incompressibility in the NSE, since they are linear operators.In reduced order modeling, POD-DF was first used in [47] in a periodic, one-dimensional (1D) setting.In this paper, we apply POD-DF to the SBE system (1).

EF-ROM for SBE
We draw inspiration from the deterministic case and consider regularized ROMs (Reg-ROMs) constructed from the POD differential filter [55].These Reg-ROMs belong to the wide class of stabilized ROMs [2,3,5,13,17,24,32,42,46,54].The main difference between Reg-ROMs and the other stabilized ROMs is that Reg-ROMs increase the numerical stability of the model by using explicit spatial filtering, which is a relatively new concept in the ROM field [47,54].Other ROMs use closure modeling both physically and mathematically.In this paper, we will use the Evolve-Then-Filter ROM (EF-ROM) [47,55] based on a specific way of filtering the convective term in the SBE (1) as explained below.
The Evolve-Then-Filter model has been used as a numerical tool in the simulation of convection-dominated deterministic flows with standard numerical methods [23,35].It has also been used to derive Reg-ROMs for deterministic systems in [47,55].The construction of the EF-ROM to the stochastic problem (1) is straightforward, which contains two steps.There is only one crucial difference in its derivation compared to the derivation of the G-ROM as outlined in Section 3.2, which consists of applying POD-DF after evolving the dynamic system.
The r-dimensional EF-ROM approximation u r of the SBE solution u takes the form (6). The time-varying coefficients {a j (t, ω)} r j=1 are determined by solving: The first "evolve" step in the EF-ROM ( 11) is just one step of the time discretization of the standard G-ROM (9).The "filter" step in the EF-ROM consists of filtering of the intermediate solution obtained in the "evolve" step: This could give us the following linear system, Where M r = (ϕ i , ϕ j ) and S r = (∇ϕ i , ∇ϕ j ) are the POD mass matrix and stiffness matrix respectively, and b is the filtered POD coefficient.Thus the r-dimensional EF-ROM for SBE (1) is given by: where j = 1, • • • , r.As mentioned in Section 4, a forward Euler time discretization was used in (11), but other time discretizations are possible [19].
Unlike the Leray-ROM (L-ROM) [29,58], which only filtering the nonlinear term, the EF-ROM filter the whole dynamics of the coefficients after the "evolve" step.Some numerical analysis regard these two methods for standard turbulent flows have been studied in [35].A full comparison of the Reg-ROMs for deterministic case was studied in [55].We emphasize that a numerical comparison of the EF-ROM and L-ROM for stochastic Burgers system is beyound the scope of this paper.A further study with more discussions and complex stochastic systems will be investigated for future research.

Numerical Results
In this Section, we present our numerical results for the EF-ROM and compared it with the standard G-ROM.The data that we used to construct our ROM is generated by the method describled in Sec.This can be observed from the reconstructed ROM solution field in Fig. 2. For the SBE problem studied here, as we said before, the purpose of EF-ROM is to alleviate the spurious oscillations genearted in standard G-ROM.We can see from Fig. 2 that, indeed, the oscillations are signifcantly reduced in the spatio-temporal numerical reconstruction by EF-ROM resulting in a better approximation to the original SBE system.Also note that as the dimension of the ROM increases, the overall performance of both ROMs improves as can be seen in Fig. 2.This behavior is expected since increasing dimension r increases the amount of energy used to the dynamic system of ROM, which accurately approximate the SBE.The parameter δ that defined as the radius of the ROM spatial filtering of EF-ROM in eqn.( 13), has to be appropriately calibrated to reach a good performance.Large value means filtering too much of the spatial field which generate very bad results, while small value (identical to zero) means filtering nothing just like the G-ROM.The optimal value (δ) is choosed by minimizing the L 2 -error of the EF-ROM in numerical approximating the SBE's spatio-temporal field.The noise (σ), dimension r and random variable ζ n in the numerical algorithm (16) can change the performance of the different δ.To reduce the numerical efforts, the nearly optimal δ = 0.0011 is reached when σ = 0 for r = 4, 6, 8, and we fix this δ for all the numerical (statistical) experiments.
Another comparison of the two ROMs can be made by looking the time evolution of the projected coefficients onto each POD modes.The dynamics of POD coefficients can revel how the model perform from the magnitude and the trajectory of each coefficient.Fig. 3 shows the evolution of POD coefficients correspond to each POD basis.The two ROMs are perform quite well and similar for the leading coefficients a 2 and a 3 .For high frequency modes, however, G-ROM models badly about the dynamics in terms of magnitude whereas EF-ROM generates a closer trajectory to SBE, see coefficients a 4 − a 8 in Fig. 3.It is interesting to note that the EF-ROM leads a slight deterioration on the dynamic of first mode a 1 , see Fig. 3.This deterioration is exist even if the optimal δ is reached.The conjecutre is that the DF spatial filtering affect this little deterioration.As the first POD mode contains the most dominant energy, the filtering algorithm on the first mode would reduce its magnitude.The G-ROM, however, uses exactly the same amount of energy which would approximate the first coefficient (a 1 ) dynamic

Robustness of EF-ROM.
We also did numerical experiments regarding the statistical relevance of the ROM results.
Especially, we investigated the effect of the magintude of the noise on the results.The following relative L 2 error formula is used when assess the performance of the ROMs: For this experiment, we use 13 noise magnitude σ that equally spaced between 0 and 0.6, and perform 1000 simulations for each ROM.The related SBE solution data generated by the same size of simulations via eqn.(2), and POD basis also updated at each simulation.The differential filter radius δ is fixed to be 0.0011.Fig. 4 plot the ensemble averages of the relative errors where the error bars indicate the standard deviations.This result shows that the EF-ROM is significantly more accurate to noise variations than G-ROM.The ensemble averages of error are above 40% for GROM with r = 6 and r = 8, while the EF-ROM relative error is around 30% (r = 6) or below (r = 8).

Conclusions
The numerical instability of Galerkin projection based ROMs is a very important challenge and has been widely studied.We are investigating this challenge in the stochastic fluid flows background.
Motivated by few previous work [29], we introduced the evolve-then-filter (EF) regularized ROM for stochastic fluids by performing a computational study of SBE.The EF-ROM uses the explicit spatial filtering to regularize outputs from the ROM.The numerical results studied in this paper indicated that the EF-ROM indeed alleviate the spurious oscillations that existed in the standard G-ROM.It turned out that EF-ROM generates significant better approximation than G-ROM and less sensitive to noise magnitude variaitons.We emphasize that although we use the same filtering method as in regularized L-ROM [29], the model is fundamentally different.A comparison of EF-ROM and other regularized ROMs (Reg-ROMs) is beyond the scope of this paper.We plan to have a thorough study of Reg-ROMs for stochastic fluids in future research.
There are still many unclear questions need to be investigated.For example, does the EF-ROM works for other different types of noise?e.g, addivie noise, correlated noise etc.How this ROM perform for realistic 3D stochastic flows?Also how to propose new ROM method with the recently popular data-driven ROM idea which applied machine learning or neural network inference [43,48,57,59].It is meaningful to research the robustness of the dynamics of ROM system with parameters (e.g, δ, ν, r, σ).

Figure 1 .
Figure 1.The numerical solution of SBE with σ = 0.3 and the POD basis functions generated from the solution data

Figure 2 .
Figure 2. The space-time numerical reconstruction of SBE from G-ROM (9) (top row) and EF-ROM (16) (bot row) with dimension r = 4 (left panel), r = 6 (middle panel) and r = 8 (right panel), respectively.The noise path is the same as used in numerical solution of SBE field plotted in Fig.1

Figure 3 .
Figure 3.Time evolution of the projected POD coefficients from the solution of G-ROM, EF-ROM and SBE system.The ROM solutions are obtained with σ = 0.3 and r = 8

)Figure 4 .
Figure 4.The ensemble averages of the relative L 2 error of G-ROM (dark line) and EF-ROM (red line) computed via (18) for dimension r = 6 (top) and r = 8 (bottom).The noise amplitude σ equally spaced between 0 and 0.6.For each σ, 1000 simulations are carried out for SBE and ROMs.The error bars show the standard deviations.

Table 1 .
The energy captured by the first few POD basis from SBE data with σ = 0.3.
Even though the first few POD modes extract the most dominate percentage of energy, the corresponding G-ROM generates a very high numerical oscillations, which yield inaccurate results.Preprints (www.

Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 13 August 2018 doi:10.20944/preprints201808.0219.v1
Peer-reviewed version available at Fluids 2018, 3, 84; doi:10.3390/fluids3040084better.Since this is our initial study, we intend to further investigate this issue together with more complex stochastic systems and numerical analysis in our further research.