Closure Learning for Nonlinear Model Reduction Using Deep Residual Neural Network
Abstract
:1. Introduction
 A novel ROM closure learning framework centered around deep neural networks.
 A hybrid framework that synthesizes the strengths of physical modeling and datadriven modeling.
 Very good performance in numerical tests, in both the reconstructive and the predictive regime.
 Significant improvement in numerical accuracy compared with state of the art ROM closure models.
2. Reduced Order Model
3. Closure Learning
3.1. Residual Neural Network (ResNet)
3.2. ROM Closure Modeling
3.3. ROM Closure Learning
Algorithm 1 ResNetROM 

4. Numerical Experiments
4.1. Implementation
4.2. Reconstruction
4.3. Prediction
4.4. Comparison
4.5. Sensitivity
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
Abbreviations
ResNet  Residual Neural Network 
ROM  Reduced Order Modeling 
GPROM  Galerkin Projection Reduced Order Model 
POD  Proper Orthogonal Decomposition 
FOM  Full Order Model 
LES  Large Eddy Simulation 
VMS  Variational Multiscale 
NSE  NavierStokes Equations 
References
 Holmes, P.; Lumley, J.L.; Berkooz, G. Turbulence, Coherent Structures, Dynamical Systems and Symmetry; Cambridge University Press: Cambridge, UK, 1996. [Google Scholar]
 Noack, B.R.; Morzynski, M.; Tadmor, G. ReducedOrder Modelling for Flow Control; Springer: Berlin/Heidelberg, Germany, 2011; Volume 528. [Google Scholar]
 Hesthaven, J.S.; Rozza, G.; Stamm, B. Certified Reduced Basis Methods for Parametrized Partial Differential Equations; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
 Quarteroni, A.; Manzoni, A.; Negri, F. Reduced Basis Methods for Partial Differential Equations: An Introduction; Springer: Berlin/Heidelberg, Germany, 2015; Volume 92. [Google Scholar]
 Mohebujjaman, M.; Rebholz, L.G.; Xie, X.; Iliescu, T. Energy balance and mass conservation in reduced order models of fluid flows. J. Comput. Phys. 2017, 346, 262–277. [Google Scholar] [CrossRef]
 Akkari, N.; Casenave, F.; Moureau, V. Time stable reduced order modeling by an enhanced reduced order basis of the turbulent and incompressible 3D Navier–Stokes equations. Math. Comput. Appl. 2019, 24, 45. [Google Scholar] [CrossRef] [Green Version]
 Cagniart, N.; Maday, Y.; Stamm, B. Model order reduction for problems with large convection effects. In Contributions to Partial Differential Equations and Applications; Springer: Berlin, Germany, 2019; pp. 131–150. [Google Scholar]
 Iollo, A.; Lanteri, S.; Désidéri, J.A. Stability properties of POD–Galerkin approximations for the compressible Navier–Stokes equations. Theoret. Comput. Fluid Dyn. 2000, 13, 377–396. [Google Scholar] [CrossRef] [Green Version]
 Stabile, G.; Hijazi, S.; Mola, A.; Lorenzi, S.; Rozza, G. PODGalerkin reduced order methods for CFD using finite volume discretisation: Vortex shedding around a circular cylinder. Commun. Appl. Ind. Math. 2017, 8, 210–236. [Google Scholar] [CrossRef] [Green Version]
 Loiseau, J.C.; Brunton, S.L. Constrained sparse Galerkin regression. J. Fluid Mech. 2018, 838, 42–67. [Google Scholar] [CrossRef] [Green Version]
 Noack, B.R.; Stankiewicz, W.; Morzyński, M.; Schmid, P.J. Recursive dynamic mode decomposition of transient and posttransient wake flows. J. Fluid Mech. 2016, 809, 843–872. [Google Scholar] [CrossRef] [Green Version]
 Amsallem, D.; Farhat, C. Stabilization of projectionbased reducedorder models. Int. J. Num. Meth. Eng. 2012, 91, 358–377. [Google Scholar] [CrossRef]
 Carlberg, K.; Farhat, C.; Cortial, J.; Amsallem, D. The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows. J. Comput. Phys. 2013, 242, 623–647. [Google Scholar] [CrossRef] [Green Version]
 Baiges, J.; Codina, R.; Idelsohn, S. Reducedorder subscales for POD models. Comput. Methods Appl. Mech. Eng. 2015, 291, 173–196. [Google Scholar] [CrossRef] [Green Version]
 Feppon, F.; Lermusiaux, P.F.J. Dynamically orthogonal numerical schemes for efficient stochastic advection and Lagrangian transport. SIAM Rev. 2018, 60, 595–625. [Google Scholar] [CrossRef]
 Fick, L.; Maday, Y.; Patera, A.T.; Taddei, T. A stabilized POD model for turbulent flows over a range of Reynolds numbers: Optimal parameter sampling and constrained projection. J. Comput. Phys. 2018, 371, 214–243. [Google Scholar] [CrossRef]
 Hijazi, S.; Stabile, G.; Mola, A.; Rozza, G. Datadriven PODGalerkin reduced order model for turbulent flows. arXiv 2019, arXiv:1907.09909. [Google Scholar]
 Lu, F.; Lin, K.K.; Chorin, A.J. Databased stochastic model reduction for the Kuramoto–Sivashinsky equation. Phys. D 2017, 340, 46–57. [Google Scholar] [CrossRef] [Green Version]
 Majda, A.J.; Harlim, J. Physics constrained nonlinear regression models for time series. Nonlinearity 2012, 26, 201. [Google Scholar] [CrossRef]
 Rebollo, T.C.; Ávila, E.D.; Mármol, M.G.; Ballarin, F.; Rozza, G. On a certified Smagorinsky reduced basis turbulence model. SIAM J. Numer. Anal. 2017, 55, 3047–3067. [Google Scholar] [CrossRef] [Green Version]
 San, O.; Iliescu, T. A stabilized proper orthogonal decomposition reducedorder model for large scale quasigeostrophic ocean circulation. Adv. Comput. Math. 2015, 41, 1289–1319. [Google Scholar] [CrossRef] [Green Version]
 San, O.; Maulik, R. Neural network closures for nonlinear model order reduction. Adv. Comput. Math. 2018, 44, 1–34. [Google Scholar] [CrossRef] [Green Version]
 Stabile, G.; Ballarin, F.; Zuccarino, G.; Rozza, G. A reduced order variational multiscale approach for turbulent flows. Adv. Comput. Math. 2019, 45, 2349–2368. [Google Scholar] [CrossRef] [Green Version]
 Wang, Z.; Akhtar, I.; Borggaard, J.; Iliescu, T. Proper orthogonal decomposition closure models for turbulent flows: A numerical comparison. Comput. Methods Appl. Mech. Eng. 2012, 237–240, 10–26. [Google Scholar] [CrossRef] [Green Version]
 Bergmann, M.; Bruneau, C.H.; Iollo, A. Enablers for robust POD models. J. Comput. Phys. 2009, 228, 516–538. [Google Scholar] [CrossRef] [Green Version]
 Carlberg, K.; BouMosleh, C.; Farhat, C. Efficient nonlinear model reduction via a leastsquares Petrov–Galerkin projection and compressive tensor approximations. Int. J. Num. Meth. Eng. 2011, 86, 155–181. [Google Scholar] [CrossRef]
 Parish, E.J.; Wentland, C.; Duraisamy, K. The adjoint PetrovGalerkin method for nonlinear model reduction. arXiv 2018, arXiv:1810.03455. [Google Scholar] [CrossRef] [Green Version]
 Xie, X.; Wells, D.; Wang, Z.; Iliescu, T. Approximate deconvolution reduced order modeling. Comput. Methods Appl. Mech. Eng. 2017, 313, 512–534. [Google Scholar] [CrossRef] [Green Version]
 Xie, X.; Mohebujjaman, M.; Rebholz, L.; Iliescu, T. Datadriven filtered reduced order modeling of fluid flows. SIAM J. Sci. Comput. 2018, 40, B834–B857. [Google Scholar] [CrossRef] [Green Version]
 Pope, S. Turbulent Flows; Cambridge University Press: Cambridge, UK, 2000. [Google Scholar]
 Sagaut, P. Large Eddy Simulation for Incompressible Flows, 3rd ed.; Scientific Computation; Springer: Berlin, Germany, 2006. [Google Scholar]
 Couplet, M.; Sagaut, P.; Basdevant, C. Intermodal energy transfers in a proper orthogonal decomposition—Galerkin representation of a turbulent separated flow. J. Fluid Mech. 2003, 491, 275–284. [Google Scholar] [CrossRef]
 He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27 June 2016; pp. 770–778. [Google Scholar]
 Szegedy, C.; Ioffe, S.; Vanhoucke, V.; Alemi, A.A. Inceptionv4, inceptionresnet and the impact of residual connections on learning. In Proceedings of the ThirtyFirst AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017. [Google Scholar]
 Lu, Y.; Zhong, A.; Li, Q.; Dong, B. Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations. arXiv 2017, arXiv:1710.10121. [Google Scholar]
 Chang, B.; Meng, L.; Haber, E.; Tung, F.; Begert, D. Multilevel residual networks from dynamical systems view. arXiv 2017, arXiv:1710.10348. [Google Scholar]
 Chang, B.; Meng, L.; Haber, E.; Ruthotto, L.; Begert, D.; Holtham, E. Reversible architectures for arbitrarily deep residual neural networks. In Proceedings of the ThirtySecond AAAI Conference on Artificial Intelligence, New Orleans, LO, USA, 2 February 2018. [Google Scholar]
 Maulik, R.; Mohan, A.; Lusch, B.; Madireddy, S.; Balaprakash, P. Timeseries learning of latentspace dynamics for reducedorder model closure. arXiv 2019, arXiv:1906.07815. [Google Scholar] [CrossRef] [Green Version]
 Rahman, S.M.; Pawar, S.; San, O.; Rasheed, A.; Iliescu, T. A nonintrusive reduced order modeling framework for quasigeostrophic turbulence. arXiv 2019, arXiv:11906.11617. [Google Scholar]
 Ahmed, S.E.; San, O.; Rasheed, A.; Iliescu, T. A long shortterm memory embedding for hybrid uplifted reduced order models. arXiv 2019, arXiv:1912.06756. [Google Scholar]
 Maulik, R.; Lusch, B.; Balaprakash, P. Reducedorder modeling of advectiondominated systems with recurrent neural networks and convolutional autoencoders. arXiv 2020, arXiv:2002.00470. [Google Scholar]
 San, O.; Maulik, R. Machine learning closures for model order reduction of thermal fluids. Appl. Math. Model. 2018, 60, 681–710. [Google Scholar] [CrossRef]
 Koc, B.; Mohebujjaman, M.; Mou, C.; Iliescu, T. Commutation error in reduced order modeling of fluid flows. Adv. Comput. Math. 2019, 45, 2587–2621. [Google Scholar] [CrossRef] [Green Version]
 Chen, T.Q.; Rubanova, Y.; Bettencourt, J.; Duvenaud, D. Neural ordinary differential equations. arXiv 2018, arXiv:1806.07366. [Google Scholar]
 Baiges, J.; Codina, R.; Castanar, I.; Castillo, E. A finite element reduced order model based on adaptive mesh refinement and artificial neural networks. Int. J. Numer. Methods Eng. 2019, 121, 588–601. [Google Scholar] [CrossRef]
 Chekroun, M.D.; Liu, H.; McWilliams, J.C. Variational approach to closure of nonlinear dynamical systems: Autonomous case. J. Stat. Phys. 2019, 1–88. [Google Scholar] [CrossRef] [Green Version]
 Lin, K.K.; Lu, F. Datadriven model reduction, Wiener projections, and the MoriZwanzig formalism. arXiv 2019, arXiv:1908.07725. [Google Scholar]
 Mohebujjaman, M.; Rebholz, L.G.; Iliescu, T. Physicallyconstrained datadriven correction for reduced order modeling of fluid flows. Int. J. Num. Methods Fluids 2019, 89, 103–122. [Google Scholar] [CrossRef] [Green Version]
 John, V. Finite Element Methods for Incompressible Flow Problems; Springer: Berlin, Germany, 2016. [Google Scholar]
 Mou, C.; Koc, B.; San, O.; Iliescu, T. Datadriven variational multiscale reduced order models. arXiv 2020, arXiv:2002.06457. [Google Scholar]
 Borggaard, J.; Iliescu, T.; Wang, Z. Artificial viscosity proper orthogonal decomposition. Math. Comput. Model. 2011, 53, 269–279. [Google Scholar] [CrossRef]
 Kunisch, K.; Volkwein, S. Galerkin proper orthogonal decomposition methods for parabolic problems. Numer. Math. 2001, 90, 117–148. [Google Scholar] [CrossRef]
 Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
 Akhtar, I.; Wang, Z.; Borggaard, J.; Iliescu, T. A new closure strategy for proper orthogonal decomposition reducedorder models. J. Comput. Nonlinear Dyn. 2012, 7, 39–54. [Google Scholar] [CrossRef]
 Wells, D.; Wang, Z.; Xie, X.; Iliescu, T. An evolvethenfilter regularized reduced order model for convectiondominated flows. Int. J. Num. Methods Fluids 2017, 84, 598–615. [Google Scholar] [CrossRef]
 Caiazzo, A.; Iliescu, T.; John, V.; Schyschlowa, S. A numerical investigation of velocitypressure reduced order models for incompressible flows. J. Comput. Phys. 2014, 259, 598–616. [Google Scholar] [CrossRef]
 Giere, S.; Iliescu, T.; John, V.; Wells, D. SUPG reduced order models for convectiondominated convectiondiffusionreaction equations. Comput. Methods Appl. Mech. Eng. 2015, 289, 454–474. [Google Scholar] [CrossRef]
 Wang, Z. ReducedOrder Modeling of Complex Engineering and Geophysical Flows: Analysis and Computations. Ph.D. Thesis, Virginia Tech, Blacksburg, VA, USA, 2012. [Google Scholar]
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xie, X.; Webster, C.; Iliescu, T. Closure Learning for Nonlinear Model Reduction Using Deep Residual Neural Network. Fluids 2020, 5, 39. https://doi.org/10.3390/fluids5010039
Xie X, Webster C, Iliescu T. Closure Learning for Nonlinear Model Reduction Using Deep Residual Neural Network. Fluids. 2020; 5(1):39. https://doi.org/10.3390/fluids5010039
Chicago/Turabian StyleXie, Xuping, Clayton Webster, and Traian Iliescu. 2020. "Closure Learning for Nonlinear Model Reduction Using Deep Residual Neural Network" Fluids 5, no. 1: 39. https://doi.org/10.3390/fluids5010039