Load-Balancing Strategies in Discrete Element Method Simulations
Abstract
:1. Introduction
2. Problem Definition
- single-step (also referred to as once in this article): In this load-balancing strategy, we only call load-balancing once per simulation at a given iteration;
- frequent: In frequent load-balancing, the software calls load-balancing at a constant frequency (every iterations);
- dynamic: In dynamic load-balancing, the software automatically detects if load-balancing is required by measuring the load imbalance. At a predefined frequency, the software checks the computational weights of all the processes. If the weight difference between the processes with the highest and lowest loads exceeds a threshold based on the average load (if , where is a defined threshold), load-balancing is performed. L and denote the total process load (summation of the weights of the cells, in Equations (1) and (2), owned by each process) and a user-defined dynamic load-balancing threshold, respectively.
- Packing of particles;
- A rotating drum;
- A silo;
- A V-blender.
3. Results and Discussion
3.1. Benchmark Cases
3.2. Influence of the Weighting Strategy (Linear and Quadratic)
3.3. Packing in Box
3.4. Rotating Drum
3.5. Silo
3.6. V Blender
4. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Nomenclature
Particle diameter | |
Time-step | |
e | Coefficient of restitution |
Load-balancing frequency | |
L | Total computational load of a process |
Number of cells | |
Number of particles | |
Number of processes | |
t | Time |
Simulation time | |
Load-balancing time | |
Simulation time | |
Cell weight | |
Y | Young’s modulus |
Greek letters | |
Particle weight | |
Cell weight | |
Dynamic load-balancing threshold | |
Coefficient of friction | |
Coefficient of rolling friction | |
Density of particle | |
Poisson’s ratio |
References
- Richard, P.; Nicodemi, M.; Delannay, R.; Ribiere, P.; Bideau, D. Slow relaxation and compaction of granular systems. Nat. Mater. 2005, 4, 121–128. [Google Scholar] [CrossRef] [PubMed]
- Ketterhagen, W.R.; am Ende, M.T.; Hancock, B.C. Process modeling in the pharmaceutical industry using the discrete element method. J. Pharm. Sci. 2009, 98, 442–470. [Google Scholar] [CrossRef]
- Boac, J.M.; Ambrose, R.K.; Casada, M.E.; Maghirang, R.G.; Maier, D.E. Applications of discrete element method in modeling of grain postharvest operations. Food Eng. Rev. 2014, 6, 128–149. [Google Scholar] [CrossRef] [Green Version]
- Blais, B.; Vidal, D.; Bertrand, F.; Patience, G.S.; Chaouki, J. Experimental methods in chemical engineering: Discrete element method—DEM. Can. J. Chem. Eng. 2019, 97, 1964–1973. [Google Scholar] [CrossRef]
- Golshan, S.; Sotudeh-Gharebagh, R.; Zarghami, R.; Mostoufi, N.; Blais, B.; Kuipers, J. Review and implementation of CFD-DEM applied to chemical process systems. Chem. Eng. Sci. 2020, 221, 115646. [Google Scholar] [CrossRef]
- Sawley, M.L.; Cleary, P.W. A parallel discrete element method for industrial granular flow simulations. EPFL Supercomput. Rev. 1999, 11, 23–29. [Google Scholar]
- Norouzi, H.R.; Zarghami, R.; Sotudeh-Gharebagh, R.; Mostoufi, N. Coupled CFD-DEM Modeling: Formulation, Implementation and Application to Multiphase Flows; John Wiley & Sons: Hoboken, NJ, USA, 2016. [Google Scholar]
- Golshan, S.; Munch, P.; Gassmoller, R.; Kronbichler, M.; Blais, B. Lethe-DEM: An open-source parallel discrete element solver with load balancing. arXiv 2021, arXiv:2106.09576. [Google Scholar]
- Norouzi, H.; Zarghami, R.; Mostoufi, N. New hybrid CPU-GPU solver for CFD-DEM simulation of fluidized beds. Powder Technol. 2017, 316, 233–244. [Google Scholar] [CrossRef]
- Eibl, S.; Rüde, U. A systematic comparison of runtime load balancing algorithms for massively parallel rigid particle dynamics. Comput. Phys. Commun. 2019, 244, 76–85. [Google Scholar] [CrossRef] [Green Version]
- Rettinger, C.; Rüde, U. Dynamic load balancing techniques for particulate flow simulations. Computation 2019, 7, 9. [Google Scholar] [CrossRef] [Green Version]
- Tsuzuki, S.; Aoki, T. Large-scale granular simulations using Dynamic load balance on a GPU supercomputer. In Proceedings of the Poster at the 26th IEEE/ACM International Conference on High Performance Computing, Networking, Storage and Analysis, New Orleans, LA, USA, 16–21 November 2014. [Google Scholar]
- Fleissner, F.; Eberhard, P. Load balanced parallel simulation of particle-fluid dem-sph systems with moving boundaries. Parallel Comput. Archit. Algorithms Appl. 2007, 48, 37–44. [Google Scholar]
- Kloss, C.; Goniva, C.; Hager, A.; Amberger, S.; Pirker, S. Models, algorithms and validation for opensource DEM and CFD–DEM. Prog. Comput. Fluid Dyn. Int. J. 2012, 12, 140–152. [Google Scholar] [CrossRef]
- Weinhart, T.; Orefice, L.; Post, M.; van Schrojenstein Lantman, M.P.; Denissen, I.F.; Tunuguntla, D.R.; Tsang, J.; Cheng, H.; Shaheen, M.Y.; Shi, H.; et al. Fast, flexible particle simulations—An introduction to MercuryDPM. Comput. Phys. Commun. 2020, 249, 107129. [Google Scholar] [CrossRef]
- Forgber, T.; Toson, P.; Madlmeir, S.; Kureck, H.; Khinast, J.G.; Jajcevic, D. Extended validation and verification of XPS/AVL-Fire™, a computational CFD-DEM software platform. Powder Technol. 2020, 361, 880–893. [Google Scholar] [CrossRef]
- Blais, B.; Barbeau, L.; Bibeau, V.; Gauvin, S.; El Geitani, T.; Golshan, S.; Kamble, R.; Mikahori, G.; Chaouki, J. Lethe: An open-source parallel high-order adaptative CFD solver for incompressible flows. SoftwareX 2020, 12, 100579. [Google Scholar] [CrossRef]
- Fleissner, F.; Eberhard, P. Parallel load-balanced simulation for short-range interaction particle methods with hierarchical particle grouping based on orthogonal recursive bisection. Int. J. Numer. Methods Eng. 2008, 74, 531–553. [Google Scholar] [CrossRef]
- Warren, M.S.; Salmon, J.K. A parallel hashed oct-tree n-body algorithm. In Proceedings of the 1993 ACM/IEEE Conference on Supercomputing, Portland, OR, USA, 19 November 1993; pp. 12–21. [Google Scholar]
- Berger, R.; Kloss, C.; Kohlmeyer, A.; Pirker, S. Hybrid parallelization of the LIGGGHTS open-source DEM code. Powder Technol. 2015, 278, 234–247. [Google Scholar] [CrossRef]
- Cintra, D.T.; Willmersdorf, R.B.; Lyra, P.R.M.; Lira, W.W.M. A hybrid parallel DEM approach with workload balancing based on HSFC. Eng. Comput. 2016, 33, 2264–2287. [Google Scholar] [CrossRef]
- Cintra, D.T.; Willmersdorf, R.B.; Lyra, P.R.M.; Lira, W.W.M. A parallel DEM approach with memory access optimization using HSFC. Eng. Comput. 2016, 33, 2463–2488. [Google Scholar] [CrossRef]
- Markauskas, D.; Kačeniauskas, A. The comparison of two domain repartitioning methods used for parallel discrete element computations of the hopper discharge. Adv. Eng. Softw. 2015, 84, 68–76. [Google Scholar] [CrossRef]
- Arndt, D.; Bangerth, W.; Blais, B.; Clevenger, T.C.; Fehling, M.; Grayver, A.V.; Heister, T.; Heltai, L.; Kronbichler, M.; Maier, M.; et al. The deal. II library, version 9.2. J. Numer. Math. 2020, 28, 131–146. [Google Scholar] [CrossRef]
- Arndt, D.; Bangerth, W.; Blais, B.; Fehling, M.; Gassmöller, R.; Heister, T.; Heltai, L.; Köcher, U.; Kronbichler, M.; Maier, M.; et al. The deal. II library, version 9.3. J. Numer. Math. 2021, 29, 171–186. [Google Scholar] [CrossRef]
- Gassmöller, R.; Lokavarapu, H.; Heien, E.; Puckett, E.G.; Bangerth, W. Flexible and scalable particle-in-cell methods with adaptive mesh refinement for geodynamic computations. Geochem. Geophys. Geosystems 2018, 19, 3596–3604. [Google Scholar] [CrossRef]
- Burstedde, C.; Wilcox, L.C.; Ghattas, O. p4est: Scalable algorithms for parallel adaptive mesh refinement on forests of octrees. SIAM J. Sci. Comput. 2011, 33, 1103–1133. [Google Scholar] [CrossRef] [Green Version]
- Bangerth, W.; Burstedde, C.; Heister, T.; Kronbichler, M. Algorithms and Data Structures for Massively Parallel Generic Adaptive Finite Element Codes. Acm Trans. Math. Softw. 2012, 38, 1–38. [Google Scholar] [CrossRef]
- Geuzaine, C.; Remacle, J.F. Gmsh: A 3-D finite element mesh generator with built-in pre-and post-processing facilities. Int. J. Numer. Methods Eng. 2009, 79, 1309–1331. [Google Scholar] [CrossRef]
- Owen, D.; Feng, Y.; Han, K.; Peric, D. Dynamic domain decomposition and load balancing in parallel simulation of finite/discrete elements. In Proceedings of the ECCOMAS 2000, Barcelona, Spain, 11–14 September 2000. [Google Scholar]
- Golshan, S.; Esgandari, B.; Zarghami, R.; Blais, B.; Saleh, K. Experimental and DEM studies of velocity profiles and residence time distribution of non-spherical particles in silos. Powder Technol. 2020, 373, 510–521. [Google Scholar] [CrossRef]
- Alizadeh, E.; Dubé, O.; Bertrand, F.; Chaouki, J. Characterization of mixing and size segregation in a rotating drum by a particle tracking method. AIChE J. 2013, 59, 1894–1905. [Google Scholar] [CrossRef]
Packing | Drum | Silo | V-Blender | |
---|---|---|---|---|
3 | ||||
1000 | 2500 | 600 | 2000 | |
100 | 100 | 5 | 10 | |
e | ||||
10 | 40 | 20 | ||
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Golshan, S.; Blais, B. Load-Balancing Strategies in Discrete Element Method Simulations. Processes 2022, 10, 79. https://doi.org/10.3390/pr10010079
Golshan S, Blais B. Load-Balancing Strategies in Discrete Element Method Simulations. Processes. 2022; 10(1):79. https://doi.org/10.3390/pr10010079
Chicago/Turabian StyleGolshan, Shahab, and Bruno Blais. 2022. "Load-Balancing Strategies in Discrete Element Method Simulations" Processes 10, no. 1: 79. https://doi.org/10.3390/pr10010079
APA StyleGolshan, S., & Blais, B. (2022). Load-Balancing Strategies in Discrete Element Method Simulations. Processes, 10(1), 79. https://doi.org/10.3390/pr10010079