# A Long Short-Term Memory Neural Network for the Low-Cost Prediction of Soot Concentration in a Time-Dependent Flame

^{*}

## Abstract

**:**

## 1. Introduction

_{2}concentration, respectively. The relationship between the soot characteristics and the gas-related parameters (i.e., the function f in Equation (3)) is central to the soot estimator concept and was a major focus in previous developments [14,15,16,17].

## 2. Methodology

#### 2.1. CFD Simulations and Flame Conditions

#### 2.2. LSTM Theory

#### 2.3. LSTM Application

## 3. Results and Discussion

## 4. Conclusions and Recommendations

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Shindell, D.; Kuylenstierna, J.C.; Vignati, E.; van Dingenen, R.; Amann, M.; Klimont, Z.; Anenberg, S.C.; Muller, N.; Janssens-Maenhout, G.; Raes, F.; et al. Simultaneously mitigating near-term climate change and improving human health and food security. Science
**2012**, 335, 183–189. [Google Scholar] [CrossRef] [PubMed][Green Version] - Eaves, N.A.; Zhang, Q.; Liu, F.; Guo, H.; Dworkin, S.B.; Thomson, M.J. CoFlame: A refined and validated numerical algorithm for modeling sooting laminar coflow diffusion flames. Comput. Phys. Commun.
**2016**, 207, 464–477. [Google Scholar] [CrossRef] - Eaves, N.; Dworkin, S.; Thomson, M. The importance of reversibility in modeling soot nucleation and condensation processes. Proc. Combust. Inst.
**2015**, 35, 1787–1794. [Google Scholar] [CrossRef] - Veshkini, A.; Eaves, N.A.; Dworkin, S.B.; Thomson, M.J. Application of PAH-condensation reversibility in modeling soot growth in laminar premixed and nonpremixed flames. Combust. Flame
**2016**, 167, 335–352. [Google Scholar] [CrossRef] - Aubagnac-Karkar, D.; El Bakali, A.; Desgroux, P. Soot particles inception and PAH condensation modelling applied in a soot model utilizing a sectional method. Combust. Flame
**2018**, 189, 190–206. [Google Scholar] [CrossRef] - Kholghy, M.R.; Veshkini, A.; Thomson, M.J. The core–shell internal nanostructure of soot–A criterion to model soot maturity. Carbon
**2016**, 100, 508–536. [Google Scholar] [CrossRef] - D’Anna, A.; Sirignano, M.; Kent, J. A model of particle nucleation in premixed ethylene flames. Combust. Flame
**2010**, 157, 2106–2115. [Google Scholar] [CrossRef] - Kholghy, M.R.; Eaves, N.A.; Veshkini, A.; Thomson, M.J. The role of reactive PAH dimerization in reducing soot nucleation reversibility. Proc. Combust. Inst.
**2019**, 37, 1003–1011. [Google Scholar] [CrossRef] - Frenklach, M.; Mebel, A.M. On the mechanism of soot nucleation. Phys. Chem. Chem. Phys.
**2020**, 22, 5314–5331. [Google Scholar] [CrossRef] - Mueller, M.E.; Pitsch, H. LES model for sooting turbulent nonpremixed flames. Combust. Flame
**2012**, 159, 2166–2180. [Google Scholar] [CrossRef] - Mueller, M.E.; Pitsch, H. Large eddy simulation of soot evolution in an aircraft combustor. Phys. Fluids
**2013**, 25, 110812. [Google Scholar] [CrossRef][Green Version] - Tolpadi, A.K.; Danis, A.M.; Mongia, H.C.; Lindstedt, R.P. Soot Modeling in Gas Turbine Combustors. In Proceedings of the ASME Turbo Expo, Orlando, FL, USA, 2–5 June 1997. [Google Scholar]
- Brocklehurst, H.T.; Priddin, C.H.; Moss, J.B. Soot predictions within an aero gas turbine combustion chamber. In Proceedings of the ASME Turbo Expo, Orlando, FL, USA, 2–5 June 1997. [Google Scholar]
- Bozorgzadeh, S. Development of a Soot Concentration Estimator for Industrial Combustion Applications. Master’s Thesis, Ryerson University, Toronto, ON, Canada, 2014. [Google Scholar]
- Alexander, R.; Bozorgzadeh, S.; Khosousi, A.; Dworkin, S.B. Development and testing of a soot particle concentration estimator using Lagrangian post-processing. Eng. Appl. Comput. Fluid Mech.
**2018**, 12, 236–249. [Google Scholar] [CrossRef] - Zimmer, L.; Kostic, S.; Dworkin, S. A novel soot concentration field estimator applied to sooting ethylene/air laminar flames. Eng. Appl. Comput. Fluid Mech.
**2019**, 13, 470–481. [Google Scholar] [CrossRef][Green Version] - Jadidi, M.; Kostic, S.; Zimmer, L.; Dworkin, S.B. An artificial neural network for the low-cost prediction of soot emissions. Energies
**2020**, 13, 4787. [Google Scholar] [CrossRef] - Christo, F.; Masri, A.; Nebot, E.; Turányi, T. Utilising artificial neural network and repro-modelling in turbulent combustion. Proceedings of ICNN’95—International Conference on Neural Networks, Perth, Australia, 27 November–1 December 1995; Volume 2, pp. 911–916. [Google Scholar]
- Christo, F.; Masri, A.; Nebot, E.; Pope, S. An integrated PDF/neural network approach for simulating turbulent reacting systems. Symp. Int. Combust.
**1996**, 26, 43–48. [Google Scholar] [CrossRef] - Ranade, R.; Alqahtani, S.; Farooq, A.; Echekki, T. An ANN based hybrid chemistry framework for complex fuels. Fuel
**2019**, 241, 625–636. [Google Scholar] [CrossRef] - Pulga, L.; Bianchi, G.; Falfari, S.; Forte, C. A machine learning methodology for improving the accuracy of laminar flame simulations with reduced chemical kinetics mechanisms. Combust. Flame
**2020**, 216, 72–81. [Google Scholar] [CrossRef] - Ranade, R.; Echekki, T. A framework for data-based turbulent combustion closure: A posteriori validation. Combust. Flame
**2019**, 210, 279–291. [Google Scholar] [CrossRef][Green Version] - Laubscher, R. Utilization of basic multi-layer perceptron artificial neural networks to resolve turbulent fine structure chemical kinetics applied to a CFD model of a methane/air piloted jet flame. J. Therm. Eng.
**2018**, 4, 1828–1846. [Google Scholar] [CrossRef] - Li, S.; Yang, B.; Qi, F. Accelerate global sensitivity analysis using artificial neural network algorithm: Case studies for combustion kinetic model. Combust. Flame
**2016**, 168, 53–64. [Google Scholar] [CrossRef] - Aceves, S.M.; Flowers, D.L.; Chen, J.Y.; Babajimopoulos, A. Fast Prediction of HCCI Combustion with an Artificial Neural Network Linked to a Fluid Mechanics Code; Technical Report, SAE Technical Paper; SAE: Warrendale, PA, USA, 2006. [Google Scholar]
- Blasco, J.; Fueyo, N.; Dopazo, C.; Ballester, J. Modelling the temporal evolution of a reduced combustion chemical system with an artificial neural network. Combust. Flame
**1998**, 113, 38–52. [Google Scholar] [CrossRef] - Blasco, J.A.; Fueyo, N.; Larroya, J.; Dopazo, C.; Chen, Y.J. A single-step time-integrator of a methane–air chemical system using artificial neural networks. Comput. Chem. Eng.
**1999**, 23, 1127–1133. [Google Scholar] [CrossRef] - Talebi-Moghaddam, S.; Bauer, F.; Huber, F.; Will, S.; Daun, K. Inferring soot morphology through multi-angle light scattering using an artificial neural network. J. Quant. Spectrosc. Radiat. Transf.
**2020**, 251, 106957. [Google Scholar] [CrossRef] - Taghavifar, H.; Taghavifar, H.; Mardani, A.; Mohebbi, A.; Khalilarya, S.; Jafarmadar, S. Appraisal of artificial neural networks to the emission analysis and prediction of CO
_{2}, soot, and NO_{x}of n-heptane fueled engine. J. Clean. Prod.**2016**, 112, 1729–1739. [Google Scholar] [CrossRef] - Ghazikhani, M.; Mirzaii, I. Soot emission prediction of a waste-gated turbo-charged DI diesel engine using artificial neural network. Neural Comput. Appl.
**2011**, 20, 303–308. [Google Scholar] [CrossRef] - Alcan, G.; Yilmaz, E.; Unel, M.; Aran, V.; Yilmaz, M.; Gurel, C.; Koprubasi, K. Estimating soot emission in diesel engines using gated recurrent unit networks. IFAC-PapersOnLine
**2019**, 52, 544–549. [Google Scholar] [CrossRef] - Yang, G.; Wang, Y.; Li, X. Prediction of the NOx emissions from thermal power plant using long-short term memory neural network. Energy
**2020**, 192, 116597. [Google Scholar] [CrossRef] - Dworkin, S.; Cooke, J.; Bennett, B.; Connelly, B.; Long, M.; Smooke, M.; Hall, R.; Colket, M. Distributed-memory parallel computation of a forced, time-dependent, sooting, ethylene/air coflow diffusion flame. Combust. Theory Model.
**2009**, 13, 795–822. [Google Scholar] [CrossRef] - Dworkin, S.B.; Zhang, Q.; Thomson, M.J.; Slavinskaya, N.A.; Riedel, U. Application of an enhanced PAH growth model to soot formation in a laminar coflow ethylene/air diffusion flame. Combust. Flame
**2011**, 158, 1682–1695. [Google Scholar] [CrossRef][Green Version] - Mueller, M.; Blanquart, G.; Pitsch, H. Hybrid method of moments for modeling soot formation and growth. Combust. Flame
**2009**, 156, 1143–1155. [Google Scholar] [CrossRef] - LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature
**2015**, 521, 436–444. [Google Scholar] [CrossRef] [PubMed] - Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed] - Yu, Y.; Si, X.; Hu, C.; Zhang, J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput.
**2019**, 31, 1235–1270. [Google Scholar] [CrossRef] [PubMed] - Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res.
**2014**, 15, 1929–1958. [Google Scholar] - The Mathworks Inc. TrainingOptions. Available online: https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html (accessed on 12 January 2021).

**Figure 3.**An example of part of the input data: H2 mass fraction at 10 time steps. The first 9 time steps is the training data while the last time step is the test data.

**Figure 4.**The concatenation process to convert the 3 dimensional input (time plus the axial and radial dimensions in space) into a spatiotemporal array. Every input variable, as well as the output variable of soot volume fraction, has its own spatiotemporal array.

**Figure 6.**The architecture of the selected neural network. Note that the dropout bubbles are not their own layers of nodes, but they signal that dropout has been applied on the previous layer of nodes.

**Figure 7.**Soot volume fraction ${f}_{v}$ (ppm) compared at different time steps between CFD and the neural network (training data only). Left: LSTM neural network, right: CFD.

**Figure 8.**Soot volume fraction ${f}_{v}$ (ppm) compared between CFD and the neural network for the test data (the $10\mathrm{th}$ time step). Left: LSTM neural network, right: CFD.

**Table 1.**Hyperparameters tested for the LSTM structure. Bold entries represent the selected hyperparameters.

No. of LSTM layers | 1, 2, 3 |

No. of dropout layers | 1, 2, 3 |

No. of hidden units/LSTM layer | 10, 20, 30, 50, 75, 100, 150, 200, 250, 300 |

No. of epochs | 20, 25, 30, 35, 50, 75, 100, 200, 300, 500, 1000 |

Gradient threshold | 0.1, 0.25, 0.5, 1, 1.5, 2, 5 |

Learning rate schedule | Piecewise |

Initial learning rate | 0.001, 0.005, 0.01, 0.05, 0.1, 0.2, 0.5 |

Learning rate drop period | 10, 20, 50, 100 |

Learning rate drop factor | 0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.1 |

Dropout (%) | 1, 5, 10, 50 |

Training algorithm | Adam, Stochastic Gradient Descent |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jadidi, M.; Di Liddo, L.; Dworkin, S.B. A Long Short-Term Memory Neural Network for the Low-Cost Prediction of Soot Concentration in a Time-Dependent Flame. *Energies* **2021**, *14*, 1394.
https://doi.org/10.3390/en14051394

**AMA Style**

Jadidi M, Di Liddo L, Dworkin SB. A Long Short-Term Memory Neural Network for the Low-Cost Prediction of Soot Concentration in a Time-Dependent Flame. *Energies*. 2021; 14(5):1394.
https://doi.org/10.3390/en14051394

**Chicago/Turabian Style**

Jadidi, Mehdi, Luke Di Liddo, and Seth B. Dworkin. 2021. "A Long Short-Term Memory Neural Network for the Low-Cost Prediction of Soot Concentration in a Time-Dependent Flame" *Energies* 14, no. 5: 1394.
https://doi.org/10.3390/en14051394