Hybrid Modelling Framework for Reactor Model Discovery Using Artificial Neural Networks Classifiers ^†

Abstract

Developing and identifying the correct reactor model for a reaction system characterized by a high number of reaction pathways and flow regimes can be challenging. In this work, artificial neural networks (ANNs), used in deep learning, are used to develop a hybrid modelling framework for physics-based model discovery in reactions systems. The model discovery accuracy of the framework is investigated considering kinetic model parametric uncertainty, noise level, features in the data structure and experimental design optimization via a differential evolution algorithm (DEA). The hydrodynamic behaviours of both a continuously stirred tank reactor and a plug flow reactor and rival chemical kinetics models are combined to generate candidate physics-based models to describe a benzoic acid esterification synthesis in a rotating cylindrical reactor. ANNs are trained and validated from in silico data simulated by sampling the parameter space of the physics-based models. Results show that, when monitored using test data classification accuracy, ANN performance improved when the kinetic parameters uncertainty decreased. The performance improved further by increasing the number of features in the data set, optimizing the experimental design and decreasing the measurements error (low noise level).

1. Introduction

Increasing applications of artificial neural networks (ANNs) for regression and classification tasks in chemical engineering areas including separation processes, chemical thermodynamics and reaction kinetics have been published [1]. Quaglio et al. [2] developed an ANN framework for kinetic model recognition in a batch reaction system composed of series and parallel chemical steps. Using power law kinetics, the authors derived possible candidate kinetic models from the reaction network. Then, they sampled the parameter space to create a large data set satisfying reaction yield and selectivity constraints and simulated the derived kinetic models to generate training data for the ANN, considering fixed experimental conditions and different levels of noise in the simulated data, to mirror real experimental measurement error. Recently, Sangoi et al. [3] extended the ANN framework to employ optimal design of experiments (DoE) to design a set of experimental conditions providing the highest classification accuracy using a differential evolution algorithm (DEA) in the DoE optimization.

While a kinetic model represents an important set of constitutive equations for a reaction system, a complete mathematical description considers reactor flow hydrodynamics coupled with chemical kinetics [4]. In this work, we develop a general hybrid modelling framework that combines physics-based modelling and artificial neural networks for reactor model discovery, considering a reaction system for benzoic acid esterification. This reaction system has been described using two rival kinetic expressions reported [5]. We further consider the impacts of parametric uncertainty, level of noise, number of features and of using DEA.

2. Hybrid Modelling Framework

Figure 1 illustrates the general methodology comprising a chemical system and the modelling framework. The chemical system is a flow reactor whose input and process variables (${\mathit{x}}_0,\mathit{u})$ at steady state are contained in the experimental setpoint denoted by $\mathit{\phi }$. ${\mathit{x}}_0$ is the set of reactor inlet concentrations and $\mathit{u}$ is the set of control variables such as the reactor temperature $T$ and residence time $\tau$. The progress of the reaction can then be monitored using the experimental data denoted by $\widehat{\mathit{y}}$ as observed in the reactor output stream.

The modelling framework is illustrated in Figure 1 and is constituted by Parts 1 and 2, relating the experimental setpoint to the observed experimental data. Part 1 employs a library of physics-based models, each denoted in general form as ${\mathit{f}}_i\left(\mathit{x},\mathit{u},{\mathit{\theta }}_j\right)=0$ and computed with parametric values ${\mathit{\theta }}_j$ sampled in the model parameters space (${\mathit{\theta }}_j\in \mathsf{\Theta }$), to generate simulated data of the chemical system. Part 2, on the other hand, employs an ANN model composed of layers of nodes and activation function parameters ($\mathit{w},b$): $\rho =\phi ({\mathit{w}}^T.\mathit{n}+b)$, to learn the simulated data and predict the most suitable physics-based model for the experimental data.

The DEA used in the experimental design optimization, shown in the lower half of Figure 1 employs the two core parts in the whole experimental design space $\mathsf{\Phi }$ to propose optimal experiments for the reactor ${\mathit{\phi }}^{\mathit{o}\mathit{p}\mathit{t}}$.

3. Reaction System Case Study

Benzoic acid esterification, a synthesis for producing esters used as intermediates in the pharmaceutical and fine chemical industry, is considered [5]:

$$BA+EtOH\to W+EB$$

(1)

Previous model investigation on this synthesis conducted within the experimental design space reported in

Table 1. Benzoic acid esterification experimental design space [5].

Limits	${\mathit{c}}_{\mathit{B}\mathit{A}}\left(0\right)\left(\mathbf{M}\right)$	$\mathit{F}\left(\mathsf{\mu }\mathbf{L}/\mathbf{m}\mathbf{i}\mathbf{n}\right)$	$\mathit{T}\left(°\mathbf{C}\right)$
Upper	$1.0$	$20$	$100$
Lower	$1.5$	$40$	$120$

considered two chemical kinetics [5]:

$$M_1: k_1{C}_{BA}{C}_{EtOH}$$

(2)

$$M_2: {\displaystyle \frac{k_1{C}_{BA}{C}_{EtOH}}{{\left(1+K_W{C}_W\right)}^2}}$$

(3)

where $k_1=\exp \left(-{KP}_1-{\displaystyle \frac{{KP}_2\times 10000}{RT}}\left[{\displaystyle \frac{1}{T}}-{\displaystyle \frac{1}{T_{ref}}}\right]\right)$; $T_{ref}=378.15\mathrm{K}$

Table 2. Benzoic acid esterification kinetic parameter space [5].

	${\mathit{K}\mathit{P}}_1$	${\mathit{K}\mathit{P}}_1$	${\mathit{K}}_{\mathit{w}}$
Imprecise	$16.7\pm 0.07$	$5.93\pm 0.60$	$0.28\pm 0.39$
Precise	$16.7\pm 0.02$	$5.93\pm 0.20$	$0.27\pm 0.10$

introduces two potential scenarios describing kinetic parameters uncertainty: (1) “Imprecise”: a scenario characterized by wide confidence intervals; (2) “Precise”: a scenario characterized by narrow confidence intervals reported for the reaction kinetic expressions.

The reactor is configured to have two concentric cylinders with the reaction occurring in the annulus, where the fluid is mixed by rotating the internal cylinder. This configuration can be described using a cascade of CSTRs. In this work we consider 1 CSTR and infinite CSTRs-in-series or a PFR, expressed mathematically as R1 and R2, respectively [6].

$$R_1: c_i=c_{i0}+\tau r_i; \mathrm{and} R_2: {\displaystyle \frac{d{c}_i}{d\tau }}=r_i; \forall i=BA, EtOH, W and EB$$

(4)

A matrix of possible candidate physics-based models is therefore generated as reported in

Table 3. A matrix of 4 candidate physics-based models.

	${\mathit{M}}_1$	${\mathit{M}}_2$
$R_1$	$(R_1,M_1)$	$(R_1,M_2)$
$R_2$	$(R_2,M_1)$	$(R_2,M_2)$

.

4. Results

For simulation, the physics-based models in Table 3, labelled as 1: ($R_1,M_1$), 2: ($R_1,M_2$), 3: ($R_2,M_1$), and 4: ($R_1,M_1$), employed the experimental design space shown in Table 1. The ANN employed 2 hidden layers, 32 nodes per hidden layer and the SoftMax function for quaternary classification in the output layer. The input layer nodes were a multiple of the 4 outlet reactant concentrations simulated, increasing with the number of experiments designed $N_{exp}$,which is one of the key variable factors in the modelling framework that can impact performance. In this work two instances were investigated:(1) the design of one experiment ($N_{exp}=1$); (2) the design of two experiments $(N_{exp}=2)$. Other variable factors considered include parametric uncertainty, and noise level. Two scenarios for the kinetic model parametric information regions were considered: imprecise and precise confidence regions as shown in Table 2, by sampling 1000 sets of parameter values in the corresponding parameter space. Three levels of noise (0%, 1%, and 10% constant relative standard deviation) were added to the simulated data, using a split ratio of 60:20:20 for training, validation and testing of the ANN architecture.

First, we considered a single experiment fixed in the experimental design space reported in Table 1: $c_{BA}\left(0\right) =1.5 \mathrm{M}; F=40{\displaystyle \frac{\mathsf{\mu }\mathrm{L}}{\min }}; T=120 °\mathrm{C}$ and simulated without noise, because the no-noise scenario does not imply a perfect classification. Figure 2 shows the resulting ANN confusion matrices (labelled, 1: ($R_1,M_1$), 2: ($R_1,M_2$), 3: ($R_2,M_1$), and 4: ($R_1,M_1$) on the test data for the imprecise and precise prior parametric information, equivalent to 75 and 90% ANN accuracy, respectively. The prior information on model parameters determines the physics-based model precision with precise prior information on kinetic parameters yielding model simulations different from other physics-based models and thus improving the ANN performance by 15%. In all cases considered, the modelling framework performed better with precise prior parametric information than with imprecise prior parametric information. For example, corrupting with noise (1% and then 10%), the ANN performance with the precise parameter space decreased to 85 and 44% at 1 and 10% constant relative variance noise, respectively. These performances were better than those with the imprecise parameter space: 74 and 41% for 1 and 10% constant relative variance noise, respectively. The ANN performance can be improved by increasing the number of simulated experiments and employing the DEA design space optimization. A perfect classification accuracy of 100% was achieved by using the DEA to design two optimal experiments and simulating the physics-based models with the precise parametric information without adding noise.

5. Conclusions

A hybrid modelling framework for physics-based model discovery using an ANN classifier has been developed and tested for parametric uncertainty, level of noise, number of features and DEA optimization. Within the hybrid framework, a library of physics-based candidate models with parametric values sampled from the model parameter space generated large in silico data used for training the ANN classifier. On testing, the framework performance increased with number of features and design space optimization but decreased with level of noise and parametric uncertainty. The developed modelling framework can be used to recognize a suitable reactor model, based on the availability of actual experimental data for the rapid characterization and digitalization of reactor systems.