# Artificial Neural Networks as Metamodels for the Multiobjective Optimization of Biobutanol Production

^{*}

## Abstract

**:**

## 1. Introduction

^{®}Design R430 process flowsheet took a few days to obtain one Pareto domain [1]. One alternative is to use a metamodel, or a surrogate model, where complete or partial operation of the chemical process is modelled with a representative model, often empirical, for which the computational cost would be significantly reduced. One such model is an artificial neural network (ANN), which is a computational network that attempts to mimic the functionality of neurons within the biological central nervous system. In an ANN, adaptable nodes store experiential knowledge acquired via learning algorithms, allowing the network to recognize and predict patterns with no knowledge of the underlying governing equations. This allows ANNs to be used as black-box tools where no prior knowledge about the system is required, thereby achieving high accuracy in multifactorial and nonlinear analysis of complex processes such as fermentation. However, training a neural network requires a good number of representative solutions, which may counteract efforts to reduce the computation time. Strategies to develop effective neural network models for smaller sets of data using a combination of experimental design and stacked neural networks have been previously proposed [2,3]. In an effort to maximize process information given a small set of data, it is possible to use experimental design. Numerous optimization problems could benefit from such a strategy.

## 2. Description of the Integrated ABE Fermentation–Membrane Pervaporation Process

^{3}fermenter with a constant flow rate and sugar concentration. Initial species concentrations in the fermenter are assumed. The set of kinetic reactions as described by Mulchandani and Volesky [11], accounting for the production of solvents (acetone, butanol, and ethanol), intermediate products (acetic acid and butyric acid), and microbial cells as well as the consumption of glucose in the fermentation broth was used. This model is still the most commonly used in the literature, because it accounts for the carbon substrate limitation in addition to the inhibition of butanol and butyric acid. Other models were recently proposed in the literature and were tested in this investigation. Shinto et al. [12] presented a kinetic model that considers numerous intermediates in the metabolic pathway as well as product and glucose inhibition. However, this model could not be used, as it leads to unrealistically high butanol concentration, far beyond the inhibitory concentration levels. The dynamic model of Buehler and Mesbah [13] initially seemed interesting, as it accounts for the pH of the fermentation broth, but some dynamic model constants are extremely high, therefore the system of ordinary differential equations cannot be solved with a practical step size in terms of time.

## 3. Methodology

#### 3.1. Multiobjective Process Optimization

#### 3.2. Uniform Design of Experiment

_{n}(q

^{s}), given the number of runs (n), factors (s), and levels (q) [20]. Each table provides the desired number of design points; each design point contains the number of factors (or decision variables) and their respective level, which can be used to determine the actual values of the decision variables using their allowable ranges. These design points (decision variables) are then used to generate the respective objective functions. In this investigation, uniform design U

_{30}(5

^{5}) and U

_{10}(5

^{5}) were used to generate the learning and validation datasets for the artificial neural network (ANN), respectively.

#### 3.3. Artificial Neural Network Metamodels

_{30}(5

^{5}) and U

_{10}(5

^{5}) were assigned for training and validation of the neural networks, respectively. The architecture of the ANN with inputs and different responses is schematically depicted in Figure 3. The inputs to the ANNs are the five decision variables: dilution rate (D), inlet glucose concentration (S

_{o}), cell retention factor (α

_{8/12}), number of pervaporation modules in series (N

_{mod}), and number of stacked membranes in their respective membrane module (N

_{Stack}). Two strategies were used in the derivation of ANNs. The first one was aimed at replacing the entire continuous integrated ABE fermentation process, whereby each of the four objective functions (butanol productivity, overall butanol concentration, sugar conversion, and number of membrane modules in series) is modelled. For this first strategy, four ANNs were derived and could be used independent of the actual integrated fermentation model. For a given set of decision variables, the four neural networks can be used in the multiobjective optimization algorithm to calculate the four objective functions. The Pareto domain can therefore be circumscribed using the ANN surrogate models. In the second strategy, it is desired to use the actual integrated ABE fermentation model to converge to steady-state operation but use ANNs to predict the initial concentrations of the main species within the fermentation broth (acetone, acetic acid, butanol, butyric acid, ethanol, glucose, bacteria, and water). The motivation of the latter strategy, with the derivation of eight ANNs, is to converge more rapidly to a steady state, resulting in reduced computation time.

## 4. Results and Discussion

_{8/12}.

^{2}) are included on each graph and clearly indicate a significant improvement of the metamodels when the reduced ranges are used. Moreover, for all variables, the numerous low values and zeros present with the wider ranges of decision variables disappeared for the ANN predictions obtained with the reduced decision variable ranges. The main change in the ranges was the cell retention factor, which now assumes a unique value of 0.1, allowing elimination of the fermenter washout and retention of a higher microorganism concentration within the fermenter.

^{3}∙h and an overall butanol concentration of 21.74 kg/m

^{3}(Table 3, column A). When the decision variables associated with the optimal solution obtained with the ANN were used in the phenomenological simulator to validate this optimal point, values of 12.14 kg/m

^{3}∙h and 24.04 kg/m

^{3}were obtained, respectively. This is comparable to the best-ranked solution obtained by the genetic algorithm using the phenomenological simulator, as depicted in Figure 8b, where butanol productivity of 11.83 kg/m

^{3}∙ was obtained (Table 3, column C). Figure 8 shows the positive correlation between butanol productivity and overall butanol concentration at a constant dilution rate. The dilution rate increases diagonally (left to right) from a lower bound of 0.5 h

^{−1}to an upper bound of 1.4 h

^{−1}. The empty portion depicted in the Pareto domain of Figure 8b corresponds to an inoperable range due to the constraints that exist between elevated dilution rate, butanol concentration, and flow rates to achieve steady state. The metamodel failed to recognize these limitations and instead filled the empty region accordingly by interpolating the data. This observation is expected, as a metamodel is based on input–output observations without accounting for the intrinsic constraints of the system. Nevertheless, the best-ranked solution and the solutions ranked in the first 5% were well identified with the metamodels.

^{3}and a dilution rate of 0.5 h

^{−1}for both strategies. The third decision variable, the cell retention factor, was kept constant at 0.1. A small value of the cell retention factor implies that a higher microorganism concentration prevails inside the fermenter. The optimal operating conditions as determined with the Pareto domains obtained with the three strategies described in Figure 4 are summarized in Table 3. The two sub-columns of Table 3 for strategy A correspond to the first three objectives predicted with the five decision variables by the ANN metamodel and the phenomenological simulator, respectively. The latter is recorded for validation purposes.

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Thibault, J.; Filho, R.M.; Dias, M.O.S.; Junqueira, T.L.; Cavalett, O.; Jesus, C.D.F.; Rossell, C.E.V.; Bonomi, A. Simulation and multi-objective optimization of vacuum ethanol fermentation. In Proceedings of the 2nd International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH 2012), Rome, Italy, 28–31 July 2012; pp. 79–86. [Google Scholar]
- Lanouette, R.; Thibault, J.; Valade, J.L. Process modeling with neural networks using small experimental datasets. Comput. Chem. Eng.
**1999**, 23, 1167–1176. [Google Scholar] [CrossRef] - Lin, Y.; Zhang, Z.; Thibault, J. Comparison of experimental designs using neural networks. Can. J. Chem. Eng.
**2009**, 87, 965–971. [Google Scholar] [CrossRef] - Green, E.M.; Boynton, Z.L.; Harris, L.M.; Rudolph, F.B.; Papoutsakis, E.T.; Bennett, G.N. Genetic manipulation of acid formation pathways by gene inactivation in Clostridium acetobutylicum ATCC 824. Microbiology
**1996**, 142, 2079–2086. [Google Scholar] [CrossRef] [PubMed][Green Version] - Friedl, A.; Qureshi, N.; Maddox, I.S. Continuous acetone-butanol-ethanol (ABE) fermentation using immobilized cells ofClostridium acetobutylicum in a packed bed reactor and integration with product removal by pervaporation. Biotechnol. Bioeng.
**1991**, 38, 518–527. [Google Scholar] [CrossRef] [PubMed] - Millat, T.; Winzer, K. Mathematical modelling of clostridial acetone-butanol-ethanol fermentation. Appl. Microbiol. Biotechnol.
**2017**, 101, 2251–2271. [Google Scholar] [CrossRef] [PubMed][Green Version] - Liao, C.; Seo, S.-O.; Lu, T. System-level modeling of acetone–butanol–ethanol fermentation. FEMS Microbiol. Lett.
**2016**, 363. [Google Scholar] [CrossRef] [PubMed][Green Version] - Abdehagh, N.; Tezel, F.H.; Thibault, J. Separation techniques in butanol production: Challenges and developments. Biomass Bioenergy
**2014**, 60, 222–246. [Google Scholar] [CrossRef] - Dong, Z.; Liu, G.; Liu, S.; Liu, Z.; Jin, W. High performance ceramic hollow fiber supported PDMS composite pervaporation membrane for bio-butanol recovery. J. Membr. Sci.
**2014**, 450, 38–47. [Google Scholar] [CrossRef] - Azimi, H. Pervaporation Separation of Butanol Using PDMS Mixed Matrix Membranes. Ph.D. Thesis, University of Ottawa, Ottawa, ON, Canada, 2017. [Google Scholar]
- Mulchandani, A.; Volesky, B. Modelling of the acetone-butanol fermentation with cell retention. Can. J. Chem. Eng.
**1986**, 64, 625–631. [Google Scholar] [CrossRef] - Shinto, H.; Tashiro, Y.; Yamashita, M.; Kobayashi, G.; Sekiguchi, T.; Hanai, T.; Kuriya, Y.; Okamoto, M.; Sonomoto, K. Kinetic modeling and sensitivity analysis of acetone-butanol-ethanol production. J. Biotechnol.
**2007**, 131, 45–56. [Google Scholar] [CrossRef] [PubMed] - Buehler, E.A.; Mesbah, A. Kinetic study of acetone-butanol-ethanol fermentation in continuous culture. PLoS ONE
**2016**. [Google Scholar] [CrossRef] [PubMed] - Ezeji, T.C.; Qureshi, N.; Blaschek, H.P. Microbial production of a biofuel (acetone–butanol–ethanol) in a continuous bioreactor: Impact of bleed and simultaneous product removal. Bioprocess Biosyst. Eng.
**2013**, 36, 109–116. [Google Scholar] [CrossRef] [PubMed] - Halsall-Whitney, H.; Thibault, J. Multi-objective optimization for chemical processes and controller design: Approximating and classifying the Pareto domain. Comput. Chem. Eng.
**2006**, 30, 1155–1168. [Google Scholar] [CrossRef] - Thibault, J. Net Flow and Rough Sets: Two Methods for Ranking the Pareto Domain. In Multi-Objective Optimization—Techniques and Applications in Chemical Engineering. Advances in Process Systems Engineering; Rangaiah, P.G., Ed.; World Scientific: Singapore, 2009; Volume 1. [Google Scholar]
- Lin, Y.; Zhang, Z.; Thibault, J. Aureobasidium pullulans batch cultivations based on a factorial design for improving the production and molecular weight of exopolysaccharides. Process Biochem.
**2007**, 42, 820–827. [Google Scholar] [CrossRef] - Caballero, J.A.; Grossmann, I.E. An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE J.
**2008**, 54, 2633–2650. [Google Scholar] [CrossRef] - Fang, K.-T.; Lin, D.K.J.; Winker, P.; Zhang, Y. Uniform Design: Theory and Application. Technometrics
**2000**, 42, 237–248. [Google Scholar] [CrossRef] - Yeung, K. The Uniform Design. Available online: http://www.math.hkbu.edu.hk/UniformDesign (accessed on 13 January 2018).
- Holeña, M.; Linke, D.; Rodemerck, U.; Bajer, L. Neural Networks as Surrogate Models for Measurements in Optimization Algorithms. In Proceedings of the International Conference on Analytical and Stochastic Modeling Techniques and Applications (ASMTA 2010), Cardiff, UK, 14–16 June 2010; Al-Begain, K., Fiems, D., Knottenbelt, W., Eds.; Springer: Berlin, Germany, 2010; pp. 351–366. [Google Scholar]
- Anna, H.R.S.; Barreto, A.G.; Tavares, F.W.; de Souza, M.B. Machine learning model and optimization of a PSA unit for methane-nitrogen separation. Comput. Chem. Eng.
**2017**, 104, 377–391. [Google Scholar] [CrossRef] - Field, R.J.; Noyes, R.M. Oscillations in chemical systems. 18. Mechanisms of chemical oscillators: Conceptual bases. Acc. Chem. Res.
**1977**, 10, 214–221. [Google Scholar] [CrossRef]

**Figure 1.**Schematic diagram of the butanol fermentation system integrated with a membrane pervaporation separation unit. Numbers correspond to process streams.

**Figure 2.**Decision variables and objective functions defining the optimization process of the integrated biobutanol production process.

**Figure 3.**Architecture of the artificial neural network (ANN) used for the modelling of the four objectives and eight initial species concentrations of the integrated fermentation–membrane pervaporation system.

**Figure 4.**Multiobjective optimization and analysis of the integrated fermentation–membrane pervaporation system flowchart: (A) ANN metamodels of the four objective functions; (B) process simulator with ANN metamodels providing the eight initial species concentrations; and (C) process simulator with arbitrary initial concentrations.

**Figure 5.**Predicted versus actual butanol concentration within the fermenter under steady state for (

**a**) initial ranges and (

**b**) reduced ranges of the five decision variables.

**Figure 6.**Predicted versus actual butanol productivity for (

**a**) initial ranges and (

**b**) reduced ranges of the five decision variables.

**Figure 7.**Predicted versus actual overall process butanol concentration for (

**a**) initial ranges and (

**b**) reduced ranges of the five decision variables.

**Figure 8.**Plots of the Pareto-optimal overall butanol concentration versus butanol productivity for the integrated fermentation–membrane pervaporation system: (

**a**) ANN metamodels; (

**b**) actual integrated acetone–butanol–ethanol (ABE) fermentation simulation.

**Figure 9.**Plots of sugar conversion versus butanol productivity for the integrated fermentation–membrane pervaporation system: (

**a**) ANN metamodels; (

**b**) actual integrated system simulation.

**Figure 10.**Plots of the numbers of membrane modules versus butanol productivity for the integrated fermentation–membrane pervaporation system: (

**a**) ANN metamodels; (

**b**) actual integrated system simulation.

**Figure 11.**Plots of feed sugar concentration versus dilution rate for the integrated fermentation–membrane pervaporation system: (

**a**) ANN metamodels; (

**b**) actual integrated system simulation.

**Figure 12.**Butanol and biomass concentration as a function of time with (

**a**) initial concentrations predicted by ANNs and (

**b**) arbitrary initial concentrations. (1) Best ranked solution; (2) medium-ranked solution; (3) lowest-ranked solution.

Range | D (h^{−1}) | S_{o} (kg/m^{3}) | α_{8/12} | N_{mod} | N_{Stack} |
---|---|---|---|---|---|

Initial | 0.5–1.5 | 50–150 | 0.1–0.3 | 2–10 | 1000–2500 |

Reduced | 0.5–1.4 | 50–125 | 0.1 | 4–10 | 2000–2500 |

Objective | Relative Weight | Thresholds | ||
---|---|---|---|---|

Indifference | Preference | Veto | ||

Butanol productivity | 0.35 | 0.75 | 1.50 | 3.00 |

Butanol concentration | 0.35 | 0.75 | 1.50 | 3.00 |

Sugar conversion | 0.25 | 0.04 | 0.08 | 0.16 |

Number of membrane modules | 0.05 | 1.00 | 2.00 | 4.00 |

**Table 3.**Decision variables and objectives of best solutions of Pareto domains with a population of 5000 individuals obtained with strategies A, B and C (see Figure 4).

Decision Variable/Objective | Strategy | |||
---|---|---|---|---|

A | B | C | ||

Dilution rate D (h^{−1}) | 0.51 | 0.51 | 0.50 | |

Feed sugar concentration S_{0} (kg/m^{3}) | 125 | 122 | 125 | |

Cell retention factor α_{8/12} | 0.1 | 0.1 | 0.1 | |

Number of membrane modules N_{Mod} | 7 | 7 | 7 | |

Number of membrane stacks N_{Stack} | 2500 | 2480 | 2389 | |

Butanol productivity (kg/m^{3}∙h) | 10.95 ^{1} | 12.14 ^{2} | 12.10 | 11.83 |

Butanol concentration (kg/m^{3}) | 21.74 ^{1} | 24.04 ^{2} | 23.68 | 23.65 |

Sugar conversion | 0.96 ^{1} | 0.96 ^{2} | 0.95 | 0.93 |

Computation time (s) | 5.6 | 8353 | 14052 | |

Number of actual model evaluations | 81 | 21573 | 21347 |

^{1}Values determined by the genetic algorithm using the ANN metamodels.

^{2}Values determined by the phenomenological simulator for validation using decision variables of Column A.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Elmeligy, A.; Mehrani, P.; Thibault, J. Artificial Neural Networks as Metamodels for the Multiobjective Optimization of Biobutanol Production. *Appl. Sci.* **2018**, *8*, 961.
https://doi.org/10.3390/app8060961

**AMA Style**

Elmeligy A, Mehrani P, Thibault J. Artificial Neural Networks as Metamodels for the Multiobjective Optimization of Biobutanol Production. *Applied Sciences*. 2018; 8(6):961.
https://doi.org/10.3390/app8060961

**Chicago/Turabian Style**

Elmeligy, Ahmed, Poupak Mehrani, and Jules Thibault. 2018. "Artificial Neural Networks as Metamodels for the Multiobjective Optimization of Biobutanol Production" *Applied Sciences* 8, no. 6: 961.
https://doi.org/10.3390/app8060961