Surrogate Modeling for Liquid–Liquid Equilibria Using a Parameterization of the Binodal Curve

Abstract

Computational effort and convergence problems can pose serious challenges when employing advanced thermodynamic models in process simulation and optimization. Data-based surrogate modeling helps to overcome these problems at the cost of additional modeling effort. The present work extends the range of methods for efficient data-based surrogate modeling of liquid–liquid equilibria. A new model formulation is presented that enables smaller surrogates with box-constrained input domains and reduced input dimensions. Sample data are generated efficiently by using numerical continuation. The new methods are demonstrated for the surrogate modeling and optimization of a process for the hydroformylation of 1-decene in a thermomorphic multiphase system.

1. Introduction

In chemical process design, advanced thermodynamic models offer increasing predictive accuracy for physical properties of mixtures. However, this correlates with increasingly complex models, which introduce serious challenges when used for process simulation and optimization. Problems may arise from embedded iterative solution procedures for the thermodynamic models, which may not converge to a valid solution or require a large number of iteration steps. The benefits of advanced thermodynamic models can be used by replacing them with surrogates that are computationally easy to handle [1].

The concept of surrogate modeling comprises a number of different approaches [2] that depend on the type of the original model and the desired properties of the surrogate. In chemical process engineering, data-driven surrogate modeling is usually employed [3]. This class of methods generally treats the original model as a black box with input–output data. The surrogate that replaces the original model is constructed such that it approximates the data and is easy to evaluate. These requirements define major steps of data-driven surrogate modeling.

In a first step, inputs and outputs have to be selected from the original model. This determines the properties of the input–output data and thereby affects both the accuracy and size of the surrogate model. A larger input dimension increases the size of the surrogate and necessitates more sample data. A difficult input domain may require modeling with an additional surrogate [4]. In addition, output multiplicities have to be treated with special methods [5].

Several data-driven surrogate models are available, e.g., polynomials, neural networks, and Gaussian processes [2]. The choice for the surrogate model structure is usually not trivial and a number of different models should be tried. Tools for automatically selecting a surrogate model structure are available (e.g., [6]).

Parameter fitting for surrogate models requires sample data that fully characterize the input–output behavior. The additional cost for sample generation is reduced by employing efficient sampling methods. Adaptive sampling approaches that employ exploration and exploitation strategies are preferred over one-shot methods in recent literature (see [3,4] for references).

Although various strategies are available for each aspect of surrogate modeling, there is still significant potential for improvements, especially when problem-specific features are taken into account.

The present contribution extends the methods of surrogate modeling for liquid–liquid phase equilibria (LLE). This includes efficient sample generation by employing a numerical continuation method and an efficient surrogate formulation that has a box-constrained input domain and a reduced input dimension compared to established formulations. In Section 2, LLE modeling is introduced briefly and a description of surrogate modeling of LLE is given that summarizes recent work on the topic. Section 3 describes the methods of sample data generation and surrogate modeling proposed in this work. A case study based on a process for the hydroformylation of 1-decene in a thermomorphic multiphase system is used in Section 4 to demonstrate the proposed methods. A short conclusion is given in Section 5.

2. Background

This section summarizes the required background for LLE modeling and recent work on surrogate modeling of LLE.

2.1. Liquid–Liquid Equilibrium Modeling

The liquid–liquid equilibrium is described by a two-phase flash model comprising component mass balance conditions

$$\begin{array}{c}\hfill N_i^{\mathrm{in}}=N_i^{\mathrm{I}}+N_i^{\mathrm{II}},i=1,\dots ,N_{\mathrm{c}},\end{array}$$

(1)

and thermodynamic equilibrium conditions

$$\begin{array}{cc}\hfill & T^{\mathrm{I}}=T^{\mathrm{II}}=T,\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill & p^{\mathrm{I}}=p^{\mathrm{II}}=p,\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & {\mu }_i^{\mathrm{I}}={\mu }_i^{\mathrm{II}},i=1,\dots ,N_{\mathrm{c}}.\hfill \end{array}$$

(4)

Here, $N_i^j$ is the mole amount, ${\mu }_i^j$ is the chemical potential, $T^j$ is the temperature and $p^j$ is the pressure in phase $j\in \{\mathrm{I},\mathrm{II}\}$ for component i. The mole amounts of the feed mixture are denoted by $N_i^{\mathrm{in}}$ and the total number of components in the mixture is $N_{\mathrm{c}}$.

The conditions for equal chemical potential can be reformulated for liquid–liquid equilibria as the isoactivity conditions

$$\begin{array}{cc}\hfill & x_i^{\mathrm{I}}{\gamma }_i^{\mathrm{I}}({\mathit{x}}^{\mathrm{I}},T,p)=x_i^{\mathrm{II}}{\gamma }_i^{\mathrm{II}}({\mathit{x}}^{\mathrm{II}},T,p),i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(5)

with mole fractions $x_i^j$, vectors of mole fractions ${\mathit{x}}^j$, and activity coefficients ${\gamma }_i^j$.

Iterative procedures are usually utilized to find a solution of the implicit equilibrium model, evaluating the activity coefficients provided by the thermodynamic model in each iteration. Therefore, using advanced models such as the perturbed-chain statistical associating fluid theory (PC-SAFT) [7] or universal quasichemical functional-group activity coefficients (UNIFAC) [8] directly for flowsheet simulation and optimization is unfavorable in terms of convergence behavior and computational effort. Furthermore, excluding undesired trivial solutions of the equation system given above requires additional effort. The following section describes the technique of replacing such expensive calculations by data-driven surrogate models that are computationally easier to handle.

2.2. Surrogate Modeling of LLE

Surrogate modeling aims at replacing a function $\mathit{f}$,

$$\begin{array}{c}\hfill \left\{\right(\mathit{x},\mathit{f}\left(\mathit{x}\right)):\mathit{x}\in X\},\end{array}$$

(6)

with a surrogate function $\widehat{\mathit{f}}$ that is similar to the original function $\mathit{f}$ and exhibits favorable properties, e.g., much lower computational effort. More specifically, data-driven surrogate modeling generally uses samples $({\mathit{x}}_k,\mathit{f}\left({\mathit{x}}_k\right))$ to fit parameters of the surrogate $\widehat{\mathit{f}}$ such that some performance criterion is satisfied. If an approximation of the original function $\widehat{\mathit{f}}\approx \mathit{f}$ is desired, typical performance criteria include minimizing the mean squared error or mean absolute error with respect to the samples. Another performance criterion is the exact reproduction of the sample points $\widehat{\mathit{f}}\left({\mathit{x}}_k\right)=\mathit{f}\left({\mathit{x}}_k\right)$, which can be used for instance in Kriging interpolation (e.g., [9]). In this sense, data-driven surrogate modeling is another name for nonlinear regression and interpolation methods. Applications for surrogate modeling include, but are not limited to, cases where the original function is expensive to evaluate, e.g., computational fluid dynamics simulations, or where a functional form may not be available, e.g., experimental results.

This section gives a summary of recent efforts [4,10] on using data-driven surrogate models for phase equilibrium calculations, particularly for LLE, to improve computational performance while preserving high model accuracy. For the sake of readability, some details of each approach are omitted here and we refer to the original publications for comprehensive descriptions. A broader overview on general surrogate modeling is available in the literature [2,3,11,12].

2.2.1. Selection of Inputs and Outputs

Distribution coefficients $K_i:=N_i^{\mathrm{I}}/N_i^{\mathrm{in}}$ are introduced, which allows one to rewrite the equilibrium model as

$$\begin{array}{cc}\hfill & N_i^{\mathrm{I}}=K_i{N}_i^{\mathrm{in}},i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill & K_i=f_i(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T),i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & x_i^{\mathrm{in}}=\frac{N_i^{\mathrm{in}}}{{\sum }_{l=1}^{N_{\mathrm{c}}}{N}_l^{\mathrm{in}}},i=1,\dots ,N_{\mathrm{c}}-1,\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & N_i^{\mathrm{II}}=N_i^{\mathrm{in}}-N_i^{\mathrm{I}},i=1,\dots ,N_{\mathrm{c}}.\hfill \end{array}$$

(10)

In the model comprising Equations (7)–(10), $f_i$ is replaced by a surrogate ${\widehat{f}}_i\approx f_i$. Kriging interpolation is used for the surrogates ${\widehat{f}}_i$ in [4,10]. The inputs of the surrogate are $N_{\mathrm{c}}-1$ mole fractions of the feed mixture and the temperature. The outputs comprise the $N_{\mathrm{c}}$ distribution coefficients. The pressure does not appear as an input since its influence on the liquid–liquid equilibrium is assumed to be negligible.

It should be emphasized that, given a feasible feed mixture and temperature, all quantities in Equations (7)–(10) can be calculated in a sequential fashion.

2.2.2. Description of the Input Domain X

A natural choice for the input domain, given molar fractions and the temperature as inputs, would be

$$\begin{array}{c}\hfill \tilde{X}=\{(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T)\in {[0,1]}^{N_{\mathrm{c}}-1}\times {\mathbb{R}}^{+}:x_1^{\mathrm{in}}+\dots +x_{N_{\mathrm{c}}-1}^{\mathrm{in}}\le 1\}.\end{array}$$

(11)

However, the distribution coefficient model in Equations (7)–(10) is only valid within the miscibility gap of the mixture. Therefore, box constraints are not suitable to accurately describe the complete input domain X of the surrogate ${\widehat{f}}_i$. Instead, the input domain can be defined by a classifier $\mathit{g}(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T)$, which is less than zero only within the miscibility gap:

$$\begin{array}{c}\hfill X=\{(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T)\in \tilde{X}:\mathit{g}(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T)\le \mathbf{0}\}.\end{array}$$

(12)

Possible choices for the classifier model include a set of linear expressions $\mathit{g}=\mathit{A}{(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T)}^{\top }-\mathit{b}$ as in [10] or a scalar function $g(x_1^{\mathrm{in}},\dots ,x_{N_{\mathrm{c}}-1}^{\mathrm{in}},T)$ as in [4]. Since the scalar classifier g in [4] is as difficult to calculate as the equilibrium model, it is approximated by another surrogate model $\widehat{g}\approx g$, in this case a support vector machine.

2.2.3. Sample Data and Surrogate Generation

The selection of sample locations for data generation can be categorized in one-shot space-filling designs, which select locations according to pre-defined strategies, and adaptive sampling approaches, which exploit information from available sampling data to select new sampling locations. In [10], sample values are calculated by evaluating the equilibrium model at pre-defined sample locations generated by a space-filling design. A major contribution of Nentwich and Engell [4] is a method of adaptively choosing new sample locations based on available sample data to improve the quality of the approximation compared to one-shot designs with the same number of samples. In principle, new sample locations are chosen by maximizing a weighted sum of the minimum distance to existing sample locations and the predicted variance, i.e., by exploration and exploitation strategies. The surrogate model needs to be updated in each iteration of this method.

In both references, the sample generation takes a substantial amount of time. For a constant sample density, the number of required samples scales exponentially with the dimension of the input domain. In addition, the equilibrium model is solved at each sample location independently, i.e., the solution algorithm needs to converge starting from an independent initial guess.

3. Methods

The literature approaches to surrogate modeling of LLE rely on a regression model for the distribution of components between phases and an additional classification model to determine whether a given set of input values lies in the two-phase region. This section describes new approaches aiming at extending the pool of methods for surrogate modeling of LLE. For this, a model reformulation that enables smaller surrogate models and a method for efficiently calculating sample data are presented.

3.1. Parameterization of Binodal Curves

The strategy for reducing the computational effort for surrogates in LLE modeling presented here comprises two ideas. First, instead of modeling the entire miscibility gap, only the binodal curve is described by a surrogate, which reduces the surrogate input dimension by one. As shown in Figure 1 for a ternary mixture, the binodal curve has one dimension fewer than the miscibility gap, i.e., it is a line instead of an area. Note that for more than three components $N_{\mathrm{c}}>3$, the set of equilibrium compositions is a $(N_{\mathrm{c}}-2)$-dimensional surface. For a constant sample density, the number of required sample points for a surrogate model depends exponentially on its input dimension. Reducing the number of inputs therefore improves computational tractability of the surrogate.

Second, instead of using molar fractions as inputs of the surrogate, the position on the binodal curve is introduced as a new coordinate, marked by $t_1$ in Figure 1. Each value of this coordinate corresponds to the endpoints of a tie line, i.e., the compositions of two phases in equilibrium. The parameterization variable $t_1$ can be defined from the border of the phase diagram to the critical point, if the data are available, or just on a section of the binodal curve. This removes the need for a complicated (surrogate) model of the input domain, since the parameterization of the binodal curve can be chosen such that box constraints are sufficient, i.e., $t_1\in [0,1]$.

Using these ideas, the equilibrium model is reformulated as follows.

$$\begin{array}{cc}\hfill & N_i^{\mathrm{in}}=N_i^{\mathrm{I}}+N_i^{\mathrm{II}},i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & \frac{N_i^j}{{\sum }_{l=1}^{N_{\mathrm{c}}}{N}_l^j}=f_i^j(t_1,\dots ,t_{N_{\mathrm{c}}-2},T),i=1,\dots ,N_{\mathrm{c}}-1,j=\mathrm{I},\mathrm{II}\hfill \end{array}$$

(14)

Here, the parameterization variables for the binodal curve are denoted by $t_1,\dots ,t_{N_{\mathrm{c}}-2}$. In the model comprising Equations (13) and (14), $f_i^j$ is replaced by a surrogate ${\widehat{f}}_i^j\approx f_i^j$. The inputs of the surrogate are the temperature and $N_{\mathrm{c}}-2$ variables for the parameterization of the binodal curve. The input domain for the surrogate functions ${\widehat{f}}_i^j$ reads simply

$$\begin{array}{c}\hfill X={[0,1]}^{N_{\mathrm{c}}-2}\times {\mathbb{R}}^{+}.\end{array}$$

(15)

The outputs comprise $2(N_{\mathrm{c}}-1)$ independent mole fractions. We decided to include all $2N_{\mathrm{c}}$ molar fractions here, because eliminating one molar fraction by using the summation condition would make it harder to have the same error scaling for all components during surrogate fitting and it would increase the maximum error for the eliminated component in the final surrogate model.

In summary, the parameterization of binodal curves enables the generation of smaller surrogate models due to the reduction of the input dimension and the definition of the input domain by box constraints. The overall model is rendered implicit by this reformulation. Implicit models are generally harder to solve than explicit models. However, the reduced model size, and thereby the reduced number of nonlinear expressions that have to be evaluated, is favorable for the computational performance.

3.2. Numerical Continuation for Sample Calculation

Calculations of phase equilibria, especially for LLE, often require considerable computational effort due to iterative solution procedures and in some cases good initial guesses to calculate valid equilibrium compositions. Therefore, fast surrogates are employed for simulation and optimization. The original equilibrium model is only used during sample generation. However, the samples need to cover the entire miscibility gap to enable accurate surrogate modeling and their acquisition can take a considerable amount of time.

The computational cost of sample generation can be reduced by exploiting existing sample data. For instance, in [4], new sample locations are chosen such that they are most likely to improve the surrogate model, thus reducing the overall number of required sample points.

The basic idea followed in the present work is using numerical continuation methods (e.g., [13]) to improve convergence behavior of equilibrium calculations and thereby to reduce the computation time. Continuation methods are used to find solutions of an equation system that depend on a parameter, e.g., solutions of the equilibrium model depending on the feed composition. Varying this parameter, a prediction for the new solution is calculated based on previous solutions and then the prediction is corrected. Since initial guesses are improved using systematic predictions, the correction step converges more quickly to an accurate solution.

In an earlier work [14], the phase stability of an arbitrary point in the phase diagram is tested by tracking valid solutions of the equilibrium model from a known starting composition to the desired location using homotopy continuation. This strategy is adapted here to track solutions of the equilibrium model along the binodal curve by varying the feed composition.

We apply a dynamic method for minimization problems (e.g., [15]) to solve the equilibrium model for a given feed composition. The following ordinary differential equation system

$$\begin{array}{cc}\hfill {\left(N_i^{\mathrm{I}}\right)}^{\prime }& =-k_{\mathrm{s}}(x_i^{\mathrm{I}}{\gamma }_i^{\mathrm{I}}-x_i^{\mathrm{II}}{\gamma }_i^{\mathrm{II}}),i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\left(N_i^{\mathrm{II}}\right)}^{\prime }& =+k_{\mathrm{s}}(x_i^{\mathrm{I}}{\gamma }_i^{\mathrm{I}}-x_i^{\mathrm{II}}{\gamma }_i^{\mathrm{II}}),i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(17)

converges to a valid solution if the initial guess of the phase compositions $x_i^j$ is sufficiently close to that solution. Mole fractions are calculated by $x_i^j=\frac{N_i^j}{{\sum }_{l=1}^{N_{\mathrm{c}}}{N}_l^j}$ and the activity coefficients ${\gamma }_i^j={\gamma }_i^j({\mathit{x}}^j,T)$ are functions of composition and temperature. The scaling factor $k_{\mathrm{s}}:=1\mathrm{mol}$ ensures consistent units on both sides of the equations.

In an effort to track solutions along the binodal curve, the feed composition is varied along a line with a constant phase ratio. This trajectory, depicted in Figure 2, starts from a point on the border of the phase diagram and covers the entire miscibility gap of a ternary mixture. For all feed compositions along the trajectory, corresponding points on the binodal curve are obtained by solving the equilibrium model using Equations (16) and (17). In comparison to the previous approaches in [4,10], this strategy only requires sampling on a line in the miscibility gap instead of sampling in the whole area. If the starting point does not lie on the border of the phase diagram, the line has to be tracked in both directions from the starting point.

An efficient approach to characterize the overall phase behavior of a mixture by evaluating the convex envelope of the Gibbs energy, and thus to identify viable starting points within a miscibility gap, is given in [16].

The continuation method is implemented here by augmenting Equations (16) and (17) with additional expressions that offer convergence to the selected phase ratio and movement of the feed composition perpendicular to the current tie line. Each additional expression is chosen such that the dynamic it introduces is orders of magnitude slower than that of the equilibrium conditions. The resulting stiff ordinary differential equation system is solved using the MATLAB function ode15s [17].

In the case of mixtures with more than three components, the equilibrium compositions form a $(N_{\mathrm{c}}-2)$-dimensional surface instead of a one-dimensional curve. In the present work, the continuation method is adapted to this case by fixing the molar fractions of additional components to constant values and repeat the calculations for all sets of fixed values.

In an alternative approach, instead of varying the feed composition, the compositions of both equilibrium phases could be directly tracked along the binodal curve using continuation methods [18] with Equations (16) and (17) as a possible choice for a robust corrector.

4. Results

This section demonstrates the application of the methods presented in the previous section to the modeling and optimization of a liquid–liquid extraction process.

4.1. Description of the Case Study

A multistage liquid–liquid extraction process adapted from [10] is used here as a case study for the methods presented above. In [10], the optimization of a process for the homogeneously catalyzed hydroformylation of 1-dodecene in a thermomorphic multiphase system is considered. The mixture is homogeneous at reaction conditions and biphasic at lower temperatures. This is used to separate the catalyst from the product in an extraction cascade and to recycle the catalyst back to the reactor. The extraction cascade considered here is highlighted in Figure 3, which shows a modified flowsheet of the overall process from [10].

In the present work, the mixture consists of the polar solvent dimethylformamide, the nonpolar solvent dodecane, the product n-undecanal and the substrate 1-decene. As in [10], the mixture includes a catalyst, which is distributed between the two phases depending on their equilibrium composition. The catalyst concentration is higher in the more polar phase than in the less polar phase. Moreover, it is assumed that the catalyst has no influence on the phase equilibrium due to it having a very low concentration. Thus, the catalyst partition coefficient is modeled at infinite dilution and therefore it is added to the set of surrogate outputs, but not to the inputs. All extraction stages are operated at the same constant temperature $T=298.15\mathrm{K}$, as suggested by the results in [10].

The resulting equation system for a single extraction stage reads

$$\begin{array}{cc}\hfill & {\dot{n}}_i^{\mathrm{in}}={\dot{n}}_i^{\mathrm{I}}+{\dot{n}}_i^{\mathrm{II}},i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & {\dot{n}}_{\mathrm{cat}}^{\mathrm{in}}={\dot{n}}_{\mathrm{cat}}^{\mathrm{I}}+{\dot{n}}_{\mathrm{cat}}^{\mathrm{II}},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill & \frac{{\dot{n}}_i^j}{{\sum }_{l=1}^{N_{\mathrm{c}}}{\dot{n}}_l^j}=f_i^j(t_1,\dots ,t_{N_{\mathrm{c}}-2}),i=1,\dots ,N_{\mathrm{c}},j=\mathrm{I},\mathrm{II},\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill & {log}_{10}\left(\frac{{\dot{n}}_{\mathrm{cat}}^{\mathrm{I}}{\sum }_{l=1}^{N_{\mathrm{c}}}{\dot{n}}_l^{\mathrm{II}}}{{\dot{n}}_{\mathrm{cat}}^{\mathrm{II}}{\sum }_{l=1}^{N_{\mathrm{c}}}{\dot{n}}_l^{\mathrm{I}}}\right)=f_{\mathrm{cat}}(t_1,\dots ,t_{N_{\mathrm{c}}-2}),\hfill \end{array}$$

(21)

with ${\dot{n}}_i^j$ denoting molar flows of component i in phase j and $f_{\mathrm{cat}}$ calculating the logarithmic partition coefficient ${log}_{10}{P}_{12}$ for the catalyst between the less polar phase $\mathrm{I}$ and the more polar phase $\mathrm{II}$.

Table 1. Component labels.

Index i	Name	Description
1	dimethylformamide	polar solvent
2	dodecane	nonpolar solvent
3	n-undecanal	product
4	1-decene	substrate

lists the names of the components corresponding to the index i.

In the model consisting of Equations (18)–(21), $f_i^j$ and $f_{\mathrm{cat}}$ are replaced by a surrogate that comprises both the mole fractions ${\widehat{f}}_i^j\approx f_i^j$ and the logarithmic partition coefficient ${\widehat{f}}_{\mathrm{cat}}\approx f_{\mathrm{cat}}$. With $N_{\mathrm{c}}=4$, the inputs of the surrogate are two parameterization variables $t_1$ and $t_2$. The outputs are eight mole fractions for the equilibrium compositions in both phases as well as the catalyst partition coefficient.

The extraction cascade consists of $k=1,\dots ,N_{\mathrm{s}}$ stages, numbered from $k=1$ for the output stage of the more polar phase $\mathrm{II}$ to $k=N_{\mathrm{s}}$ for the output stage of the less polar phase $\mathrm{I}$. The stage $k=1$ is also the feed stage. Some of the polar solvent is separated using distillation column 1 and recycled to the extraction cascade at stage $k=N_{\mathrm{s}}$. The mass balances that describe the extraction cascade read

$$\begin{array}{cc}\hfill {\dot{n}}_{i,1}^{\mathrm{in}}& ={\dot{n}}_{i,2}^{\mathrm{II}}+{\dot{n}}_i^{\mathrm{feed}},i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill {\dot{n}}_{\mathrm{cat},1}^{\mathrm{in}}& ={\dot{n}}_{\mathrm{cat},2}^{\mathrm{II}}+{\dot{n}}_{\mathrm{cat}}^{\mathrm{feed}},\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill {\dot{n}}_{i,k}^{\mathrm{in}}& ={\dot{n}}_{i,k-1}^{\mathrm{I}}+{\dot{n}}_{i,k+1}^{\mathrm{II}},i=1,\dots ,N_{\mathrm{c}},k=2,\dots ,N_{\mathrm{s}}-1,\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill {\dot{n}}_{\mathrm{cat},k}^{\mathrm{in}}& ={\dot{n}}_{\mathrm{cat},k-1}^{\mathrm{I}}+{\dot{n}}_{\mathrm{cat},k+1}^{\mathrm{II}},k=2,\dots ,N_{\mathrm{s}}-1,\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\dot{n}}_{i,N_{\mathrm{s}}}^{\mathrm{in}}& ={\dot{n}}_{i,N_{\mathrm{s}-1}}^{\mathrm{I}}+{\dot{n}}_i^{\mathrm{col}1},i=1,\dots ,N_{\mathrm{c}},\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\dot{n}}_{\mathrm{cat},N_{\mathrm{s}}}^{\mathrm{in}}& ={\dot{n}}_{\mathrm{cat},N_{\mathrm{s}-1}}^{\mathrm{I}},\hfill \end{array}$$

(27)

where ${\dot{n}}_i^{\mathrm{col}}$ is the distillate molar flow of distillation column 1 that recycles the extraction solvent and ${\dot{n}}_i^{\mathrm{feed}}$ is the feed flow of the extraction cascade for component i.

The mass balances of the distillation column are replaced by the approximation that the distillate of column 1 contains only the polar solvent ${\dot{n}}^{\mathrm{col}}={\dot{n}}_1^{\mathrm{col}}$. The distillate flow is a degree of freedom for this process. A larger distillate flow is able to extract more catalyst, but requires, e.g., more heat for vaporization, a larger distillation column, and larger extraction stages.

The costs considered for this process comprise the catalyst that is retained in the nonpolar product stream, the investment for the decanter vessels, and the operation and investment for distillation column 1. The catalyst costs are proportional to the amount of catalyst that is lost. The investment costs for the distillation column and the decanter vessels are calculated based on [19]. The sizing of the column utilizes the Fenske–Underwood–Gilliland correlations [20,21,22,23] with mean mixture and component properties. A simplified cost function is generated by fitting a polynomial function to data obtained by evaluating the cost calculation for a set of different compositions of the column feed and different values of the distillate flow rate at a distillate purity of 99 % (see [24] for more details). The overall cost J in $ year${}^{-1}$ for this process reads

$$\begin{array}{c}\hfill J={\kappa }_1\sum _{k=1}^{N_{\mathrm{s}}}\sum _{i=1}^{N_{\mathrm{c}}}{\left({\lambda }_i{\dot{n}}_{i,k}^{\mathrm{in}}\right)}^{{\kappa }_2}+{\kappa }_3{\left({\dot{n}}_1^{\mathrm{col}1}\right)}^2+{\kappa }_4{\dot{n}}_1^{\mathrm{col}1}+{\kappa }_5+{\kappa }_6{\dot{n}}_{\mathrm{cat},1}^{\mathrm{I}},\end{array}$$

(28)

with parameters given in

Table 2. Cost function parameters.

Parameter	Value	Unit	Parameter	Value	Unit
${\lambda }_1$	$0.0791506\times {10}^{-3}$	mol${}^{-1}$ s	${\kappa }_1$	504155	$ year${}^{-1}$
${\lambda }_2$	$0.233642\times {10}^{-3}$	mol${}^{-1}$ s	${\kappa }_2$	$0.586667$	1
${\lambda }_3$	$0.210410\times {10}^{-3}$	mol${}^{-1}$ s	${\kappa }_3$	$-328.230$	mol${}^{-2}$ s${}^2$ $ year${}^{-1}$
${\lambda }_4$	$0.195184\times {10}^{-3}$	mol${}^{-1}$ s	${\kappa }_4$	$27175.9$	mol${}^{-1}$ s $ year${}^{-1}$
			${\kappa }_5$	$10439.2$	$ year${}^{-1}$
			${\kappa }_6$	$4.44896\times {10}^{12}$	mol${}^{-1}$ s $ year${}^{-1}$

.

4.2. Data Generation

The method presented in Section 3.2 is used to generate sample data for $f_i^j$. The activity coefficients are calculated utilizing a modified UNIFAC (Dortmund) model [8] with some parameters fit to experimental data as in [10]. Recall that, for four components, one molar fraction is required to be fixed to a constant value for each continuation run. Therefore, a set of binodal curves is calculated for different fixed values of the molar fraction of 1-dodecene in the less polar phase $x_4^{\mathrm{I}}\in \{0,0.01,\dots ,0.25\}$. Each continuation run is initialized at the boundary of the phase diagram with zero n-undecanal, i.e., $x_3^{\mathrm{I}}=x_3^{\mathrm{II}}=0$. The starting point is calculated by solving the dynamic equation system using ode15s in MATLAB with an arbitrary initial guess for the molar fractions of the polar solvent and the nonpolar solvent. Beginning at the calculated starting point, the feed composition is continued along the line of a constant phase ratio of $0.5$. The continuation is stopped when the condition $x_1^{\mathrm{I}}=x_2^{\mathrm{I}}$ is met, i.e., when the amount of polar solvent in the less polar phase is equal to that of the nonpolar solvent. This stopping criterion is chosen here because the extraction of catalyst is ineffective beyond that point due to low values of the partition coefficient. An example of the resulting binodal curves is shown in Figure 4.

The total number of 26 continuation runs produce more than $4\times {10}^4$ samples with an accuracy of ${max}_i\left(\right|x_i^{\mathrm{I}}{\gamma }_i^{\mathrm{I}}-x_i^{\mathrm{II}}{\gamma }_i^{\mathrm{II}}\left|\right)\le 1\times {10}^{-9}$. The calculations require fewer than $2\times {10}^5$ calls to the UNIFAC implementation of the activity coefficient model $({\gamma }_1^j,\dots ,{\gamma }_{N_{\mathrm{c}}}^j)={\mathit{f}}_{\mathrm{UNIFAC}}({\mathit{x}}^j,T,p)$ and a computation time of less than 1 min on a standard desktop computer without utilizing parallelization. For each binodal curve with a fixed value of $x_4^{\mathrm{I}}$, this corresponds to more than 1500 samples and less than 2 s of computation time. Calculating each starting point requires approximately 1000 function calls to the activity coefficient model and each binodal curve requires fewer than 5000 function calls. In other words, the continuation method required fewer than four function calls per sample point. Note that each evaluation of the right-hand-side of Equations (16) and (17) requires two function calls to the activity coefficient model, one for each phase.

The results from $f_i^j$ are used in COSMOtherm [25] to calculate the logarithmic partition coefficient of the catalyst $f_{\mathrm{cat}}$ using the same parameterization and catalyst COSMO input file as in [10]. Continuation is not available for COSMOtherm. Therefore, the logarithmic partition coefficient needs to be calculated separately for each phase composition, which takes a few seconds per sample point. To limit the computational effort, only 51 equally spaced sample points are used for each binodal curve.

The overall dataset used for the surrogate generation comprises $51\times 26=1326$ sample locations on a regular grid for the input variables $(t_1,t_2)\in {[0,1]}^2$. The variables $(t_1,t_2)$ determine the equilibrium compositions of the two phases, i.e., the position on the two-dimensional binodal “curve”. The value of the molar fraction of 1-dodecene in the less polar phase is defined as $x_4^{\mathrm{I}}:=0.25t_2$. At each sample location, the output vector contains nine values, namely the phase compositions and the logarithmic catalyst partition coefficient:

$$\begin{array}{c}\hfill {({\mathit{x}}^{\mathrm{I}},{\mathit{x}}^{\mathrm{II}},{log}_{10}{P}_{12})}^{\top }=\mathit{f}(t_1,t_2),\end{array}$$

(29)

with $\mathit{f}={(f_1^{\mathrm{I}},\dots ,f_{N_{\mathrm{c}}}^{\mathrm{I}},f_1^{\mathrm{II}},\dots ,f_{N_{\mathrm{c}}}^{\mathrm{II}},f_{\mathrm{cat}})}^{\top }$. Because the definition for $t_2$ makes interpolating $x_4^{\mathrm{I}}$ redundant, only eight outputs are used for the surrogate model generation, which corresponds to 1326 × 8 = 10,608 sample values.

4.3. Surrogate Fitting

In the present work, the surrogate model is constructed as a sum of a second order polynomial function and a shallow artificial neural network (ANN) [26], i.e., an ANN with one hidden layer:

$$\begin{array}{c}\hfill \widehat{\mathit{f}}(t_1,t_2)={\widehat{\mathit{f}}}_{\mathrm{poly}}(t_1,t_2)+{\widehat{\mathit{f}}}_{\mathrm{ANN}}(t_1,t_2).\end{array}$$

(30)

The polynomial function captures most of the information contained in the data using only simple expressions. The ANN is employed to further reduce the interpolation error of the surrogate model.

The second-order polynomial function

$$\begin{array}{c}\hfill {\widehat{\mathit{f}}}_{\mathrm{poly}}(t_1,t_2)={\mathit{a}}_{00}+{\mathit{a}}_{10}{t}_1+{\mathit{a}}_{01}{t}_2+{\mathit{a}}_{11}{t}_1{t}_2+{\mathit{a}}_{20}{t}_1^2+{\mathit{a}}_{02}{t}_2^2\end{array}$$

(31)

is fitted to the original data in three steps. First, for each output $f_i$, all parameters $a_{ij}$ of Equation (31) are determined by minimizing the sum of squared errors. Second, all terms with a low contribution to an output value are set to zero for that output value to reduce the number of nonlinear expressions, i.e.,

$$\begin{array}{c}\hfill \mathrm{if}{a}_{ij\ne 00}\le 0.05\underset{kl\ne 00}{max}\left|a_{kl}\right|,\mathrm{then}{a}_{ij}:=0.\end{array}$$

(32)

Third, the remaining parameter values are recalculated by minimizing the sum of squared errors. The polynomial function has a maximum of $6\times 8=48$ nonzero parameters.

Afterwards, the shallow ANN ${\widehat{\mathit{f}}}_{\mathrm{ANN}}$ is fitted to the residual of the data $\mathit{f}(t_{1,k},t_{2,k})-{\widehat{\mathit{f}}}_{\mathrm{poly}}(t_{1,k},t_{2,k})$. The model is fitted using the train command of MATLABs deep learning toolbox with the following options: the hyperbolic tangent as the activation function, the mean squared error as the performance function, data division using interleaved indices, Levenberg–Marquardt backpropagation, the regularization parameter equal to $1\times {10}^{-11}$, and a minimum gradient equal to $1\times {10}^{-16}$. The error weights $w_{\mathrm{e},l}$ for the outputs l are chosen proportionally to the inverse squared data range over all samples $k\in \{1,\dots ,N_{\mathrm{samples}}\}$

$$\begin{array}{cc}\hfill & w_{\mathrm{e},l}\propto \mathrm{range}{\left(f_l\right)}^{-2},l=1,\dots ,N_{\mathrm{outputs}},\hfill \end{array}$$

(33)

with

$$\begin{array}{cc}\hfill \sum _{l=1}^{N_{\mathrm{outputs}}}{w}_{\mathrm{e},l}& =N_{\mathrm{outputs}},\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill \mathrm{range}\left(f_l\right)& =\underset{k}{max}{f}_l(t_{1,k},t_{2,k})-\underset{k}{min}{f}_l(t_{1,k},t_{2,k}),l=1,\dots ,N_{\mathrm{outputs}}.\hfill \end{array}$$

(35)

The remaining options are at default values. The number of neurons in the hidden layer $N_{\mathrm{neurons}}$ is varied from 5 to 15 to show the trade-off between the model accuracy and the number of parameters and nonlinear expressions.

The results of the parameter fitting are shown in

Table 3. Performance metrics for the surrogate model. The sum of squared parameter values of the ANN is denoted by sumParANN${}^2$.

${\mathit{N}}_{\mathbf{neurons}}$	# of Parameters	${\mathit{e}}_{\mathbf{norm}}^{\mathbf{mean}}$	${\mathit{e}}_{\mathbf{norm}}^{\mathbf{max}}$	${\mathit{e}}_{\mathbf{norm},\mathbf{poly}}^{\mathbf{max}}$	sumParANN${}^2$
5	103	0.00202	0.0163	0.1002	644.7
6	114	0.00227	0.0195	0.1002	2311
7	125	0.00127	0.0080	0.1002	667.7
8	136	0.00070	0.0083	0.1002	618.1
9	147	0.00058	0.0087	0.1002	311.6
10	158	0.00039	0.0038	0.1002	369.8
11	169	0.00054	0.0044	0.1002	291.9
12	180	0.00033	0.0034	0.1002	391.5
13	191	0.00030	0.0035	0.1002	357.7
14	202	0.00035	0.0064	0.1002	246.3
15	213	0.00022	0.0020	0.1002	254.9

with the error metrics defined below. A part of the training algorithm relies on random number generation. The variations in model performance caused by this are not accounted for. The ANN training was executed once for each number of neurons $N_{\mathrm{neurons}}$ in the hidden layer.

The mean normalized error $e_{\mathrm{norm}}^{\mathrm{mean}}$ is the algebraic mean, over all outputs $l\in \{1,\dots ,N_{\mathrm{outputs}}\}$ and samples $k\in \{1,\dots ,N_{\mathrm{samples}}\}$, of the absolute error of an output value divided by the range of values for that output, i.e.,

$$\begin{array}{cc}\hfill & e_{\mathrm{norm}}^{\mathrm{mean}}=\frac{1}{N_{\mathrm{samples}}{N}_{\mathrm{outputs}}}\sum _k\sum _l\frac{|{\widehat{f}}_l(t_{1,k},t_{2,k})-f_l(t_{1,k},t_{2,k})|}{\mathrm{range}\left(f_l\right)}.\hfill \end{array}$$

(36)

The maximum normalized error $e_{\mathrm{norm}}^{\max }$ is the maximum error of any output l over all samples k divided by the range of values for that output, i.e.,

$$\begin{array}{c}\hfill e_{\mathrm{norm}}^{\max }=\underset{l}{max}\left(\frac{\underset{k}{max}|{\widehat{f}}_l(t_{1,k},t_{2,k})-f_l(t_{1,k},t_{2,k})|}{\mathrm{range}\left(f_l\right)}\right).\end{array}$$

(37)

The maximum normalized error offers a data-independent estimate of the surrogate quality, e.g., a surrogate with $e_{\mathrm{norm}}^{\max }\ge 0.5$ is not better than a constant function.

The error metric $e_{\mathrm{norm},\mathrm{poly}}^{\max }$ is the same as $e_{\mathrm{norm}}^{\max }$, but with the surrogate comprising only the polynomial function $\widehat{\mathit{f}}={\widehat{\mathit{f}}}_{\mathrm{poly}}$.

The surrogate fitting includes all molar fractions of a phase, although one of the molar fractions could be replaced by the summation condition. However, if the summation condition is used to replace one molar fraction, the worst case error for this quantity is the sum of absolute errors for all other molar fractions. This is increasingly relevant for a larger number of components. Including all molar fractions allows one to define suitable error weights for all quantities.

The results in Table 3 show that the surrogate can reproduce the data to a high degree of accuracy with a relatively low number of parameters. Based on the results for different numbers of neurons in the hidden layer, a surrogate model with $N_{\mathrm{neurons}}=10$ is used for the following surrogate based optimization. The mean error over all sample data and surrogate outputs normalized by the data range for this surrogate model is $0.00039$ and the maximum normalized error is $0.0038$. For 10608 sample values, this is achieved using 158 nonzero parameter values. There is no meaningful distinction between training set and validation set for this ratio of data points to number of parameters. The difference between surrogate model and sample data is also illustrated in Figure 4.

If only a second-order polynomial function is used as a surrogate, the maximum normalized error is $0.1002$.

4.4. Optimization

In this section, the surrogate model is tested for its suitability to be used in process optimization problems. The process model introduced in Section 4.1 is used as a case study. A mixture containing four components plus a catalyst is considered for the liquid–liquid extraction at a fixed temperature and pressure. The aim is to find the optimal extraction solvent recycle such that the costs, comprising energy, investment, and catalyst loss, are minimized. Surrogate models are employed for the liquid–liquid equilibrium in each extraction stage to keep the computational burden of the optimization problem low.

The original phase equilibrium model is replaced by the surrogate model and slack variables $\delta$ that account for regression errors

$$\begin{array}{cc}\hfill & f_i^j(t_1,t_2):={\widehat{f}}_i^j(t_1,t_2)+{\delta }_i^j,i=1,\dots ,N_{\mathrm{c}},j=\mathrm{I},\mathrm{II},\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & f_{\mathrm{cat}}(t_1,t_2):={\widehat{f}}_{\mathrm{cat}}(t_1,t_2)+{\delta }_{\mathrm{cat}},\hfill \end{array}$$

(39)

with the values of the slack variables lying between lower bounds ${\underset{¯}{\delta }}_i^j$ and upper bounds ${\overline{\delta }}_i^j$

$$\begin{array}{cc}\hfill & {\delta }_i^j\in [{\underset{¯}{\delta }}_i^j,{\overline{\delta }}_i^j],i=1,\dots ,N_{\mathrm{c}},j=\mathrm{I},\mathrm{II},\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & {\delta }_{\mathrm{cat}}\in [{\underset{¯}{\delta }}_{\mathrm{cat}},{\overline{\delta }}_{\mathrm{cat}}].\hfill \end{array}$$

(41)

The values of the upper and lower bounds are estimated by the maximum error of the surrogate model at the sample points $k\in \{1,\dots ,N_{\mathrm{samples}}\}$.

$$\begin{array}{cc}\hfill & {\overline{\delta }}_i^j=\underset{k}{max}{f}_i^j(t_{1,k},t_{2,k})-{\widehat{f}}_i^j(t_{1,k},t_{2,k}),i=1,\dots ,N_{\mathrm{c}},j=\mathrm{I},\mathrm{II},\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill & {\underset{¯}{\delta }}_i^j=\underset{k}{min}{f}_i^j(t_{1,k},t_{2,k})-{\widehat{f}}_i^j(t_{1,k},t_{2,k}),i=1,\dots ,N_{\mathrm{c}},j=\mathrm{I},\mathrm{II},\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill & {\overline{\delta }}_{\mathrm{cat}}=\underset{k}{max}{f}_{\mathrm{cat}}(t_{1,k},t_{2,k})-{\widehat{f}}_{\mathrm{cat}}(t_{1,k},t_{2,k}),\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill & {\underset{¯}{\delta }}_{\mathrm{cat}}=\underset{k}{min}{f}_{\mathrm{cat}}(t_{1,k},t_{2,k})-{\widehat{f}}_{\mathrm{cat}}(t_{1,k},t_{2,k}).\hfill \end{array}$$

(45)

The optimization problem reads

$$\begin{array}{cc}\hfill \mathrm{minimize}& J\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& \mathrm{Equations}\left(18\right)–\left(28\right),\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill & \mathrm{Equations}\left(38\right)–\left(39\right),\hfill \end{array}$$

(48)

with domains for all variables set to appropriate values and additional redundant constraints that enable tighter bounds for deterministic global optimization. The hyperbolic tangent is formulated as $tanh\left(x\right)=1-\frac{2}{1+exp\left(2x\right)}$ as suggested in [26]. The feed stream ${\dot{n}}_i^{\mathrm{feed}}$ is given in

Table 4. Feed stream parameters.

Parameter	Value	Unit
${\dot{n}}_1^{\mathrm{feed}}$	18.8	mol s${}^{-1}$
${\dot{n}}_2^{\mathrm{feed}}$	16.5	mol s${}^{-1}$
${\dot{n}}_3^{\mathrm{feed}}$	0.563	mol s${}^{-1}$
${\dot{n}}_4^{\mathrm{feed}}$	2.25	mol s${}^{-1}$
${\dot{n}}_{\mathrm{cat}}^{\mathrm{feed}}$	$1\times {10}^{-6}$	mol s${}^{-1}$

. The number of stages is fixed to values between $N_{\mathrm{s}}=1$ and $N_{\mathrm{s}}=6$.

The nonlinear programming (NLP) problem is implemented in GAMS 26.1.0 [27] using the deterministic global solver BARON 18.11.12 [28] with CONOPT 4.09 [29,30] as the nonlinear programming subsolver and CPLEX 12.8.0 [31] as the mixed-integer programming subsolver, and it is solved on the same standard desktop computer as used in Section 4.2. All options are at default values except for the relative optimality criterion, which is set to $1\times {10}^{-3}$. The optimization is repeated for different numbers of stages $N_{\mathrm{s}}$. The computation time for global optimization is the total CPU time used as reported by BARON. Performance variability [32] is not accounted for: Each optimization problem is solved once. The time and the iterations required to find a local solution are determined by solving each problem with CONOPT and using the total time elapsed as reported by GAMS. Default values chosen by GAMS are used as initial guesses for all variables. Although this initial point is always infeasible, the selected solvers found feasible solutions in all observed cases.

The results in

Table 5. Optimization results.

	Objective	Solvent Recycle	Catalyst Retention	Iterations	Solve Time	Solve Time
${\mathit{N}}_{\mathbf{s}}$	J in $ year${}^{-1}$	${\dot{\mathit{n}}}_1^{\mathbf{col}}$ in mol s${}^{-1}$	${\dot{\mathit{n}}}_{\mathbf{cat},1}^{\mathbf{II}}/{\dot{\mathit{n}}}_{\mathbf{cat}}^{\mathbf{feed}}$	(CONOPT)	(CONOPT)	(BARON)
1	541,760	2.3524	0.9002	89	0.035 s	0.38 s
2	305,686	4.8854	0.9725	191	0.066 s	3.74 s
3	249,418	4.1435	0.9859	330	0.122 s	8.39 s
4	236,250	3.6599	0.9910	398	0.175 s	28.71 s
5	238,577	3.3459	0.9936	464	0.285 s	56.33 s
6	248,022	3.1299	0.9950	608	0.405 s	664.44 s

show that the cost of the process falls steeply until a minimum is reached at $N_{\mathrm{s}}=4$. The cost increases slightly for $N_{\mathrm{s}}>4$. This behavior is due to large solvent recycles being required for good catalyst retention at low numbers of stages and the relatively small costs of increasing the number of stages. The overall optimum at $N_{\mathrm{s}}=4$ resembles a trade-off between the cost for catalyst loss and solvent recycle on one hand and the investment cost for additional stages on the other hand. The solvent recycle for $N_{\mathrm{s}}=1$ is an exception. The feed of the single stage already contains a large amount of the extraction solvent. Increasing the solvent amount further has only a marginal effect on catalyst retention, which does not compensate for the additional cost of solvent recycle at some point.

The surrogate model implementation is significantly faster than the original model considering that we observed computation times of a few seconds for a single COSMOtherm evaluation. The computation times using BARON show that the surrogate model formulation is also suitable for small to medium sized deterministic global optimization problems. CONOPT was able to find a global solution for each of the considered cases. BARON requires the computation time to prove globality by converging the lower bound.

5. Conclusions

In this work, we introduce a new surrogate formulation that uses a parameterization of the binodal curve to reduce the input dimension and simplify the input domain. We also propose using numerical continuation methods for sampling of LLE data based on advanced thermodynamic models. The presented methods reduce the time required for sample data generation and allow for highly accurate reproduction of the data using smaller surrogate models.

Parameterization of complex input domains and sampling by numerical continuation are general strategies that may be extended to other applications in future work. Another possible improvement is replacing the activity coefficient model currently used for LLE calculations with more predictive equation of state models. The computation time of global optimization may be further reduced by employing reduced-space formulations of the optimization problem as in [26,33,34].