Rigorous Connectionist Models to Predict Carbon Dioxide Solubility in Various Ionic Liquids

: Estimating the solubility of carbon dioxide in ionic liquids, using reliable models, is of paramount importance from both environmental and economic points of view. In this regard, the current research aims at evaluating the performance of two data ‐ driven techniques, namely multilayer perceptron (MLP) and gene expression programming (GEP), for predicting the solubility of carbon dioxide (CO 2 ) in ionic liquids (ILs) as the function of pressure, temperature, and four thermodynamical parameters of the ionic liquid. To develop the above techniques, 744 experimental data points derived from the literature including 13 ILs were used (80% of the points for training and 20% for validation). Two backpropagation ‐ based methods, namely Levenberg–Marquardt (LM) and Bayesian Regularization (BR), were applied to optimize the MLP algorithm. Various statistical and graphical assessments were applied to check the credibility of the developed techniques. The results were then compared with those calculated using Peng–Robinson (PR) or Soave–Redlich–Kwong (SRK) equations of state (EoS). The highest coefficient of determination (R 2 = 0.9965) and the lowest root mean square error (RMSE = 0.0116) were recorded for the MLP ‐ LMA model on the full dataset (with a negligible difference to the MLP ‐ BR model). The comparison of results from this model with the vastly applied thermodynamic equation of state models revealed slightly better performance, but the EoS approaches also performed well with R 2 from 0.984 up to 0.996. Lastly, the newly established correlation based on the GEP model exhibited very satisfactory results with overall values of R2 = 0.9896 and RMSE = 0.0201.


Introduction
The natural gas produced from the subterranean gas fields and subsequently transported through pipelines should meet certain specifications such as environmental and safety standards as well as those of sale gas sectors. The products destined for sale should be free of undesirable contaminants, e.g., carbon dioxide (CO2) and hydrogen sulfide (H2S), which are both toxic and unfriendly from an environmental point of view. For instance, CO2 is considered the main contributor to global warming and climate change. Several treatments are employed to remove acidic gases from natural gas. The most famous are the alkanolamine-based treatment operations [1][2][3]. This technique was firstly introduced for carbon dioxide removal in 1991 [4,5]. The used alkanolamines are organic compounds, such as monoethanolamine (MEA), diethanolamine (DEA), and triethanolamine (TEA). By far, MEA was the most preferred alkanolamine compared to DEA and TEA because of its reactivity, low molecular weight, and lower required circulation to maintain a given amine to acid-gas mole ratio.
The gas-liquid absorption in amine-based solvents is an efficient process in gas sweetening. Nevertheless, some imperfections have been observed, such as the creation of corrosive byproducts due to the amine degradation, water transfer to the gas stream during the desorption stage, and loss of the feedstock (amine), making the treatment operations expensive [6][7][8][9][10]. As an alternative, a new class of non-aqueous and environmentally friendly innovating fluids, known as ionic liquids (ILs), has emerged. ILs have many industrial applications such as catalysis for clean technology [11] and the removal of contaminants from refinery feedstock [12]. Furthermore, Ion Engineering Company is intended to use the know-how of ionic liquids for industrial-scale sweetening of natural gas and flue gas CO2 separation [13,14], as stated by Hasib-ur-Rahman et al. [15].
Ionic liquids are molten salts, which are liquid (non-volatile) at room temperature. They are comprised exclusively of positively and negatively charged ions. Due to their bulky and asymmetrical cation structure, ILs have a low affinity to constitute crystals [16]. Manipulation of the cation and/or anion allows designing ILs adaptable to any particular application requirements [17]. Moreover, ILs are a perfect medium for acid gas solubilization over wide ranges of temperature and pressure. Thus, great attention was paid to evaluate the performance of ILs as a gas-cleaning agent in gas refinery plants [10]. The only and the most discussed disadvantage related to the use of ILs is their high viscosity, but it can be bypassed as the viscosity can be regulated over a reasonable range of about <50 cP to >10,000 cP by selecting an adequate mixture of cation and anion [15].
In the past few years, many experimental studies were conducted to estimate the solubility of acid gases in ILs [1,6,8,10,18,19], especially the carbon dioxide solubility [20][21][22][23][24]. The obtained results confirmed the ILs to be very efficient in carbon dioxide removal. Unfortunately, the experimental studies require many laboratory tests, which are expensive, difficult, tedious, and time-consuming. As an alternative strategy, the solubility of acid gases in ILs has been modeled using thermodynamical laws and the equation of state (EoS). The thermodynamic laws used for modeling the solubility of acid gases in ILs can be divided into four groups that include cubic equations, quantum mechanics-based methods, activity coefficient methods, and statistical mechanics-based molecular approaches [25]. The most used models are the Peng-Robinson equation of state (EoS), the generic Van der Waals EoS, the generalized Redlich-Kwong cubic EoS, the law and extended law of Henry, and the equation of Krichevsky-Kasarnovsky [8,[26][27][28][29][30][31][32]. It was noticed that the models describe well the systems at low and moderate pressures [33], however, the equations of state suffer from many weaknesses. The equation of state can be reliable only for an individual system and not for more interestingly, multiple systems. They require various adjustable parameters, which should be optimized based on real data within a particular and limited range of thermodynamic conditions. Consequently, developing more general and powerful models to predict the solubility of acid gases, especially carbon dioxide, in ILs is of paramount importance.
Recently, many soft computing methods have been applied to model gas solubility and phase equilibrium. One of these methods is the artificial neural network (ANN), which represents an important embranchment computational intelligence method that can be used without any preassumption of the input-output relationship [34,35]. Multilayer perceptron (MLP), radial basis function network (RBF), multi-layer feed-forward network, and gene expression programming (GEP) are the general categories of ANN. Fuzzy logic (FL) is also one of the computational intelligence methods, which can model complicated nonlinear relations [36]. Likewise, the support vector machine (SVM) proposed by Vapnik was shown to be a very performant smart model [37]. Several researchers have used these smart models in the petroleum industry and to predict the solubility of acid gases in ILs. Baghban et al. [36] have predicted CO2 solubility in the presence of various ILs using MLP and Adaptive Neuro-Fuzzy Inference System (ANFIS). Amedi et al. [38] have used MLP, RBF, and ANFIS to predict H2S solubility in the presence of various ILs. Otherwise, in 2017, Rostami et al. [39] have applied the GEP method to model CO2 solubility in crude oil during carbon dioxide enhanced oil recovery.
The current work aims at developing highly robust and easy-to-use machine learning models that can be applied for forecasting the solubility of CO2 in 13 different ionic liquids at different temperature and pressure conditions. Two rigorous connectionist techniques, namely multilayer perceptron (MLP), and gene expression programming (GEP) are applied on a set of experimental data that was gathered from different literature sources [40][41][42][43][44]. The MLP method was optimized using either Levenberg-Marquardt (LMA) or Bayesian Regularization (BR) techniques. The results obtained using three methods (MLP-LMA, MLP-BR, GEP) are then compared with results calculated using thermodynamic models based on Peng-Robinson (PR) and Soave-Redlich-Kwong (SRK) equation of states. Statistical indicators, including the determination coefficient (R 2 ) and Root Mean Square Error (RMSE), are used to evaluate the accuracy of the methods, in addition to graphical assessments using cross plots and bar plots. In the end, outliers detection is performed to test and analyze the validity of the best-developed model and quantify the doubtful experimental points from the database. It is worth noting that the two backpropagation-based learning algorithms (LMA and BR) that were employed in the training process of MLP, alongside the explicit correlations established to predict the CO2 solubility in ILs, make the current work different from previously published works in the literature. This paper is constructed as follows; Section 2 depicts the data used in the study and the input and output parameters in the models. Section 3 describes in detail the rigorous connectionist models and optimization techniques. An overview of the PR and SRK equations of state is presented in Section 4. In Section 5, the results are presented and discussed, and in Section 6, the conclusions of the study are summarized.

Data Collection and Preparation
In order to develop reliable models, it was crucial to use a large number of experimental data points representing a variety of conditions. In the current work, 744 experimental data points for carbon dioxide solubility in 13 different ILs (36 to 80 points for each IL) were collected from the literature [40][41][42][43][44]. Table 1 presents the references from where the data were gathered and the observed range of experimental measurements. The dissolved mole fraction (termed ) of CO2 in each IL was the parameter we wished to match and predict based on the corresponding input of temperature T, pressure P, and the given IL. To represent the IL, we specified the thermodynamic properties of the IL, including molecular weight (Mw), critical temperature (Tc), critical pressure (Pc), and acentric factor (w), as summarized in Table 2. These 6 input parameters were hence used for predicting the dissolved mole fraction of CO2. All the 744 experimental data points were randomly divided into a training data set (80% of the database) and a test data set (20% of the database).

Multilayer Perceptron (MLP)
Artificial neural network (ANN) is one of the well-known machine learning tools, which exhibits reliable ability for recognizing and identifying relationships between input and output parameters in complex systems [45]. The conception and the mathematical formulation of an ANN model were inspired by the human brain and its strategy for processing information. Multilayer perceptron (MLP) is one of the most applied types of ANN for modeling purposes.
An MLP model includes three kinds of layers, an input layer: From which the inputs enter into the model. The th input parameter of the model is noted .
One or more hidden layers: Their role is to transform the inputs into a higher feature in order to capture the non-linearity of the system. This mechanism is done by means of the activation functions. Tansig and logsig are among the frequently applied transfer functions and they are defined as follows: Logsig: (1) Tansig: An output layer: The outputs of the model are returned from this layer. Pureline is generally considered the transfer function for this layer. Pureline: ( If the total number of hidden layers is considered to be , the input layer is defined as layer 0. Suppose that the number of neurons in the hidden layer is , 1, 2, … , . Considered the weight of the linkage between the th neuron of the 1 th hidden layer and th neuron of the th hidden layer, and the bias term of ith neuron of the th hidden layer. Hence, the output ( ) of th neuron of the th hidden layer can be determined using the following equation: . (4) Each of the above-mentioned layers includes neurons. The number of neurons in the input and output layers corresponds to the number of input and output parameters of the system, respectively. The number of hidden layers and their neurons depends on the complexity of the system. Generally, one hidden layer is sufficient to model systems having moderate complexity, while more than one hidden layer is recommended for highly complex systems. The trial and error method is usually applied for the proper determination of the number of hidden layers and their numbers of neurons.
The training phase of an MLP model consists of determining appropriate values for the weights and bias terms that result in the minimization of the difference between the experimental data (measured dissolved CO2 mole fraction) and the predictions of the model (calculated CO2 mole fraction as a function of the input parameters). Backpropagation learning techniques were applied for the training of the MLP model, namely the Bayesian Regularization (BR) and Levenberg-Marquardt (LM) algorithms. More details about these two algorithms can be found in prior published works [46][47][48].

Gene Expression Programming (GEP)
In spite of computer-based artificial intelligent models, based genetic calculations, such as genetic programming (GP) can construct formulas, which have high precision. It has been demonstrated that GP can be more accurate than traditional empirical models. Furthermore, models developed by genetic calculations and simulators and software systems are easy to use [49].
Using correction and development of the GP method [50] led to a more relevant version, which was GEP [51]. Indeed, the latter removed the found weaknesses in GP, i.e., the big number of possible programs that can be built by the algorithm and the high sensitivity to small changes in fitness of the created solution. In general, the GEP [51] had 2 significant parts: The chromosome and the symbolic expression trees (ETs). The chromosome encoded the potential solutions, and after that, these solutions were converted to the particular applicant solution named ET [52][53][54]. Reliable ETs continuously were reproduced by unlimited genetic conversions because of the conversion method of the structural organization of the chromosome to the ET [52]. In fact, the unique genes' placements in the GEP method produced impressive development of the solutions leading to encryption of any downside [55]. Preceding researches recognized that the speed of convergence in the GEP scheme was much more than the GP scheme [53,54].
The chromosome included unchanging factors and variables as terminals and predetermined functions with equal length in one or more genes [52]. By the user, the constants were produced by the GEP in a specific range, however, the functions and variables were the input information. Each gene consisted of a tail, which comprised terminals and also a head that contained constants and variables in the role of terminals and functions [52]. The given value of the gene's head is taken into account as an input of the GEP method, and the value of gene's tail (t) is given by the following equation: t = h(r − 1) + 1 (6) with t denotes the length of the tail, h is the length of the head, and r is the number of arguments of the function with the most arguments. The essential stages of the GEP method are illustrated in Figure 1. These steps can be elucidated briefly as follows: Firstly, a generation of chromosomes of an initial population was created randomly. In a second step, the chromosomes were expressed with ET, and their fitness was computed. Then, the individuals were selected to perform reproduction based on their fitness values. This reproduction created offspring with fresh signs. The same procedure was followed by individuals of the resulted generation, i.e., genomes expression, selection, and reproduction with adjustment. The stages expressed above were repeated until reaching the stopping criteria. More explanation of the GEP technique can be found in the literature [51,55]. In the current work, the mathematical expression used to predict the CO2 solubility in ILs was generated in the following form: 2 , , , , , .

Techniques Evaluation
Statistical indicators alongside the graphical techniques were used to assess the performance and robustness of the proposed models. The statistical indicators include root mean square error (RMSE) and the coefficient of determination (R 2 ), which is defined mathematically as follows: The credibility of the developed models was tested using the leverage statistical approach. In this approach, the standardized residuals, which refer to the difference between the predicted results and the experimental points, were presented versus the statistical Hat matrix leverage values; this graph is called the Williams plot. The Hat can be calculated using the Hat matrix of the following form [56,57]: (9) where X is an (k × d) matrix, k and d represent the dimension and the data points number, respectively, and X t denotes the X transpose matrix. The limit leverage value (H*) depicted on the Williams plot with a purple line is a constant calculated as 3(d + 1)/k. The selection of data points is in the range of ± 3 of the standard deviation from the mean, where the cut-off value of 3 ensures covering 99% of normally distributed data. The validity of the developed model and the credibility of the predictions were conditioned by the values of data points, which must be situated in the range of 0 ≤ H ≤ H* and −3 ≤ R ≤ 3.

Equations of State for Modeling the Solubility of Acid Gases
Many equations of state have been used by researchers to model the solubility of acid gases in ILs. Two of the most widely used EoS are the Soave-Redlich-Kwong (SRK) and the Peng-Robinson (PR) EoS, which are defined with Equations (10) and (15), respectively [35]. The results of these two EoS are used to contrast those obtained by using the proposed models.
Noting that the calculation of solubility using EoS is related to the calculation of the mole fraction, the equations below express how to determine the mole fraction.
Soave-Redlich-Kwong (SRK) EoS: (10) where T, P, v, and R indicate the temperature, pressure, molar volume, and gas constant respectively, and a and b represent the EoS variables. For computing the variables, a and b in the case of mixtures, the classic van der Waals one-fluid rules of mixing is used [58]: with and denote the mole fractions of components i and j, and N indicates the number of the mixture's components. The parameter is a parameter of binary interaction, that enlarges the molecular interactions between molecules i and j. For pure materials, the parameters can be computed as follows: with , denotes the reduced temperature. We note that the above description is valid for a system of arbitrarily many components. In our system, it was assumed that we had two components where CO2 was one and the IL the other. Peng-Robinson (PR) EoS: For mixtures, the parameters a and b are determined using the same formulas as in SRK EoS. The parameter of binary interaction ( ) is the only adjustable parameter for both EoS, SRK, and PR. This parameter is determined using a genetic algorithm (GA) with the next objective function [58]: (18) with and indicate the experimental and EoS-calculated mole fractions of the solute respectively.

Computational Procedure
For the training phase, the mean square error (MSE) was used as the assessment criterion, which is defined mathematically as follows: where stands for mole fraction of CO2, and indicate the experimental and the predicted values, respectively, and represents the number of samples. The model tuning parameters giving the lowest MSE on the training set were considered the choice for the trained model.
For the modeling task using MLP, the data points were normalized between −1 and 1. To select appropriate topologies for the MLP approach, trial and error were used. The obtained models were designated MLP-LMA, and MLP-BR, respectively, and both included 3 hidden layers with 11, 11, and 9 neurons, respectively. The suitable activation functions in all the hidden layers and for the output layer were Tansig and Pureline, respectively.

Results and Discussion
To obtain an accurate and trustworthy expression, the calculation procedure reported in Section 3.2 has been followed. Thus, taking into account the series of five independent parameters , , , , , the GEP-based model makes the mathematical based correlation for the dependent parameter 2 as follows: where P (MPa) and (MPa) are pressure and critical pressure, respectively, (Kg/Kmole) is the molecular weight, (K) indicates the critical temperature and , , …… and are expressed as below:  Table 3 summarizes the statistical parameters (RMSE and R 2 ) of the three proposed models in predicting CO2 solubility for training, test, and overall data. The graphical representation of the overall RMSE and R 2 through bar plots, as shown in Figure 2. This graphical comparison allows an alternative visualization for the performance of the different models. A concise view of the figure shows that the values of RMSE and R 2 were closest to 0 and 1, respectively, pointing out that the models were more credible and qualified for the forecast.  Table 3 and Figure 2 show that the reliability of the developed models took the following order: MLP-LMA > MLP-BR > GEP. The MLP model optimized using LMA had the highest R 2 coefficients and the lowest RMSE values for both the training and overall data, followed by the MLP-BR model. On the other hand, very satisfactory performances and high predictions potential of the GEP model were asserted by the values reported in Table 3 for training, test, and overall data.

R²
Cross plots in Figure 3 were used to compare the models' predictions against the experimental data further. In these plots, the predicted values using the three methods were represented versus the previously established experimental data for the solubility of CO2 in ILs. The criterion of precision for the different models in typical cross plots can be indicated by the compressed gathering of data points cloud nearby the 45° line, otherwise, all points falling on the unit slope would correspond to a perfect model. The accumulation of data cloud nearby the 45° line was attributed to the elevated value of R 2 close to the unit and to the low value of RMSE near zero.

Comparison with PR and SRK EoS Results
The results obtained by the MLP-LMA model were compared with values calculated using thermodynamic models based on PR and SRK EoS. Tables 4 and 5 provide the CO2 solubility and thermodynamic properties respectively. Furthermore, Table 6 summarizes a comparison, where the RMSEs and R 2 of the best-proposed model, i.e., MLP-LMA, and PR and SRK EoS were mentioned. To perform this comparison, two samples of ILs that had not been included in the training phase of the intelligent models were considered. These samples included    MLP-LMA model, and at the same time, the evolution of CO2 solubility with pressure in the two ILs at a constant temperature. As it is clear in the figures, the solubility of CO2 increased with the increase in pressure for the two investigated ILs. Henry's law can explain this phenomenon, which supposes that the gas solubility throughout a liquid is directly a function of the gas pressure above the solution's surface [28]. The increase in pressure forces gas molecules into the solution, thus the number of gas molecules dissolved raises. It can also be seen from the figures that the solubility values predicted by the MLP-LMA model are in good agreement with the experimental data, whereas those established using the EoS differs slightly.

Outlier Detection
A Williams plot of the MLP-LMA model is depicted in Figure 6. It can be seen from the figure that the majority of data points are situated in the range of −3 ≤ R ≤ 3 and 0 ≤ H ≤ 0.03, which validated and verified the implemented MLP-LMA model statistically, where 13 points are considered suspect. In summary, the leverage approach states that the developed model is reliable and can be used with high accuracy to predict CO2 solubility in ILs.

Conclusions
In the current paper, based on different experimental data gathered from the literature, MLP-LMA, GEP, PR, and SRK EoS methods were successfully used to predict carbon dioxide solubility in 13 various ionic liquids under the (298.15-373.15 K) temperature and (0.01-100.12 MPa) pressure range. Multilayer perceptron optimized with two back-propagation algorithms, viz. LMA and BR, and a reliable whitebox technique, namely GEP, were the applied data-driven methods. The various statistical indicators obtained in this study showed that MLP optimized with LMA was the most advantageous paradigm, which can be used with high precision to predict the CO2 solubility in ILs for natural gas sweetening and purification process, which may be of great interest for experts and gas engineers. Furthermore, the established correlation using the GEP-based model was reliable and gave good results. Finally, the comparison made between the MLP-LMA model and PR and SRK EoS demonstrated the high performance of the proposed model against the cubic EoS.

Conflicts of Interest:
The authors declare no conflict of interest.