Developing a Hybrid Neuro-Fuzzy Method to Predict Carbon Dioxide (CO2) Permeability in Mixed Matrix Membranes Containing SAPO-34 Zeolite

This study compares the predictive performance of different classes of adaptive neuro-fuzzy inference systems (ANFIS) in predicting the permeability of carbon dioxide (CO2) in mixed matrix membrane (MMM) containing the SAPO-34 zeolite. The hybrid neuro-fuzzy technique uses the MMM chemistry, pressure, and temperature to estimate CO2 permeability. Indeed, grid partitioning (GP), fuzzy C-means (FCM), and subtractive clustering (SC) strategies are used to divide the input space of ANFIS. Statistical analyses compare the performance of these strategies, and the spider graph technique selects the best one. As a result of the prediction of more than 100 experimental samples, the ANFIS with the subtractive clustering method shows better accuracy than the other classes. The hybrid optimization algorithm and cluster radius = 0.55 are the best hyperparameters of this ANFIS model. This neuro-fuzzy model predicts the experimental database with an absolute average relative deviation (AARD) of less than 3% and a correlation of determination higher than 0.995. Such an intelligent model is not only straightforward but also helps to find the best MMM chemistry and operating conditions to maximize CO2 separation.


Introduction
Environmental pollution [1], global warming [2], and climate change [3], are likely the most interconnected problematic issues in recent decades. Carbon dioxide (CO 2 ) as a greenhouse gas is undeniably a key factor in creating these undesirable environmental phenomena [4]. Therefore, separating and recycling CO 2 from the gaseous streams released to the atmosphere seems essential. In addition, separating the CO 2 impurity of the natural gas is significant to improve the heating value and avoid corrosion in transmission pipelines and processing equipment. Absorption [5], adsorption [6,7], cryogenic process [8,9], and membrane-based [10] operations are conventionally applied to separate CO 2 molecules from either waste or raw gaseous stream. In addition, the recovered carbon dioxide can be then used for the separation of heavy metal pollutants [11].
The membrane-based technologies, which are low cost, highly flexible, easy to operate, energy saving, and environmentally friendly [12], have seen a growing interest in the field of CO 2 capture. Due to their suitable mechanical characteristics, the polymeric membranes work better in high-pressure and temperature situations. Researchers have continuously attempted to fabricate materials that fulfill the trade-off between gas selectivity and permeability and overcome the "Robeson upper bond" curve in polymeric membranes. To do so, attention has been concentrated on the inclusion of inorganic solid materials in the polymer membrane structure [13].
The mixed matrix membranes (MMM), which combine a polymer as the continuous phase and inorganic filler as the dispersed phase, have been suggested to resolve these limitations [14]. This special class of membranes shows better gas separation performance than both polymer and inorganic fillers [15]. Activated carbon [16], nano-fibers [17], nanoparticles [18], metal- [19] and [20] covalent organic frameworks, and zeolites [21], have been successfully dispersed in polymers' structure to fabricate MMMs.
Due to high pore volume, uniform pore size distribution, and appropriate chemical/thermal stability, zeolites [22] and their inclusion in a matrix [23] are often used for the CO 2 separation. The SAPO-34 zeolite, with excellent CO 2 adsorption affinity, has achieved great popularity in synthesizing mixed matrix membranes [23]. These investigations have mainly measured the CO 2 permeability in mixed matrix membranes experimentally. The permeability measurements have been considered as a function of polymer type, SAPO-34 dosage, pressure, and temperature.
Despite a relatively extensive experimental analysis of CO 2 permeability in MMMs, there is no systematic strategy to predict this key variable. Therefore, the current research uses a hybrid neuro-fuzzy modeling approach to predict CO 2 permeability in MMMs containing SAPO-34 as a function of polymer type, SAPO-34 dosage, pressure, and temperature. A hybrid optimization algorithm and sensitivity analysis determine this model's adjustable coefficient and its hyperparameters, respectively. The proposed model in this study can help construct the MMM-based CO 2 removal facility by providing accurate predictions for CO 2 permeability as a function of MMM chemistry and operating conditions.

Literature Data
Different MMMs have already been fabricated by combining SAPO-34 zeolite and Matrimid-5218 [24], Pebax-1657 [25], polyethersulfone [26,27], polyurethane [28], polysulfone [29], and Pebax-1074 [30] polymers for the purpose of CO 2 removal. The literature measured the CO 2 permeability in these MMMs as a function of SAPO-34 dosage, temperature, and pressure. Table 1 presents numerical values of the statistical information (minimum, maximum, average, and standard deviation) of these experimental measurements, and Figure 1 illustrates their graphical version using the box plot. It can be seen that the collected experimental databank covers SAPO-34 dosage, temperature, and pressure ranges of 0-50 wt.%, 267-394 K, and 0.1-3 MPa. In this condition, the CO 2 permeability varies from 0.2-337 barrer. In addition to these statistical and graphical presentations of the experimental measurements, the Supplementary Material reports all their information. In addition to these statistical and graphical presentations of the experimental measurements, the supplementary material reports all their information.

ANFIS Structure
The function approximation [31] and classification [32] are the problems that can be easily handled by the machine learning tools. Therefore, this study aims to efficiently tune the adaptive neuro-fuzzy inference system topology to enhance the prediction accuracy of CO2 permeability in different MMMs in a broad range of operating conditions. Consequently, it seems necessary to briefly describe the working procedure of this intelligent estimation machine.
The ANFIS is a powerful modeling tool that simultaneously utilizes the inference property of fuzzy logic and the learning capabilities of artificial neural networks. Consider a multiple-input and single-output (MISO) regression problem, such as F (X, Y). A Takagi-Sugeno system with five interconnected successive layers (i.e., fuzzification, interference, normalization, interpolation, and target computation) and the following fuzzy ifthen rules, namely Equations (1) and (2) [33], can be constructed to model this problem.
Rule 1: If X is A 1 and Y is B 1 , then F 1 = C 1 × X + D 1 × Y + E 1 (1) Rule 2: If X is A 2 and Y is B 2 , then F 2 = C 2 × X + D 2 × Y + E 2 Here, A 1 , A 2 , B 1 , and B 2 are the premise parameters. On the other hand, the adjustable consequence parameters are shown by C 1 , C 2 , D 1 , D 2 , E 1 , and E 2 .
The first layer assigns a membership function (η ) to each node j and calculates the output signals (

ANFIS Structure
The function approximation [31] and classification [32] are the problems that can be easily handled by the machine learning tools. Therefore, this study aims to efficiently tune the adaptive neuro-fuzzy inference system topology to enhance the prediction accuracy of CO 2 permeability in different MMMs in a broad range of operating conditions. Consequently, it seems necessary to briefly describe the working procedure of this intelligent estimation machine.
The ANFIS is a powerful modeling tool that simultaneously utilizes the inference property of fuzzy logic and the learning capabilities of artificial neural networks. Consider a multiple-input and single-output (MISO) regression problem, such as F (X, Y). A Takagi-Sugeno system with five interconnected successive layers (i.e., fuzzification, interference, normalization, interpolation, and target computation) and the following fuzzy if-then rules, namely Equations (1) and (2) [33], can be constructed to model this problem.
Here, A 1 , A 2 , B 1 , and B 2 are the premise parameters. On the other hand, the adjustable consequence parameters are shown by C 1 , C 2 , D 1 , D 2 , E 1 , and E 2 .
The first layer assigns a membership function (η) to each node j and calculates the output signals (O 1 j ) based on Equations (3) and (4) [34].
The output of the second layer (O 2 j ) is the product of all the incoming signals [35].
The outputs of the third layer (O 3 j ) compute by normalizing the entry signals, as in Equation (6) [36].
Finally, the ANFIS prediction for the target (O ANFIS ) can be achieved by applying Equation (8) [38].

Input Space Partitioning Strategies
The literature declares that the fuzzification strategy (the first layer of ANFIS) significantly impacts the accuracy of simulating a given problem [39]. Generally, there are three well-known methods to divide the input space, i.e., grid partitioning (GP), fuzzy C-means (FCM), and subtractive clustering (SC). Although Yeom and Kwak comprehensively described GP, FCM, and SC and highlighted their weaknesses and strengths [33], the following subsections briefly explain these partitioning strategies.

Grid Partitioning
Grid partitioning is a method that breaks down input space into a non-overlapping grid-like structure. This method is suitable for a problem with a small number of independent variables in low-dimensional input space. For instance, 1024 unique areas and a rule for each area (i.e., 1024 rules) have been provided by the GP method with two membership functions for a problem with 10 independent variables [33]. Hence, the GP is a complicated strategy to divide the input space of a high-dimensional problem (high number of the independent variable of observations).

Fuzzy c-Means Clustering
The FCM separates the input space and places them in different fuzzy clusters with a specific radius. An optimization method that minimizes the observed non-similarity determines these properties. The user specifies the number of fuzzy rules by setting the number of clusters.

Subtractive Clustering
The SC is a scenario to break down the input space into several specific clusters by performing a multi-dimensional analysis. This strategy automatically divides the input space into an appropriate number of clusters utilizing a user-entered value for the cluster radius. The SC generates a high number of small-sized clusters when a small cluster radius is entered, and vice versa.

Training Algorithm
Traditionally the backpropagation or hybrid training algorithm is applied to determine adjustable parameters of membership functions during the training step of the ANFIS. The backpropagation method continuously updates the adjustable parameters of the ANFIS to minimize the deviation between actual and predicted values of a dependent variable. Indeed, the observed deviation propagates backward through the ANFIS structure, and its parameters adjust by several iterations. On the other hand, the hybrid method combines the least-squares and backpropagation algorithms to tune ANFIS parameters efficiently. To enhance the performance of the training phase, as well as to avoid errors related to the different ranges of variables, the experimental data are needed to normalize between 0 and 1 [40,41].

Results and Discussion
The most appropriate structural features for three well-known ANFIS types have been determined in this section. Four statistical uncertainty criteria, i.e., absolute average relative deviation (AARD%), mean squared error (MSE), root mean squared error (RMSE), and correlation of determination (R), are often applied to perform such comparison [42,43]. Equations (9)-(12) express that these uncertainty criteria quantize the compatibility between laboratory-measured CO 2 permeability in MMMs (λ lab CO 2 ) and its counterpart simulation by the ANFIS models (λ sim CO 2 ).
Then, the most accurate neuro-fuzzy model is identified by the spider plot analysis. Different graphical analyses then deeply assess the performance of the selected ANFIS model. Finally, the effect of filler dosage, pressure, and temperature on the CO 2 permeability in the MMM is investigated.

Developing ANFIS Models
As noted, GP, FCM, and SC strategies have been used to divide the input space of ANFIS. Therefore, it is possible to develop and compare three ANFIS types, i.e., ANFIS-GP, ANFIS-FCM, and ANFIS-SC. Table 2 reports the investigated ranges of structural features for each ANFIS type and the selected one by statistical analyses. This table states that the ANFIS-GP and ANFIS-FCM should construct with two and eleven clusters, respectively, while the cluster radius of the ANFIS-SC model should fix at 0.55. The hybrid algorithm is better at performing the training step of all three ANFIS types than the backpropagation.  Table 3 utilizes AARD%, MSE, RMSE, and R-value to measure/compare the accuracy of the selected ANFIS models. The accuracy has been separately monitored for the training, testing, and overall databank. Although all these accuracies are valuable from the modeling point of view, the performance of the ANFIS-FCM and ANFIS-SC is far better than the ANFIS-GP. The next sections apply the spider plot to select the most suitable ANFIS type to estimate CO 2 permeability in the mixed matrix membranes containing SAPO-34 zeolite.

Choosing the Best ANFIS Type
Before performing the model selection analysis, it is better to highlight that a model with a low AARD%, MSE, RMSE, and close R-value to one is preferable from a modeling point of view. The spider plot is a graphical method to compare the statistical indexes provided by ANFIS-GP, ANFIS-FCM, and ANFIS-SC at a glance. In other words, the spider analysis presents the reported statistical values in Table 3 in a multi-dimensional graph.
The results of comparing the performance of ANFIS-GP, ANFIS-FCM, and ANFIS-SC in the training and testing groups and the whole database have been shown in Figure 2a-c, respectively. These spider graphs can be easily analyzed to compare the provided AARD%, MSE, RMSE, and R indexes by different ANFIS types.   The spider graph analyses (Figure 2a,b) indicated that the ANFIS-SC performs better when predicting the training and testing datasets of CO2 permeability in the MMMs. Since the overall databank combines the training and testing groups, it is obvious that the AN-FIS-SC accuracy is also better than the two other models (Figure 2c).

Evaluating the Performance of the Selected ANFIS Model
This section comprehensively evaluates the reliability of ANFIS-SC performance in predicting CO2 permeability in MMMs by combining statistical and visual inspections. The performance assessment has been separately performed for the training and testing groups and their combination (i.e., overall databank).

Training Step
The correlation between experimental CO2 permeabilities in MMMs and their corresponding predictions by ANFIS-SC in the training step has been plotted in Figure 3a. Since  The spider graph analyses (Figure 2a,b) indicated that the ANFIS-SC performs better when predicting the training and testing datasets of CO 2 permeability in the MMMs. Since the overall databank combines the training and testing groups, it is obvious that the ANFIS-SC accuracy is also better than the two other models (Figure 2c).

Evaluating the Performance of the Selected ANFIS Model
This section comprehensively evaluates the reliability of ANFIS-SC performance in predicting CO 2 permeability in MMMs by combining statistical and visual inspections. The performance assessment has been separately performed for the training and testing groups and their combination (i.e., overall databank).

Training Step
The correlation between experimental CO 2 permeabilities in MMMs and their corresponding predictions by ANFIS-SC in the training step has been plotted in Figure 3a. Since all square symbols are gathered around the diagonal line, it can be claimed that the ANFIS-SC model accurately predicts the training datasets.  Visually comparing the actual and predicted values of a dependent variable is a wellknown method to check the accuracy of a model [44]. Thus, numerical values of the CO2 permeability in MMMs in the training step, as well as their associated ANFIS-SC predictions, have been illustrated in Figure 3b. This figure approves excellent compatibility between laboratory-measured and ANFIS-SC predictions in the training phase.
Indeed, the ANFIS-SC predicts 88 actual CO2 permeabilities in MMMs with the AARD = 1.85%, MSE = 5.59, RMSE = 2.37, and R = 0.9996. Visually comparing the actual and predicted values of a dependent variable is a well-known method to check the accuracy of a model [44]. Thus, numerical values of the CO 2 permeability in MMMs in the training step, as well as their associated ANFIS-SC predictions, have been illustrated in Figure 3b. This figure approves excellent compatibility between laboratory-measured and ANFIS-SC predictions in the training phase.

Testing Step
This analysis correlates the experimental CO 2 permeabilities in MMMs and their related ANFIS-SC predictions in the testing step (Figure 4a). It can be seen that the model predictions have been precisely mapped on the experimentally measured data. Indeed, the ANFIS-SC predicts 15 actual CO 2    Figure 4b also presents the numerical values of actual and estimated CO 2 permeability in MMMs in the testing step of the ANFIS-SC. Since the model has seen none of these samples before, its excellent performance for simulating CO 2 separation by the MMM can be observed. Indeed, the ANFIS-SC accurately predicts all unknown CO 2 permeability in MMMs at the testing stage except for one sample.

All Experimental Data
This section analyzes the ANFIS-SC performance in predicting the overall databank of CO 2 permeability in mixed matrix membranes. Since the overall databank is a combination of the training and testing groups, it is expected that the ANFIS-SC model will also have an excellent performance in this analysis.
The experimental and simulated CO 2 permeabilities in MMMs for training and testing groups (overall databank) have been presented in Figure 5a. This figure confirms a remarkable agreement between the laboratory-measured and simulated values. Indeed, the ANFIS-SC predicts 103 actual CO 2 permeabilities in MMMs with the AARD = 2.22%, MSE = 15.08, RMSE = 3.88, and R = 0.9989.     Figure  7. This figure can readily demonstrate the outstanding compatibility between the actual and simulation profiles. This analysis also indicates that the CO2 permeability in the pure Pebax-1657 may be increased by more than 40 barrers by adding 30 wt.% of SAPO-34 zeolite. In addition, the increasing effect of temperature on the CO2 permeability in both pure Pebax-1657 and Pebax-1657/SAPO-34 MMM is observable in Figure 7. This figure also shows that the CO 2 permeability in the Pebax-1074/SAPO-34 is enhanced by increasing either pressure or filler dosage. The maximum CO 2 permeability of 250 barrer is achieved at the maximum pressure and SAPO-34 dosage of 2.4 MPa and 30 wt.%, respectively.
The simulation and experimental profiles for the isobaric CO 2 permeability (0.7 MPa) in Pebax-1657/SAPO-34 MMMs as a function of temperature have been shown in Figure 7. This figure can readily demonstrate the outstanding compatibility between the actual and simulation profiles. This analysis also indicates that the CO 2 permeability in the pure Pebax-1657 may be increased by more than 40 barrers by adding 30 wt.% of SAPO-34 zeolite. In addition, the increasing effect of temperature on the CO 2 permeability in both pure Pebax-1657 and Pebax-1657/SAPO-34 MMM is observable in Figure 7.

Identifying Undesirable Outliers
The leverage method provides a practical ground to identify undesirable outliers in an experimental database [45]. Figure 8 shows that a small number of experimental data (i.e., 4 samples) are undesirable outliers. Since almost all the samples are valid, the undesirable effect of outliers on the model prediction can be ignored.

Identifying Undesirable Outliers
The leverage method provides a practical ground to identify undesirable outliers in an experimental database [45]. Figure 8 shows that a small number of experimental data (i.e., 4 samples) are undesirable outliers. Since almost all the samples are valid, the undesirable effect of outliers on the model prediction can be ignored.

Identifying Undesirable Outliers
The leverage method provides a practical ground to identify undesirable outliers in an experimental database [45]. Figure 8 shows that a small number of experimental data (i.e., 4 samples) are undesirable outliers. Since almost all the samples are valid, the undesirable effect of outliers on the model prediction can be ignored.

Standardized residuals
Valid Outliers Up and low bounds Warning leverage Figure 8. Employing the leverage method to identify undesirable outliers.

Conclusions
The current research combined the statistical and graphical analyses to compare the performance of three neuro-fuzzy types (i.e., ANFIS-GP, ANFIS-FCM, and ANFIS-SC) in predicting the CO 2 separation ability of six different mixed matrix membranes containing SAPO-34 zeolite as a filler. The ANFIS-SC trained by the hybrid algorithm has been identified as the most reliable model for estimating the considered matter. This model predicted 103 experimentally measured CO 2 permeability values in a wide range of membrane compositions and operating conditions with the AARD = 2.22%, MSE = 15.08, RMSE = 3.88, and R = 0.9989. In addition, the visual inspections have also justified the outstanding ability of the proposed neuro-fuzzy model in simulating CO 2 separation by mixed matrix membranes. The simulation and experimental results showed that the CO 2 permeability in all investigated MMMs can be increased by increasing the SAPO-34 dosage, pressure, and temperature. The proposed intelligent model in this study can quickly determine the membrane composition and operating conditions so that the CO 2 permeability is maximized.