The Bi-Directional Prediction of Carbon Fiber Production Using a Combination of Improved Particle Swarm Optimization and Support Vector Machine

This paper creates a bi-directional prediction model to predict the performance of carbon fiber and the productive parameters based on a support vector machine (SVM) and improved particle swarm optimization (IPSO) algorithm (SVM-IPSO). In the SVM, it is crucial to select the parameters that have an important impact on the performance of prediction. The IPSO is proposed to optimize them, and then the SVM-IPSO model is applied to the bi-directional prediction of carbon fiber production. The predictive accuracy of SVM is mainly dependent on its parameters, and IPSO is thus exploited to seek the optimal parameters for SVM in order to improve its prediction capability. Inspired by a cell communication mechanism, we propose IPSO by incorporating information of the global best solution into the search strategy to improve exploitation, and we employ IPSO to establish the bi-directional prediction model: in the direction of the forward prediction, we consider productive parameters as input and property indexes as output; in the direction of the backward prediction, we consider property indexes as input and productive parameters as output, and in this case, the model becomes a scheme design for novel style carbon fibers. The results from a set of the experimental data show that the proposed model can outperform the radial basis function neural network (RNN), the basic particle swarm optimization (PSO) method and the hybrid approach of genetic algorithm and improved particle swarm optimization (GA-IPSO) method in most of the experiments. In other words, simulation results demonstrate the effectiveness and advantages of the SVM-IPSO model in dealing with the problem of forecasting.

the external environment. Therefore, it is difficult to establish a precise mathematical model to represent linearly the relation between properties indices and productive parameters.
With the rise of intelligent algorithms, the theory of intelligent algorithm has provided a powerful tool of systematic research for analyzing the unknown complex nonlinear system. Among them, artificial neural network (ANN) has proved to be an excellent adaptive method with dark-box operating performance, powerful study and generalized ability to deal with modeling the dynamic process for process control. Kadi [11] used the ANN to predict mechanical modeling for fiber-reinforced composite materials. Yu et al. [12] used a fuzzy ANN to predict the fabric hand in different fabric specimens. However, ANN needs a long training time because its topology is not compact enough. Then RNN was adopted to compensate for the weaknesses of ANN. Du et al. [13] investigated the center selection of multi-output RNN. Roy et al. [14] investigated the learning theory of the RNN. Hong et al. [15] presented a novel topology of the RNN, referred to as the boundary value constraints. Huang et al. [16] investigated the function approximation of the RNN. Qiao et al. [17] presented a self-organizing RNN to identify nonlinear systems and the model. Wang et al. [18] proposed a forecasting method based on self-correcting RNN for adapting changing conditions. However, the training procedure for ANN models is not only time consuming but it is also possible to get trapped in local minima and subjectively in selecting the model architecture [19]. SVM is a relatively new artificial intelligence technique which is increasingly being applied to geotechnical problems and has yielded encouraging results [20,21]. SVM implements the structural risk minimization principle rather than the empirical risk minimization principle implemented by most traditional ANN models [22,23]. Based on this principle, SVM achieves an optimum network structure; meanwhile, it can lower the global error of the model. It raises the generalization capability of the model, which is more prominent in small-sample learning. SVM have found a wide application in the fields of pattern recognition, bio-informatics, and nonlinear regression estimation problems. Particularly, support vector regression (SVR) is an extension of SVM. Wang et al. [24] proposed a hybrid load forecasting model combining differential evolution algorithm and SVR. Gilan et al. [25] developed a hybrid SVR and PSO model to predict the compressive and rapid chloride penetration test (RCPT) results of concretes containing metakaolin. Hong [26] developed a hybrid SVR and chaotic particle swarm optimization model to predict electric load. Kang et al. [27] developed swarm intelligence approaches to optimize power flow problems in power networks. Liang et al. [28] developed an adaptive PSO method based on clustering to solve multimodal function optimization problems.
In this paper, we propose the SVM-IPSO hybrid model to bi-directionally forecast a productive process for carbon fiber. We use IPSO to tune the parameters in SVM. With this model, we can predict the performance of carbon fiber and obtain a design scheme for new carbon fiber.
The main contributions of this article include the following: (1) we propose IPSO, which pushes forward the development of intelligent computing; (2) the hybrid SVM-IPSO model is used to set up a bi-directional prediction approach for carbon fiber; (3) with the proposed model, we can online forecast the productive parameters and the properties of carbon fiber, adjust technical parameters in time, reduce not only wasted time but also energy consumption.
The main structure of this article is organized as follows: The first section describes the research motivation of this paper. The second section introduces the regression algorithm of SVM. The third section introduces the parameters for selecting SVM based on IPSO. The fourth section testifies the performance of the proposed model with the experimental data and compares the results with other methods. The fifth section discusses the research conclusion.

The Process of Carbon Fiber Production
The technological process for carbon fiber is a sufficiently complex system. It is a very meaningful research subject to control the performance of the carbon fiber process in as small a range as we possibly can. For this purpose, we should observe the parameters of the productive process in real-time, synchronously predict the product properties of the carbon fiber, after which we can determine the defective carbon fiber, timely adjust the parameters of the productive process, avoid reducing the quality of the carbon fiber, and prevent significantly wasting of raw materials. The production of carbon fiber mainly comprises several processes, such as coagulation, stretching, oiling, and heat treatment. Figure 1 shows the main technological process of carbon fiber production.

Coagulation
Stretching Oiling

Heat treatment
Fiber roll Figure 1. The technological process of carbon fiber production.

The Bi-Directional Prediction Methods for Carbon Fiber Production
The bi-directional prediction methods include forward prediction and backward prediction. Forward prediction is the process of predicting property indexes from productive parameters. It means that when the productive parameters of carbon fiber are changed, the corresponding performance of carbon fiber will fluctuate accordingly. In other words, if the productive parameters have little differences, there must be the same among the carbon fiber performances. The backward prediction is a reverse process of the forward prediction, so the process can be seen as a method for designing new types of carbon fiber. In the early stage, we design a new kind of carbon fiber through previous experience, then we test this kind of carbon fiber with chemical and physical instruments to verify its expected performance, get inconsistent results, adjust the corresponding parameters in a new scheme, produce carbon fiber products again, repeat this process of production, testing, and adjusting parameters until we get a satisfactory performance of the carbon fiber. From the foregoing, this method wastes a lot of raw materials, energy, and time. The worst thing is that, if we choose inappropriate parameters for the manufacturing equipment, it will cause damage to the corresponding equipment. For this reason, a better way to solve this problem is to use computer simulation in the whole productive process, avoid unilateral decisions made by workers with interests and experience, and give out more reasonable control parameters based on the actual production process.
In this paper, we propose a bi-directional prediction model of carbon fiber production based on SVM. We adopt IPSO to tune the parameters of the model. The overall chart of bi-directional prediction approaches are as shown in Figure 2.

The SVM Model
The SVM, proposed by Vapnik in 1995 [29], is widely used for many machine learning problems such as bio-informatics, pattern recognition, linear regression analysis and nonlinear regression estimation problems.
The basic principle is as follows [30]: given a training data   1 , N ii i xy  , where n i xR  is the input vector and i yR  is the corresponding output of input xi, and N is the total number of the input vectors.
By nonlinear mapping φ(x), the data are mapped nonlinearly from the original feature space to a high-dimensional feature space, thus, it can be approximated in a linear way as follows: where f(x) denotes the regression estimate function; w denotes the weight vector; and b denotes the bias vector. As mentioned above, the problem of regression approximation is equivalent to minimizing the empirical risk, where C and ξ are regularized constants; Θξ(yi, f(x)) denotes the ξ-insensitive loss function and is defined as Then, we introduce two positive slack variables ' ε i and εi to simplify the risk function R(f) in Equation (2). Thus, Equation (2) is transformed into the objective function shown as follows: where εi is the lower training error ( ' ε i is the upper) subjected to the ξ-insensitive loss function.
After the convex quadratic programming problem with constraints is solved, the weight vector w in Equation (1) is obtained as Equation (6), where ' β i , β i are the Lagrangian multipliers and are obtained by solving a convex quadratic optimization.
Finally, the regression function is obtained as the following equation: where K(xi, xj) is called the kernel function, which is the inner product of φ(xi) and φ(xj), that is, . The values of φ(xi) and φ(xj) are produced by mapping xi and xj into the higher-dimensional feature space, respectively. It can be shown that any function meeting Mercer's criteria [29] can be used as the kernel function. However in general, three different types of kernel functions are commonly used, e.g., the sigmoid kernel function, the polynomial kernel function, and the Gaussian RBF kernel function. The Gaussian RBF is a most used for the kernel function because of the fewer free variables that need to be set and an excellent nonlinear forecasting performance [31]. Therefore, in this study, the Gaussian RBF was selected (as shown in Equation (8)) as the kernel function: Therefore, there are two variables that need to be selected in the SVM model, which are the constant "C" and the width of the Gaussian RBF kernel "σ". In this paper, IPSO is used to optimize the parameters of the SVM model.

Overview of Particle Swarm Optimization
The PSO is inspired by simulation of the behavior of birds flocking in nature [32,33]. It has won high reputation in numerical function optimization due to its precision and efficiency in the fields of stochastic optimization algorithms. It solves a problem by initializing a random population of candidate solutions (particles) and can search for the optimal solution through the search space. Each particle is characterized by a dynamic adjustable velocity, which originates from the momentum of a flying particle. Giving the PSO sufficient numbers of iterations, it is hoped to end up with a global optimal solution. The above-mentioned behavior of the i-th particles can be mathematically expressed as: In the above formulas, ω denotes the inertia weight, t iD V denotes its velocity vector of the i-th particle on dimension D; t iD pbest denotes its previously best known position of the i-th particle on dimension D and t D gbest denotes its best global position, c1 and c2 are social and personal learning parameters, r1 and r2 are random numbers in the range [0,1]; and t iD X denotes its previous position of the i-th particle at time t.

The Basic Concept of the Cell Communication
Cell communication refers to a message through a medium to another cell and interacts with the corresponding receptors of target cells, and then through a series of physiological changes inside the cell. Indeed, cells cannot live in isolation, and survive by processing and receiving information from the external environment, whether that information suits the availability of nutrients and change in temperature. Cell to cell communication is essential for orchestration and coordination of cellular events in multi-cellular systems [34]. Intercellular communication has three main ways: gap junction, cell recognition, and chemical communication.
Gap junction: narrow water-filled channels that connect the cytoplasm of adjacent epithelial cells, as well as of some other cell types.
Cell recognition: mutual identification among cells, usually by particular complementary interaction among their separate membrane glycoproteins or surface molecules.
Chemical communication: cells secrete some chemicals (such as hormones) to outside the cell, as a signal molecule in target cells, adjusts its function, is specific and can be divided into four forms: (a) endocrine; (b) paracrine; (c) synaptic; (d) autocrine.

IPSO Based on the Communication Mechanism of Cells
Although PSO has been applied to many kinds of practical problems, PSO still suffers to some degree of premature convergence and poor quality of solution. To overcome the shortcomings, we present IPSO by combining the cell communication mechanism. Based on the three main ways of cell communication, three evolutionary strategies are established. Specifically, the gap junction, the cell recognition, the chemical communication in the mode of cell communication can be replaced by evolutionary strategies, Equations (15)- (17) in the iterative and evolutionary process, respectively. IPSO applies modified update formula to maintain population diversity and enhances the convergence velocity and precision, which by incorporating the information of the global best solution into the search strategy improves the exploitation. In particular, the modified update formulas are determined as follows.
A solution of problem Pi with D variables can be expressed as: 12 =[ , , , ] 1, 2, , An intuitive illustration of how the Euclidean distance is measured is given in Figure 3. As can be seen from Figure 3, the first line represents the Euclidean distance between the 1-th solution to all solutions, the second line represents the Euclidean distance between the 2-th solution to all solutions, and so forth. The Euclidean distance between solutions of problem can be defined as: ,  The concentration of the particles can be expressed as: where Nk denotes the total number of similar particles, and N denotes the total number of particles. In this paper, two parameters of SVM are optimized, so, each particle's position is checked using the following formula: If (Ck > 0.5): where pi1 denotes the first value of the i-th particle; and xb1 denotes the first value of the best particle.
If ( ' ii ff  ): where i f denotes the fitness function value; ' i f denotes the updated fitness function value; pi2 denotes the second value of the i-th particle, and xb2 denotes the second value of the best particle. The last particle's position is checked using the following formula: where Pi denotes the i-th particle; and xb denotes the best particle of the whole particles. The values of the aforementioned parameters are usually prescribed to be selected as follows: ω = 0.8 and c1 = c2 = 2. Furthermore, ω denotes the particle's search capability, and defines the balance between local search capabilities and global search capabilities. Once the velocity of the particle is calculated, its new position is updated according to Equation (10). The range of permissible velocity of the particle is usually restricted in order to eliminate fluctuations and obtain better exploration during the iteration process.

SVM Based on the IPSO
The diagram of SVM-IPSO forecasting model is illustrated in Figure 4.  The details of IPSO for selecting parameters for the SVM model are as follows: Step 1: Initialization parameters The population size is 20, the maximum iteration number is 100, the initial particle swarm location is X, and the random flight velocity is V should first be determined. In the SVM-IPSO program, we set X = rand (1,2), where rand() is a random function in the range [0,1].
Step 2: Evolution starts Set gen = 0, and give the random position and the flight velocity for food finding of an individual particle i. In the SVM-IPSO program, we use two variables [x1,x2] to denote the new position for food finding of an individual particle i.
Step 3: Preliminary calculations Calculate the new position Xi of an individual particle i, and then input Xi into the SVM model for carbon fiber production forecasting; the parameters [C,σ] of the SVM model are replaced by [Xi(1),Xi (2)]. According to the carbon fiber production forecasting result, the fi can be calculated. The fi is represented by the root-mean-square error, as shown in Equation (18), which calculates the errors between the actual values and the forecasted values: where n is the number of training input data; yi is the i-th actual value; ˆi y denotes the i-th forecasting value.
Step 4: Offspring generation The global best value is generated according to Equations (15)-(17) after updating the position value according to Equation (10). Then the global best value is inputted into the SVM model and the fitness function value calculated again. Set gen = gen + 1.
Step 5: Circulation stops When gen is equal to the maximum iteration number, and the stop criterion is met, the optimal parameters of the SVM model are finally obtained. Otherwise, it is necessary to go back to Step 2.

The Preprocessing of Sample Data
In order to examine the performance of the new prediction model, we used the data organized from experiments in Table 1 [9]. The errors by the proposed method for the below mentioned experimental data, classified into four different types are given in Table 2, where to facilitate comparison the corresponding errors obtained by other methods on the same data are also listed. Note: sample data sources come from reference [9]. Table 2. Comparison of errors among the proposed method, the GA-IPSO-RNN [9], the basic PSO-RNN [9], and the conventional RNN [9]. In the SVM modeling, the preprocessing steps are implemented before attempting to calculate, to eliminate any bad data or outliers. These steps guarantee that the raw data retrieved from the database is completely suitable for modeling. In order to improve the accuracy of prediction and smooth the training procedure, all the sample data are normalized to fit them in the interval [0,1] using the following linear mapping formula: where ' i x is the mapped value; i x is the initial value from the experimental data; N is the total number of input data; xmax and xmin denote the maximum and minimum values of initial data, respectively. In Table 1, the values for the first six columns indicate different parameters, and the values for the last two columns indicate different property indices. These detailed parameters can synthetically represent the whole manufacturing process. Meanwhile, the two kinds of property indices can fully represent the performance of the carbon fiber. In the predictive process, we obtained training sets from the first 45 rows of samples and test sets from the last five rows of samples. Also, to examine the performance of the several algorithms, the mean absolute error, root mean square error, mean relative error and Theil's inequality coefficient were defined according to the following formulations, respectively: Mean absolute error: where yi is the true value; i ŷ is the predicted value.

The Selection of Comparative Models
As mentioned above, the technological process for carbon fiber is a nonlinear system, it involves many process parameters that must be carefully optimized and adjusted. Most of the previous work was focused on analyzing the properties of carbon fibers by means of physical or chemical instruments. However it is a kind of method which wastes time and energy. Thus, we should build the predictive model for the productive process of carbon fiber. In this way, we can effectively predict the properties of carbon fiber in real-time, and thus we can find out inferior carbon fiber with poor quality in advance, timely adjust the process parameters, and avoid wasting raw materials. However, on the contrary, the prediction of productive parameters by the properties of the carbon fiber is a backward reasoning process. Therefore, it is a better idea to analyze all antecedent production schemes through computer simulation. This will avoid partial production schemes created by operational staff who rely on their personal experience and interests.

Forward Prediction
Carbon fiber production is a positive process, and the predictable performance of carbon fiber by productive parameters is a forward prediction, too. Thus, in forward prediction, we can take the parameters of carbon fiber production as input and the corresponding performance of carbon fiber as output. By employing IPSO to optimize the parameters of SVM, we gain the best parameters for SVM. The training effectiveness of the proposed method is shown in Figure 5.
Comparing performance of the conventional RNN, the basic PSO-RNN, the GA-IPSO-RNN with the SVM-IPSO is carried out in terms of (a) mean absolute error; (b) root mean square error; (c) mean relative error and (d) Theil's inequality coefficient result. The detailed results are listed in Table 2.
From Table 2, we can see that on the basis of MAE, the proposed SVM-IPSO achieves roughly a decrease of 55.59%, 13.64% and 10.09% compared to the other three methods, respectively. Similar results can be obtained from the other errors such as MRE with a decrease of 62.69%, 16.39% and 9.74%, RMSE with a decrease of 60.59%, 16.69% and 9.90%, and TIC with a decrease of 62.89%, 20.33% and 13.17%. Similar results are obtained on the basis of strength and structure parameters.

Backward Prediction
The productive parameters of carbon fiber forecast by the performance of carbon fiber is a reverse process of forward prediction, so the predictive process can be considered inverse inference. In the backward prediction, we take the performance of carbon fiber as input and the productive parameters as output, which can be considered as a design scheme for a novel style carbon fiber. In order to achieve the above goals, by employing IPSO to optimze the parameters of SVM, we gain the best parameters for SVM. The training effectiveness of the proposed method is shown in Figure 6. We compared the effectiveness of SVM-IPSO with that of conventional RNN, the basic PSO-RNN, and the GA-IPSO-RNN optimal schemes. The detailed results are listed in Table 3. In Table 3, (1) is the viscosity average molecular weight; (2) is the conversion ratio; (3) is the solid content; (4) is the spinning jet drawing ratio; (5) is the coagulating bath temperature; (6) is the total drawing ratio.
From Table 3, we can see that on the basis of MAE, the proposed SVM-IPSO achieves roughly a decrease of 98.69%, 19.48% and 6.68% compared to the other three methods, respectively. Similar results can be obtained from the other errors such as MRE with a decrease of 98.72%, 25.78% and 7.51%, RMSE with a decrease of 99.22%, 18.70% and 7.60%, and TIC with a decrease of 93.02%, 23.26% and 11.48%. Similar results are obtained on the basis of viscosity average molecular weight, conversion ratio, solid content, spinning jet drawing ratio, and coagulating bath temperature, respectively. Table 3. Comparison of errors among the proposed methods, the GA-IPSO-RNN [9], the basic PSO-RNN [9], and the conventional RNN [9].

Conclusions
The non-linear relationship of the productive parameters for carbon fiber with its performance makes bi-directional prediction very complicated. Thus, how to improve the bi-directional prediction accuracy is worthy of study. In this paper, we present the SVM-IPSO hybrid algorithm to bi-directionally forecast the productive process for carbon fiber, which can be considered as predicting the performance of carbon fiber and designing a method for new carbon fiber production. By the proposed model, we can online forecast the performance of the carbon fiber, and adjust the productive parameters in time, while reducing time and energy. In addition, this model can provide a means to determine the productive parameters for production to meet the performance requirements of different fibers.