OPTIMIZING HYBRID BASIS FUNCTION TYPES IN FUNCTIONAL NETWORK DESIGN VIA CULTURAL ALGORITHM

-In this paper, a novel functional network designing method based on cultural algorithm is proposed. This method can get the hybrid basis function and its parameters with optimal searching achieving the learning between functional network structure and the functional parameters, and each model of the initial set is evaluated by the adjusted  R squared criterion ) ( 2 a R , to find the optimal basis functions and functional parameters. Finally, the simulation experiments show that the proposed constructive functional network design scheme is feasible, and can obtain better generalization property.


INTRODUCTION
Castillo introduced the functional network in 1998 [1].It is an extension of the standard neural network.Unlike neural networks, it deals with general functional models instead of sigmoid-like ones, in these networks there are no weights associated with the links connecting neurons, and the neural functions are unknown from given families to be estimated during the learning process.We can select appropriate families for each specific problem (such as polynomials, Fourier expansions and trigonometric functions, etc.).At present, the functional network is a very useful general framework for solving a wide range of problems: The solving of differential functional and difference equation [2], nonlinear time series and prediction modeling [3], factorization model of multivariate polynomials [4], the identification of nonlinear system [5], CAD, linear and nonlinear regression [4], etc.The functional networks have shown excellent performance in the above-mentioned problems.
Functional network achieved a greater success in the application, but its theoretic cal basis is imperfect, greatly limits the scope of application of the functional network.
The key theories include: first, what type of network structure and the family of basis function, often by experts to make judgment based on empirical knowledge; second, the network structure and functional parameters, this point is often difficult to determine by artificial; third, the network structure is determined, also exist the local minima problem.At present, there are some using genetic programming [6] to solve the above problems, and have achieve good results; but the family of basis function how to select, the theory has not yet been given a general method to avoid the family of basis function sample select insufficient, thereby lead to functional network approximation error is too large or to much which led to overfit problems.At present, the commonly can be used the generalized associatively functional networks, the separable functional networks, the generalized bisymmetry functional networks, the serial functional networks model, etc.For these network models, only applies to some deterministic background knowledge of the problem.However, for the general case, there is no system design method to a given design the approximate optimal approximation structure.
In 1994, Robert G. Reynolds from the simulation of human social and cultural evolution of the idea, proposed cultural algorithm (CA) [7].Cultural algorithm is an algorithm used to solve complex calculations in the novel global optimization search algorithm.In human society, cultural can be seen as the carrier of information, the information potentially affects all members of society, and the benefit of offspring to solve the problem of guiding their practical activity.Similarly, the cultural algorithm is that the important idea of the evolution of populations from the problem to be solved to obtain the knowledge (i.e.beliefs), and the feedback that knowledge to guide the search process.That the process is, respectively, from the micro and macro level, two different levels of simulated biological evolution and cultural dimensions of evolution, the evolution and influence each other and promote each other, so they formed a "dual evolution dual promote" mechanism.In recent years, CA has caused great concern in domestic and foreign scholars [8][9][10].
This paper will attempt to propose algorithm using cultural algorithm to achieve the functional network structure approximation model, according to the structural characteristics of the functional networks and cultural algorithm of dual evolutionary mechanism.Through the hybrid basis function is introduced, and the use of traditional genetic algorithm of binary coding method to optimize learning hybrid basis function, application of  R squared criterion ) ( 2 a R for the design of fitness function, so as to determine the optimal approximation of functional network structure model.In this model, the genetic algorithm incorporated into the cultural algorithm framework, composition based on genetic algorithm population space and knowledge space two spaces, this dual structure characteristic of the cultural algorithm, which makes the method, can be used in the process of problem solving from population obtain the experience knowledge to guide the search process, so it has better global optimization performance and faster search speed.Finally, the simulation experiment shows the feasibility and accuracy of the method.
The rest of the paper outlines as follows.In Section 2, the basic cultural algorithm is introduced.The functional network is introduction in Section 3. The optimizing hybrid basis function types in functional network design via cultural algorithm are given in Section 4. The experimental results and analysis is given in Section 5. Finally, Section 6 concludes this paper.

INTRODUCTION TO CULTURAL ALGORITHM PRINCIPLE
The traditional evolutionary computation methods have limited mechanisms for representing, storing and transmitting knowledge from one generation to the next.Cultural algorithm (CA) was proposed by Reynolds in 1994.
CA models the evolution of the cultural component in an evolutionary computational system over time.Culture helps human beings adapt to their environments.In human societies, culture can be viewed as a vehicle for encoding, generalization and storage of knowledge potentially accessible to all members of the society.It is useful to guide their problem solving activities.On defining human culture, it is generally assumed that there exists a way to represent, acquire and reason about knowledge in a cultural content.CA is a class of computational models derived from models of cultural evolution processes.These algorithms support the basic mechanisms for cultural changes described in anthropological and archaeological literature.
Seen from a technical point of view, a CA is a knowledge-based evolutionary computational system.Its basic idea is to incorporate knowledge components into traditional evolutionary computational systems.One of its frameworks can be depicted in Fig. 1.As shown in Figure 1, it models two levels of evolution: the population space level and the belief space level.The two spaces are connected together by an explicit communication protocol composed of an acceptance function and an influence function, which are denoted here as () Accept and () Influence respectively.The acceptance function is used to glean the experience of selected individuals from the population; then the belief space can be modified by an update function, denoted here as () update ; next, the influence function can make use of the problem-solving knowledge in the belief space to guide the evolution of the population component.In the population space, like traditional evolutionary population models, individuals are first evaluated by a generation function select is used to select the population for next generation.Cultural algorithm pseudo-code is given in the literature [7].(1) Dual evolutionary inheritance: In the population space and belief space are inherited parent information; (2) Population space evolution is saved by the belief space knowledge to guide; (3) Support the population space and belief space hierarchy; (4) Support the adaptive evolution of two space; (5) Different space evolution can be carried out at different speeds; (6) Support a hybrid of different algorithms to solve the problem; (7) "Cultural" change can be expressed in different models within a model.The cultural algorithm flow chart can be depicted in Figure 2.

Elements of a functional network
In general, a functional network consists of the following elements: 1) A layer of input units.This first layer contains the input data.Input units are represented by small black circles with its corresponding name.
2) A layer of output units.This is the last layer, and contains the output data.Output units are also represented by small black circles with its corresponding name.
3) One or several layers of processing units.A neuron is a computing unit which evaluates a set of input values, coming from the previous layer, of neurons or input units, and gives a set of output values to the next layer, of neurons or output units.The computing units are connected to each other, in the sense that output from one unit can serve as part of input to another neuron or to the units in the output layer.Once the input values are given, the output is determined by the neuron type, which can be defined by a function.For example, assume that we have a neuron with s inputs 12 ( , , , ) and k outputs 12 , , , k y y y  , then, we assume that there exist ( , , , ), 1,2, . The functions j f are not arbitrary, but determined by the structure of the network, as we shall see later.Neurons are represented by circles with the name of the corresponding j f function inside.4) A set of directed links.They connect the input layer to the first layer of neurons, neurons of one layer to neurons of the next layer, and the last layer of neurons to the output units.Connections are represented by arrows, indicating the information flow direction.All these elements together form the network architecture, which defines the functional capabilities of the network.Network architecture refers to the organization of the neurons and the connections involved.In multilayer networks, units are organized in series of layers.Information flows in only one direction, from the input layer to the output layer.Neuron units receive information only from previous layers of the network, and output information to the next layer of neurons, or to the output units.
In Ref [5] one example of a simple functional network is given in Fig. 3, and its corresponding neural network architecture is also given in Figure 4.  ,, x x x , the first layer of neurons contains neurons 1 f and 2 f , the second layer of neurons contains neurons 3 f , and the output layer reduces to the unit 6 x .One of the most important is the choice of neurons function ( 1, 2,3) i fi  .According to Castillo's approach will each neuron functions ( 1, 2,3) i fi  is represented as a linear combination of the known functions of a given family.Such as, polynomials, trigonometric functions, Fourier expansion etc.In the neural network, each neuron function i f , that is, the activation functions often take the Sigma function, hyperbolic tangent function, etc.In standard neural networks the neuron functions i f are fixed, and some weights associated with the links or connections have to be learned.However, in functional networks there are no weights, and the neuron functions i f must be learned.As for the other differences with the neural network, this paper will not repeat them, in Ref [12].

The general functional networks model
The general functional network model [13], When contains the transformation of the corresponding variable y , equation (1) be generalized as, Where

The additive functional networks model
The additive functional equation is Which leads to the functional network showed in Figure 6.

Figure 6. The additive functional network model
To estimate f and … 12 ,, k h h h in (3), we consider linear combinations of basic functions , that is,  , etc.

2) The hybrid family of basic functions
The hybrid basis functions form is a number of standard basis functions and operators composition expressions.This article experiments only consider the form of this hybrid basis functions between multiplied by the standard basis functions, such as: , etc.Other types of hybrid basis functions are similar to the hybrid basis function between multiplied by the standard basis functions form.
3) Populations space and Knowledge space encoding Populations space encoding and the basic idea of evolution: according to the problem of the functional network in which you want to use to the family of hybrid basis functions into the population space.For the hybrid basis function in the population space, the family of hybrid basis function composed of each model (i.e.chromosome string) as an individual, and according to the binary coding, 0 represents corresponding to of hybrid basis function of the bit will not be taken; on the contrary, 1 represents corresponding to of hybrid basis function of the bit will be taken.Encoding length according to the scale of the problem was determined.And in the population space generation and binary strings of equal length of real numbers, each time evolution, the real numbers of the corresponding also perform the genetic operation and evolution.Knowledge space and population space used the same encoding.

4) Sorting
In accordance with the problem to be solved in advance by the use of basis functions combined into a hybrid basis functions form, and prior to these hybrid basis functions according to the dictionary order.The choice of what kind of standard basis functions combined into what kind of hybrid basis functions, depending on specific problems and different.For example, there is a functional network model structure is shown in Figure 7.
Figure 7.A functional network topological structure Then, the output of the network as follows: Further can be written as If the neuron function . Similarly, other models can also be randomly obtained, so that each model corresponding to an output of the network, each output corresponding to an individual in the genetic algorithm, then all the individual random formation of initial population.
Also, for a model 2 , the model will be approximation.The basis functions from this model can be seen clearly by the use of polynomial basis functions } , , , , , and then according to the model, we can identify to need hybrid basis functions are } y y y x x x x x x x x x x x x x x x x x x x x x x x x .In ascending order according to this sort, namely the dictionary order.Whereas according to with the encoding on the above of population space, if there is a binary string of 0101010010100000000 , then the model in terms of their corresponding terms are . Moreover, if the binary string is 1111111111111111111 , then the chromosome string represents the complete model.The other family of hybrid basis functions and this case sorting is similar.

The objective function, fitness function defined
In general, for the training of neurons, training goal is the desired output and the actual output of the mean square error RMSE of the least.We may assume that the definition is to train the neuron function of the objective function as follows: ( , , , ) ( ) where, is the family of basic functions.
For each individual can be used as follows R  square 2 () a R structure learning evaluation criterion formula to evaluate the learning effect, then the fitness can be defined as: where n is the sample size, p is the number of parameters in the model, where i F ˆ can be given by a linear combination of the basis functions) and ). ( 1) (

ˆ
The larger 2 a R value is a excellent individual.
In fact, merely an indicator of the 2 a R to evaluate the degree of the advantages and disadvantages of each individual was not enough, when the same or very close to the two 2 a R the case, it is difficult to determine which individual is the best.R value maximum of individual is the optimal, and meanwhile, the individual corresponding to the network structure is optimal.

Cultural algorithm with two large space communications 1) Acceptance function-() accept
The acceptance function () accept is used to choose directly influence the knowledge of individual in the current knowledge space.In other words, in each population space of evolution of individual, each evolution AcceptStep , with each population space current global best value to replace the worst individual knowledge space.

2) Influence function-() influence
The Knowledge space by () influence influence the population space generate the next generation.In other word, in the knowledge space, each evolve tep InfluenceS generations, the knowledge space group as part of a good fitness value replace the worst individual population. where, EndStep is maximum generations number; p CurrentSte is current generation number; 1 N and 2 N are predefined constants, Here were taken at 15 and 100.

Cultural algorithm to achieve the functional networks learning algorithm
Cultural algorithm to achieve the functional networks learning algorithm steps are as follows: Step 1: To use of the hybrid basis functions are given by prior to solving problems, and in accordance with the sorting rules on which of these hybrid basis function ordering.Population space structure using a genetic algorithm, generating size P of initial population () POP t , encoding method using a binary code (0,1) .Moreover, initialize the populations and randomly generated initial population of the same length real number, each time after evolution, real with the evolution of genetic operation.Knowledge space is initialized using the same encoding scheme with the population space.
Step 2: The () POP t of each individual fitness is calculated by the objective function () objective , the fitness using R  square 2 () a R structure learning evaluation criterion.The lager fitness function value and complexity relatively minimum, the individual corresponding to the network structure is optimal.
Step 3: Using the acceptance function () accept to select some excellent individuals, to give the knowledge space, and to update the knowledge space.
Step 4: The population space of individuals selection, crossover and mutation operations.
Step 5: Through the influence function inf () luence in turn affects the lower population space, return step2.
Step 6: To determine whether a termination condition is satisfied, yes, to terminate the algorithm; no, return step2.
The algorithm flow chart is shown as follow in Figure 8.

Experimental Results
By the following three models to test the proposed algorithm, model training sample data are randomly generated by the function () Random , the sample data of number is not less than 50; crossover and mutation probabilities are 0.4 and 0.1.The fitness value is calculated by the formula (9).Whereas cultural algorithm required combined into hybrid basis function of type and number of standard basis functions are determined by the specific issues.In practice, according to the to increase or decrease, and increase of decrease the function type nothing to do with the method itself.The proposed method is validated by selecting the model respectively: where, ,sin( In the choice of model, we not only to make the 2 a R larger, while also makes its complexity is relatively small.It can be seen by Table1, which the optimal hybrid basis function relatively better function is , corresponding to functional parameters are: 0.013176671873512, 0.970100892851614.That the model is obtained by the proposed algorithm, as follow: ^2 0.013176671873512 0.950100892851614sin(2 ) Graphics of the original model (solid line) and the proposed method approximation model (dashed line) are shown in Figure 10.In the choice of model, we not only to make the 2 a R larger, while also makes its complexity is relatively small.It can be seen by Table 2, which the optimal hybrid basis function relatively better function is 0.702590726735098 0.967015829814410 0.984739013812769 1.184314851630114 Graphics of the original model (left) and the proposed method approximation model (right) are shown in Figure 12. , then the corresponding population space and knowledge space which 6 bits binary chromosome string to initialize, the initial functional parameters are randomly generated.The optimal functional network parameters can be obtained by the proposed method, respectively: 0.601412126269725, 0.182081868411385, 1.018137216399071, 0.704301728433609, 0.944199726747308, 0.120426688224212, and the fitness value ) ( 2 a R 0.999405710364281 can be obtained, its fitness value corresponding to the optimal binary string as 001010 .Thus, in accordance with the prior agreement of the sorting can know this chromosome string corresponding to the hybrid basis function 2  , sin x x y .The fitness evolution curve is shown in Fig. 13.

Experimental Analysis
From equation ( 13), ( 14) and (15) model and the 3 test models compared, it is clear that the proposed algorithm is effective and feasible; and from the Fig. 9, Fig. 11 and Fig. 13 can clearly see that the fitness function (i.e. 2 a R ) of the convergence speed is equivalent fast; whereby Fig. 10 gives the test model ① approximation curve, the solid line as the original model curve, and the dashed line is the proposed algorithm model.Approximation effect of the proposed algorithm can be seen to be very well.Fig. 12 and Fig. 14 give the test model ② and ③ curve in three-dimensional coordinate, from the original model curve (left) and the proposed method approximation model (right) can clearly see that the proposed method approximation effect is very well.

CONCLUSIONS
In this paper, functional network is regarded as structural and functional parameters of the optimization process, using the basic framework of cultural algorithm, through for the hybrid basis function form in functional networks using the standard basis functions combined treatment, and they are encoded by the traditional binary, then the genetic algorithm was embedded into the population space of cultural algorithm in the design of functional networks to carry on the learning optimization, for each individual can be used as follows R  square 2 () a R structure learning evaluation criterion formula to evaluates the learning effect, thereby, the optimal basis functions and the corresponding functional parameters were found.In the practical application, according to the practical problems that need to be selected, making the generalization performance of the network design is enhanced.

Figure 1 .
Figure 1.A framework of cultural algorithm

YesFigure 2 .
Figure 2. A flow chart of cultural algorithm

Figure 3 .
Figure 3.A typical functional network topology model

Figure 4 .
Figure 4.And Figure 3 equivalent neural network unknown parameters expressed in terms of functions s are linearly independent of the family of basic functions.The general functional network model is given in Figure5.

Figure 5 .
Figure 5.The general functional network model

4 . 1 .
basic functions, both of which can be obtained by learning.4.OPTIMIZING HYBRID FUNCTION TYPES IN FUNCTIONAL NETWORK DESIGN VIA CULTURAL ALGORITHM The hybrid family of basic functions and their coding, sorting1) The family of basic functionsIn functional networks, commonly used the family of basic functions: polynomial basis functions

3 .
Each model is then represented by a chromosome of length 10.The chromosome ] 1111111111 [  v represents the complete model, that is, all the terms are included in the model,

Figure 8 .
Figure 8.The flow chart of the proposed algorithm

1 )
For the model ①, choose sine basis function and exponential function combination set hybrid basis function is:

Figure 9 Figure 10 .Figure 11
Figure 9.The fitness function ) ( 2 a R evolution curve x x x x , corresponding to functional parameters are: 0.702590726735098, 0.967015829814410, 0.984739013812769, 1.184314851630114.That the model is obtained by the proposed algorithm,

Figure 12 .
Figure 12.The model approximation curve 3).For the model ③, choose polynomial basis function and sine function combination set hybrid basis function is } sin , sin , sin , , , 1 { 2 2 y x y x y x x, then the corresponding population space and knowledge space which 6 bits binary chromosome string to initialize, the initial functional parameters are randomly generated.The optimal functional network parameters can be obtained by the proposed method, respectively: 0.601412126269725, 0.182081868411385, 1.018137216399071, 0.704301728433609, 0.944199726747308, 0.120426688224212, and the fitness value ) ( 2

Figure 13 .
Figure 13.The fitness function ) ( 2 a R evolution curve

Figure 14 .
Figure 14.The model approximation curve

Table 1 ,
the fitness evolution curve is shown in Figure9.

Table 1 .
Selected models to approximate the simulated model in ①