FPGA Implementation of a Functional Neuro-Fuzzy Network for Nonlinear System Control

Jyun-Yu Jhang 1 ID , Kuang-Hui Tang 2,3 ID , Chuan-Kuei Huang 2, Cheng-Jian Lin 4,* and Kuu-Young Young 1 1 Institute of Electrical and Control Engineering, National Chiao Tung University, Hsinchu City 300, Taiwan; o800920@gmail.com (J.-Y.J.); kyoung@mail.nctu.edu.tw (K.-Y.Y.) 2 Department of Industrial Education and Technology, National Changhua University of Education, Changhua County 500, Taiwan; tkhf14@ncut.edu.tw (K.-H.T.); ckhuang@cc.ncue.edu.tw (C.-K.H.) 3 Department of Electronic Engineering, National Chin-Yi University of Technology, Taichung City 406, Taiwan 4 Department of Computer Science & Information Engineering, National Chin-Yi University of Technology, Taichung City 406, Taiwan * Correspondence: cjlin@ncut.edu.tw; Tel.: +886-4-2392-4505 (ext. 8753)


Introduction
Neural fuzzy networks (NFNs) have been widely applied in various fields [1][2][3].Traditional NFNs combine neural networks to learn from processes with fuzzy reasoning to handle uncertain information.These can only be applied to parameter learning based on the ordered derivative algorithm where the structure of the NFNs has been determined and fixed in advance [4][5][6].In [7,8], a neuro-fuzzy system could learn system behavior from the training data and automatically generate fuzzy rules and fuzzy sets to a prespecified accuracy level.The major disadvantage of the existing neural fuzzy networks is that their application is limited to static problems as a result of their internal feedforward network structure.For TSK-type neural fuzzy networks (TNFNs), the consequent part of each fuzzy rule is a linear combination of the input variable.However, the traditional TNFN cannot use the mapping capabilities of the linear function combination in consequent parts of the fuzzy rules.Hence, the FNFN model, which combines a neuro-fuzzy network with a FLNN [9], was proposed to improve The jth fuzzy if-then rule in FNFN model is described as follows.
IF  1      2   2 ⋯       ⋯      ,    = ∑   ∅    = 1 (1) where xi represents the input; yj is the output of the jth fuzzy rule; Bij represents the membership function; wkj denotes the link weight; N denotes the number of input variables; M presents the basis function number; and ∅k represents the trigonometric polynomial function combination of input variables.
Next, we describe the FNFN architecture layer by layer.In layer 1, no operation exists and the input signals transmit to the second layer directly: (1) ii ux  (2) where Aij presents a membership function.In layer 2, we adopted a Gaussian membership function for FNFN, which had the following advantages: (1) a small number of parameters are needed to define; (2) better robustness; and (3) the performance is superior than polygonal membership functions.The degree of the membership function is calculated um u (3) where mij and σij represent the expected value and variance, respectively.
In layer 3, the product operator is used to achieve the conditional part in the fuzzy rules.Outputs are described as follows: The jth fuzzy if-then rule in FNFN model is described as follows.
where x i represents the input; y j is the output of the jth fuzzy rule; B ij represents the membership function; w kj denotes the link weight; N denotes the number of input variables; M presents the basis function number; and ∅ k represents the trigonometric polynomial function combination of input variables.Next, we describe the FNFN architecture layer by layer.In layer 1, no operation exists and the input signals transmit to the second layer directly: where A ij presents a membership function.In layer 2, we adopted a Gaussian membership function for FNFN, which had the following advantages: (1) a small number of parameters are needed to define; (2) better robustness; and (3) the performance is superior than polygonal membership functions.The degree of the membership function is calculated where m ij and σ ij represent the expected value and variance, respectively.In layer 3, the product operator is used to achieve the conditional part in the fuzzy rules.Outputs are described as follows: u (3) where ij denotes the inference of its corresponding rule.In Figure 1, the outputs of layer 3 were used as the inputs of layer 4, and the other inputs in layer 4 were from the outputs of a FLNN.The node in layer 4 is illustrated as follows: The functional expansion (F.E.) adopts a trigonometric polynomial basis function and is described by ∅ k = [x 1 ,sin(πx 1 ), cos(πx 1 ), x 2 ,sin(πx 2 ),cos(πx 2 ), x 1 x 2 ] for two-dimensional input variables.
In layer 5, the output of the FNFN is a defuzzification operation where R and y represent the fuzzy rule number and the output of the FNFN model, respectively.

Proposed Learning Algorithm
In this study, the proposed learning algorithm was comprised of structure learning and parameter learning.The flowchart of the proposed learning algorithm is shown in Figure 2. By satisfying the fuzzy partitioning of the input variables, the entropy measurement was used to decide the fuzzy rule number in the structure learning.In parameter learning, the gradient descent method was used to minimize the error function by adjusting the parameters in the FNFN.
Electronics 2018, 7, x FOR PEER REVIEW 4 of 22 where  ij i u denotes the inference of its corresponding rule.
In Figure 1, the outputs of layer 3 were used as the inputs of layer 4, and the other inputs in layer 4 were from the outputs of a FLNN.The node in layer 4 is illustrated as follows: 1 The functional expansion (F.E.) adopts a trigonometric polynomial basis function and is described by ∅k = [x1,sin(πx1), cos(πx1), x2,sin(πx2),cos(πx2), x1 x2] for two-dimensional input variables.
In layer 5, the output of the FNFN is a defuzzification operation where R and y represent the fuzzy rule number and the output of the FNFN model, respectively.

Proposed Learning Algorithm
In this study, the proposed learning algorithm was comprised of structure learning and parameter learning.The flowchart of the proposed learning algorithm is shown in Figure 2. By satisfying the fuzzy partitioning of the input variables, the entropy measurement was used to decide the fuzzy rule number in the structure learning.In parameter learning, the gradient descent method was used to minimize the error function by adjusting the parameters in the FNFN.

Structure Learning
In structure learning, the generated fuzzy rules are decided by the training data.The entropy measurement is adopted to measure the similarity between each membership function and each input data.If the input data is close to the mean of a membership function, it has a lower entropy value.This means that the entropy values are computed to decide whether or not a new fuzzy rule is added.The entropy measurement is determined by the firing degree. where ) and S j is between zero and one.The maximum entropy measurement is described as follows: where R represents the current rule number.If S max ≤ S, then a new fuzzy rule is added.The value of S is a pre-defined threshold value and is between zero and one.Its value will decay during the learning process.During structure learning, the S value is an important parameter to determine whether a new fuzzy rule is generated.In general, the S value is pre-defined as 0.3 × N, where N denotes the number of inputs.
If a new fuzzy rule has been added, the initial expected value, variance, and weights of a new generated fuzzy rule are determined in the next step.As the learning target is to minimize the error function, the expected value, variance, and weights are adjusted as follows: where x i represents the current input data and σ init denotes a pre-defined value.

Parameter Learning
According to the current input data, the structure of the FNFN model has been adjusted.Next, the model goes into parameter learning to adjust the parameters of the FNFN model based on the same input data.The goal of parameter learning is to minimize the error function.The gradient descent method is used for this backpropagation (BP) learning.For a single output condition, the target of BP is to minimize the error function as follows: where y d (t) and y(t) are the goal output and the actual output for time t, respectively.The parameters of the FNFN model can be adjusted by using the BP learning algorithm and are defined as follows: where η denotes the learning rate.W = [m, σ, w] T denotes the adjustable parameters of the FNFN.The BP of the adjustable parameters W is derived as follows: The adjustable parameters in the FNFN model are adjusted by the chain rule in each layer.The updating rule for w j is derived as follows: Similarly, the updating rule for m ij and σ ij are derived as follows: ) where η w , η m , and η σ are the learning rates of the weight, the expected value, and the variance, respectively.

FPGA Implementation of the FNFN Controller
This section introduces the overall hardware detail design and implementation of the FNFN controller.This section illustrates the represented fixed-point data format.The various function units are implemented by Taylor expansion and look-up table (LUT) methods including Gaussian function, sine function, and cosine function.In this section, the hardware implementation overall components of the FNFN controller are also introduced and are shown in Similarly, the updating rule for mij and σij are derived as follows: where ηw, ηm, and ησ are the learning rates of the weight, the expected value, and the variance, respectively.

FPGA Implementation of the FNFN Controller
This section introduces the overall hardware detail design and implementation of the FNFN controller.This section illustrates the represented fixed-point data format.The various function units are implemented by Taylor expansion and look-up table (LUT) methods including Gaussian function, sine function, and cosine function.In this section, the hardware implementation overall components of the FNFN controller are also introduced and are shown in

The Represented Data Format
In order to keep the represented data format consistent in the FNFNs, a fixed-point number was adopted.An encoding technology adopted digital values to illustrate the represented data [8].The represented fixed-point data format can be denoted as follows: [] •  (18) where b represents a sign bit.If b is equal to 0, the value is a positive number, whereas if b is equal to 1, the value is a negative number.i and f denote the numbers of integer bits and fractional bits,

The Represented Data Format
In order to keep the represented data format consistent in the FNFNs, a fixed-point number was adopted.An encoding technology adopted digital values to illustrate the represented data [8].The represented fixed-point data format can be denoted as follows: where b represents a sign bit.If b is equal to 0, the value is a positive number, whereas if b is equal to 1, the value is a negative number.i and f denote the numbers of integer bits and fractional bits, respectively [9,10].Twenty bits were adopted as the number of a word length in this study and had more accuracy than 16 bits.The fixed-point data format included 1 sign bit, 6 integer bits, and 13 fractional bits.

Design and Implementation of Function Unit
First of all, in the process of hardware implementation of the FNFN, the problem in the implemented exponential of the Gaussian function of the TSK-type fuzzy model (Figure 3), sine function of FLNN, and cosine function of FLNN will occur (Figure 4).The functions are complex and not easily accomplished directly with FPGA implementation.Therefore, the LUT and Taylor expansion were used to approach the Gaussian function, sine function, and cosine function.

Design and Implementation of Function Unit
First of all, in the process of hardware implementation of the FNFN, the problem in the implemented exponential of the Gaussian function of the TSK-type fuzzy model (Figure 3), sine function of FLNN, and cosine function of FLNN will occur (Figure 4).The functions are complex and not easily accomplished directly with FPGA implementation.Therefore, the LUT and Taylor expansion were used to approach the Gaussian function, sine function, and cosine function.The Taylor expansion is shown in Equations ( 19)- (21), and its advantage is that it only utilizes simple operations which can accurately answer.The disadvantage of LUT is that it must obtain the correspondence of the input and output value of each datum in advance when setting up the table, so it takes a much longer time.
where x is the input variable and n is the number of order.If n is a large order, a more accurate approach will be obtained.In this case, it needs more memory spaces and logic gates.Table 1 shows the error value of the exponential function and Taylor approximation as the higher order requires more complicated operations.In this study, we chose order n = 3 to achieve the e x , sin x, and cos x hardware.The Taylor expansion is shown in Equations ( 19)-( 21), and its advantage is that it only utilizes simple operations which can accurately answer.The disadvantage of LUT is that it must obtain the correspondence of the input and output value of each datum in advance when setting up the table, so it takes a much longer time.
where x is the input variable and n is the number of order.If n is a large order, a more accurate approach will be obtained.In this case, it needs more memory spaces and logic gates.Table 1 shows the error value of the exponential function and Taylor approximation as the higher order requires more complicated operations.In this study, we chose order n = 3 to achieve the e x , sin x, and cos x hardware.

Gaussian Function Implementation
According to the operation of exponential function in Gaussian function, we utilized the Taylor expansion to implement the exponential function in FPGA.This way, we could implement the hardware, which spends reasonable gate counts.We used the multiplier operation to make x 2 and x 3 two values, and divided these values by 2!, and 3!, respectively.In the Taylor expansion, 1, 1!, 2!, and 3! are four constant regular values, so we undertook the operation of these four values at the beginning, and then put them into the Gaussian function circuit of the FPGA.Finally, we used an adder tree to obtain a similar result of the Taylor expansion.In Figure 5, the block diagram shows the implementation of the exponential function.The hardware implementation of the Gaussian function in layer 2 of the TSK-type fuzzy model is shown in Figure 6. Figure 7 shows the Gaussian function and its Taylor approximation.In the range of operation, the outputting value of Gaussian function can approach the amount outcomes of software in FPGA.However, the input values are not accurate in the Taylor expansion.The input value is either a large positive value or a large negative value.According to the aforementioned problem, the LUT was utilized to compensate for the error.Therefore, regardless of whether the input shows a large positive value or a large negative value, the LUT supported the values to the multiplexer automatically.In Figure 8, we can see that the block diagram shows the implementation of the exponential function with the Taylor expansion and LUT.The block diagram is the implementation of Gaussian function with Taylor expansion and LUT as shown in Figure 9.In Figure 10, the Gaussian function is compared with its Taylor expansion and LUT approximation.Comparing Figure 7 with Figure 10, we can see that the Gaussian function approximate was more accurate than using the Taylor expansion and LUT.

Gaussian Function Implementation
According to the operation of exponential function in Gaussian function, we utilized the Taylor expansion to implement the exponential function in FPGA.This way, we could implement the hardware, which spends reasonable gate counts.We used the multiplier operation to make x 2 and x 3 two values, and divided these values by 2!, and 3!, respectively.In the Taylor expansion, 1, 1!, 2!, and 3! are four constant regular values, so we undertook the operation of these four values at the beginning, and then put them into the Gaussian function circuit of the FPGA.Finally, we used an adder tree to obtain a similar result of the Taylor expansion.In Figure 5, the block diagram shows the implementation of the exponential function.The hardware implementation of the Gaussian function in layer 2 of the TSK-type fuzzy model is shown in Figure 6. Figure 7 shows the Gaussian function and its Taylor approximation.In the range of operation, the outputting value of Gaussian function can approach the amount outcomes of software in FPGA.However, the input values are not accurate in the Taylor expansion.The input value is either a large positive value or a large negative value.According to the aforementioned problem, the LUT was utilized to compensate for the error.Therefore, regardless of whether the input shows a large positive value or a large negative value, the LUT supported the values to the multiplexer automatically.In Figure 8, we can see that the block diagram shows the implementation of the exponential function with the Taylor expansion and LUT.The block diagram is the implementation of Gaussian function with Taylor expansion and LUT as shown in Figure 9.In Figure 10, the Gaussian function is compared with its Taylor expansion and LUT approximation.Comparing Figure 7 with Figure 10, we can see that the Gaussian function approximate was more accurate than using the Taylor expansion and LUT.

Gaussian Function Implementation
According to the operation of exponential function in Gaussian function, we utilized the Taylor expansion to implement the exponential function in FPGA.This way, we could implement the hardware, which spends reasonable gate counts.We used the multiplier operation to make x 2 and x 3 two values, and divided these values by 2!, and 3!, respectively.In the Taylor expansion, 1, 1!, 2!, and 3! are four constant regular values, so we undertook the operation of these four values at the beginning, and then put them into the Gaussian function circuit of the FPGA.Finally, we used an adder tree to obtain a similar result of the Taylor expansion.In Figure 5, the block diagram shows the implementation of the exponential function.The hardware implementation of the Gaussian function in layer 2 of the TSK-type fuzzy model is shown in Figure 6. Figure 7 shows the Gaussian function and its Taylor approximation.In the range of operation, the outputting value of Gaussian function can approach the amount outcomes of software in FPGA.However, the input values are not accurate in the Taylor expansion.The input value is either a large positive value or a large negative value.According to the aforementioned problem, the LUT was utilized to compensate for the error.Therefore, regardless of whether the input shows a large positive value or a large negative value, the LUT supported the values to the multiplexer automatically.In Figure 8, we can see that the block diagram shows the implementation of the exponential function with the Taylor expansion and LUT.The block diagram is the implementation of Gaussian function with Taylor expansion and LUT as shown in Figure 9.In Figure 10, the Gaussian function is compared with its Taylor expansion and LUT approximation.Comparing Figure 7 with Figure 10, we can see that the Gaussian function approximate was more accurate than using the Taylor expansion and LUT.

Sine Function and Cosine Function Implementation
Based on the operation of sine and cosine functions, the same method of the Taylor expansion was utilized to implementation sine and cosine functions in FPGA.Figures 11 and 12 show the implementation of sine and cosine functions with Taylor expansion.Figures 13 and 14 show the results of cosine function and sine function by the Taylor approximation.In the range of operation, the output value of the cosine function and sine function could approach the outcome of software in FPGA.However, some accurate values in the Taylor expansion will occur, such as a large positive or negative input.According to the aforementioned problem, we utilized the LUT to compensate for this error.Therefore, if the input had a large positive or negative value, the LUT will automatically support the values to the multiplexer.Figures 15 and 16 show the implementation of the cosine and sine functions with the Taylor expansion and LUT.In Figures 17a and 18a, the cosine function and sine function approach was more accurate by using the Taylor expansion and LUT.Figures 17b and  18b show the error rate and the MSE used to estimate the error rate.The MSE of the Taylor and LUT approximation of the sine function and cosine function were equal to 0.00003922 and 0.00041378, respectively.

Sine Function and Cosine Function Implementation
Based on the operation of sine and cosine functions, the same method of the Taylor expansion was utilized to implementation sine and cosine functions in FPGA.Figures 11 and 12 show the implementation of sine and cosine functions with Taylor expansion.Figures 13 and 14 show the results of cosine function and sine function by the Taylor approximation.In the range of operation, the output value of the cosine function and sine function could approach the outcome of software in FPGA.However, some accurate values in the Taylor expansion will occur, such as a large positive or negative input.According to the aforementioned problem, we utilized the LUT to compensate for this error.Therefore, if the input had a large positive or negative value, the LUT will automatically support the values to the multiplexer.Figures 15 and 16 show the implementation of the cosine and sine functions with the Taylor expansion and LUT.In Figures 17a and 18a, the cosine function and sine function approach was more accurate by using the Taylor expansion and LUT.Figures 17b and  18b show the error rate and the MSE used to estimate the error rate.The MSE of the Taylor and LUT approximation of the sine function and cosine function were equal to 0.00003922 and 0.00041378,

Sine Function and Cosine Function Implementation
Based on the operation of sine and cosine functions, the same method of the Taylor expansion was utilized to implementation sine and cosine functions in FPGA.Figures 11 and 12 show the implementation of sine and cosine functions with Taylor expansion.Figures 13 and 14 show the results of cosine function and sine function by the Taylor approximation.In the range of operation, the output value of the cosine function and sine function could approach the outcome of software in FPGA.However, some accurate values in the Taylor expansion will occur, such as a large positive or negative input.According to the aforementioned problem, we utilized the LUT to compensate for this error.Therefore, if the input had a large positive or negative value, the LUT will automatically

Sine Function and Cosine Function Implementation
Based on the operation of sine and cosine functions, the same method of the Taylor expansion was utilized to implementation sine and cosine functions in FPGA.Figures 11 and 12 show the implementation of sine and cosine functions with Taylor expansion.Figures 13 and 14 show the results of cosine function and sine function by the Taylor approximation.In the range of operation, the output value of the cosine function and sine function could approach the outcome of software in FPGA.However, some accurate values in the Taylor expansion will occur, such as a large positive or negative input.According to the aforementioned problem, we utilized the LUT to compensate for this error.Therefore, if the input had a large positive or negative value, the LUT will automatically support the values to the multiplexer.Figures 15 and 16 show the implementation of the cosine and sine functions with the Taylor expansion and LUT.In Figures 17a and 18a, the cosine function and sine function approach was more accurate by using the Taylor expansion and LUT.Figures 17b and 18b show the error rate and the MSE used to estimate the error rate.The MSE of the Taylor and LUT approximation of the sine function and cosine function were equal to 0.00003922 and 0.00041378, respectively.

Sine Function and Cosine Function Implementation
Based on the operation of sine and cosine functions, the same method of the Taylor expansion was utilized to implementation sine and cosine functions in FPGA.Figures 11 and 12 show the implementation of sine and cosine functions with Taylor expansion.Figures 13 and 14 show the results of cosine function and sine function by the Taylor approximation.In the range of operation, the output value of the cosine function and sine function could approach the outcome of software in FPGA.However, some accurate values in the Taylor expansion will occur, such as a large positive or negative input.According to the aforementioned problem, we utilized the LUT to compensate for this error.Therefore, if the input had a large positive or negative value, the LUT will automatically support the values to the multiplexer.Figures 15 and 16 show the implementation of the cosine and sine functions with the Taylor expansion and LUT.In Figures 17a and 18a, the cosine function and sine function approach was more accurate by using the Taylor expansion and LUT.Figures 17b and  18b show the error rate and the MSE used to estimate the error rate.The MSE of the Taylor and LUT approximation of the sine function and cosine function were equal to 0.00003922 and 0.00041378, respectively.

Input Fuzzifier
The fuzzification operator is implemented in Equation ( 3).The Gaussian function in this module is the main structure of the fuzzy rules in the FNFN.The implementation of the Gaussian membership function in Equation ( 3) is complex by the conventional active functions.Therefore, the Taylor expansion and LUT were used to approach the Gaussian membership function.A detailed description is given in Section 5.2.1.In Section 5.2.1, Figure 9 shows the block diagram of the Gaussian

Input Fuzzifier
The fuzzification operator is implemented in Equation ( 3).The Gaussian function in this module is the main structure of the fuzzy rules in the FNFN.The implementation of the Gaussian membership function in Equation ( 3) is complex by the conventional active functions.Therefore, the Taylor expansion and LUT were used to approach the Gaussian membership function.A detailed description is given in Section 5.2.1.In Section 5.2.1, Figure 9 shows the block diagram of the Gaussian function with Taylor expansion and LUT.Four multipliers, a subtracter, three dividers, a multiplexer, and an adder were used.In all of the components, the multiplier and divider were the main parts of the hardware implementation.

Inference Processing Unit
The multiplication operation in Equation ( 4) was performed in the inference processing unit.Figure 19 shows the block diagram of the inference processing unit.function with Taylor expansion and LUT.Four multipliers, a subtracter, three dividers, a multiplexer, and an adder were used.In all of the components, the multiplier and divider were the main parts of the hardware implementation.

Inference Processing Unit
The multiplication operation in Equation ( 4) was performed in the inference processing unit.Figure 19 shows the block diagram of the inference processing unit.

Consequence Unit
The main work of the consequence unit is to carry out the operation of consequence nodes in Equation ( 5).In layer 4, the nodes of this layer represent the consequent nodes.The inputs of layer 4 are the outputs of a FLNN and layer 3. The consequence unit module is built by multipliers and FLNN and is illustrated in Figure 20. Figure 4 presents the hardware architecture of FLNN.

Output Defuzzifier
This module implements Equation ( 6) and the block diagram is shown in Figure 21.First, the signal (3) j u and signal (4)   j u adopted the adder operation to sum all the values, respectively.After being added, the sum of

Consequence Unit
The main work of the consequence unit is to carry out the operation of consequence nodes in Equation (5).In layer 4, the nodes of this layer represent the consequent nodes.The inputs of layer 4 are the outputs of a FLNN and layer 3. The consequence unit module is built by multipliers and FLNN and is illustrated in Figure 20. Figure 4 presents the hardware architecture of FLNN.
Electronics 2018, 7, x FOR PEER REVIEW 12 of 22 function with Taylor expansion and LUT.Four multipliers, a subtracter, three dividers, a multiplexer, and an adder were used.In all of the components, the multiplier and divider were the main parts of the hardware implementation.

Inference Processing Unit
The multiplication operation in Equation ( 4) was performed in the inference processing unit.Figure 19 shows the block diagram of the inference processing unit.

Consequence Unit
The main work of the consequence unit is to carry out the operation of consequence nodes in Equation ( 5).In layer 4, the nodes of this layer represent the consequent nodes.The inputs of layer 4 are the outputs of a FLNN and layer 3. The consequence unit module is built by multipliers and FLNN and is illustrated in Figure 20. Figure 4 presents the hardware architecture of FLNN.

Output Defuzzifier
This module implements Equation ( 6) and the block diagram is shown in Figure 21.First, the signal (3) j u and signal (4)   j u adopted the adder operation to sum all the values, respectively.After being added, the sum of

FPGA Development Platform
The hardware implementation of a FNFN controller was built in SMIMS VeriEnterprise ® .Figure 22

Experimental Results
To verify the control performance of the FNFN, two control problems were tested in the experiments.The FNFN controller was used for solving the temperature control of a water bath [11] and the backing control of a car [22].The design of the FNFN controller was programmed by ISE 9.1i software and used MATLAB 7.0 software for the example.The FNFN controller as implemented in

FPGA Development Platform
The hardware implementation of a FNFN controller was built in SMIMS VeriEnterprise ® .Figure 22

FPGA Development Platform
The hardware implementation of a FNFN controller was built in SMIMS VeriEnterprise ® .Figure 22

Experimental Results
To verify the control performance of the FNFN, two control problems were tested in the experiments.The FNFN controller was used for solving the temperature control of a water bath [11] and the backing control of a car [22].The design of the FNFN controller was programmed by ISE 9.1i software and used MATLAB 7.0 software for the example.The FNFN controller as implemented in

Experimental Results
To verify the control performance of the FNFN, two control problems were tested in the experiments.The FNFN controller was used for solving the temperature control of a water bath [11] and the backing control of a car [22].The design of the FNFN controller was programmed by ISE 9.1i software and used MATLAB 7.0 software for the example.The FNFN controller as implemented in the SMIMS VeriEnterprise ® FPGA development platform with a Xilinx Virtex 4-XC4VLX60 chip [12], and its clock rate was set to 50 MHz for the two examples.

Temperature Control of a Water Bath
The objective of this experiment was to implement the temperature control of a water bath by the FNFN.The dynamic equation of a water bath is described as follows: where k is the discrete-time index; u(k) and y(k) denote the system input and output, respectively; and Ts is the sampling period.α and δ are constant values.
In this experiment, the parameters of the water bath plan were δ = 8.67973 × 10 −3 , α = 1.0015 × 10 −4 , and y 0 = 25.0 ( • C), which were obtained from a real water bath plant [11].The input u(k) was limited to 0, and V represents the voltage unit.The sampling period was Ts = 30.
A schematic diagram of the temperature control of the water bath is shown in Figure 23.This program has two processes: The training process and the control process.In the training process, switches S1 and S2 were connected to nodes 1 and 2, respectively, to form a training loop.In this loop, the training data with input vector I(k) = [y P (k + 1)y P (k)] and desired output u(k) were defined, where the inputs of the FNFN controller were the same as that used in the inverse modeling [13].In the control process, switches S1 and S2 were connected to nodes 3 and 4, respectively, to form a control loop.In this loop, the control signal û(k) was generated according to the input vector I'(k) = [y ref (k + 1)y P (k)], where y P is the plant output and y ref is the reference model output.the SMIMS VeriEnterprise ® FPGA development platform with a Xilinx Virtex 4-XC4VLX60 chip [12], and its clock rate was set to 50 MHz for the two examples.

Temperature Control of a Water Bath
The objective of this experiment was to implement the temperature control of a water bath by the FNFN.The dynamic equation of a water bath is described as follows: where k is the discrete-time index; u(k) and y(k) denote the system input and output, respectively; and Ts the sampling period.α and δ are constant values.
In this experiment, the parameters of the water bath plan were δ = 8.67973 × 10 −3 , α = 1.0015 × 10 −4 , and y0 = 25.0 (°C), which were obtained from a real water bath plant [11].The input u(k) was limited to 0, and 5 V represents the voltage unit.The sampling period was Ts = 30.
A schematic diagram of the temperature control of the water bath is shown in Figure 23.This program has two processes: The training process and the control process.In the training process, switches S1 and S2 were connected to nodes 1 and 2, respectively, to form a training loop.In this loop, the training data with input vector I(k) = [yP(k + 1)yP(k)] and desired output u(k) were defined, where the inputs of the FNFN controller were the same as that used in the inverse modeling [13].In the control process, switches S1 and S2 were connected to nodes 3 and 4, respectively, to form a control loop.In this loop, the control signal ˆ() uk was generated according to the input vector where yP is the plant output and yref is the reference model output.The random input signals urd(k) were constrained between [0, 5] V and infused directly into the dynamic equation of the water bath as described in Equation (22).Based on the input-output characteristics to cover the entire reference output, 120 training patterns were selected.The initial water temperature was set to 25 °C.When a random input signal was infused, the temperature of the water rose progressively.After 15,000 training iterations, four fuzzy rules were generated.The obtained fuzzy rules in FNFN are as follows.
Rule 1:  The random input signals u rd (k) were constrained between [0, 5] V and infused directly into dynamic equation of the water bath as described in Equation ( 22).Based on the input-output characteristics to cover the entire reference output, 120 training patterns were selected.The initial water temperature was set to 25 • C. When a random input signal was infused, the temperature of the water rose progressively.After 15,000 training iterations, four fuzzy rules were generated.The obtained fuzzy rules in FNFN are as follows.
Rule 1: In this study, we compared the control performance of the proposed FNFN with those of the Takagi-Sugeno-Kang (TSK)-type neural fuzzy network (NFN) [2], the functional link neural network (FLNN) [4], the proportional-integral-derivative (PID) controller [23], and the manually designed fuzzy controller [24].The performance evaluation consisted of the regulation of set-points, the noise influence, and the tracking capability.
The first work was to evaluate the regulation of three points.
The results of the regulation of the set-points using the FNFN controller are shown in Figure 24a.Figure 24b illustrates the errors between the FNFN output and desired output.The performance evaluation adopted the sum of absolute error (E) and is described as follows: where y(k) and y ref (k) are the actual output and desired output, respectively.In this study, we compared the control performance of the proposed FNFN with those of the Takagi-Sugeno-Kang (TSK)-type neural fuzzy network (NFN) [2], the functional link neural network (FLNN) [4], the proportional-integral-derivative (PID) controller [23], and the manually designed fuzzy controller [24].The performance evaluation consisted of the regulation of set-points, the noise influence, and the tracking capability.
The first work was to evaluate the regulation of three points.
() = { 35  ,   ≤ 40 55  ,  40 <  ≤ 80 75  ,  80 <  ≤ 120 (23) The results of the regulation of the set-points using the FNFN controller are shown in Figure 24a.Figure 24b illustrates the errors between the FNFN output and desired output.The performance evaluation adopted the sum of absolute error (E) and is described as follows:  = ∑|  () − ()|  (24) where y(k) and yref(k) are the actual output and desired output, respectively.The second experiment was to obtain the noise influence of the FNFN controller.At the 60th sampling time, a noise value was added to the output of the water bath system (i.e., −5 • C).This experiment adopted a pre-set temperature of 50 • C. For the noise influence, the outputs and the corresponding errors of the FNFN controller are presented in Figure 25a,b, respectively.After the occurrence of the noise influence, the proposed FNFN controller had a very quick and steady recovery.The second experiment was to obtain the noise influence of the FNFN controller.At the 60th sampling time, a noise value was added to the output of the water bath system (i.e., −5 °C).This experiment adopted a pre-set temperature of 50 °C.For the noise influence, the outputs and the corresponding errors of the FNFN controller are presented in Figure 25a,b, respectively.After the occurrence of the noise influence, the proposed FNFN controller had a very quick and steady recovery.The parameters of many industrial-control processes will be altered in an irregular way.The third experiment was to add a 0.7 × u(k−2) value to the input of the water bath system after the 60th sampling time for testing the robustness of the FNFN controller.This experiment adopted a pre-set temperature (i.e., 50 °C).When the plant dynamics were altered, the outputs and the corresponding errors of the FNFN controller are shown in Figure 26a,b, respectively.The training process continued for 125 epochs.Figure 27 shows the learning curve of the FNFN controller in the water bath system.The parameters of many industrial-control processes will be altered in an irregular way.The third experiment was to add a 0.7 × u(k−2) value to the input of the water bath system after the 60th sampling time for testing the robustness of the FNFN controller.This experiment adopted a pre-set temperature (i.e., 50 • C).When the plant dynamics were altered, the outputs and the corresponding errors of the FNFN controller are shown in Figure 26a,b, respectively.The training process continued for 125 epochs.Figure 27 shows the learning curve of the FNFN controller in the water bath system.The second experiment was to obtain the noise influence of the FNFN controller.At the 60th sampling time, a noise value was added to the output of the water bath system (i.e., −5 °C).This experiment adopted a pre-set temperature of 50 °C.For the noise influence, the outputs and the corresponding errors of the FNFN controller are presented in Figure 25a,b, respectively.After the occurrence of the noise influence, the proposed FNFN controller had a very quick and steady recovery.The parameters of many industrial-control processes will be altered in an irregular way.The third experiment was to add a 0.7 × u(k−2) value to the input of the water bath system after the 60th sampling time for testing the robustness of the FNFN controller.This experiment adopted a pre-set temperature (i.e., 50 °C).When the plant dynamics were altered, the outputs and the corresponding errors of the FNFN controller are shown in Figure 26a,b, respectively.The training process continued for 125 epochs.Figure 27 shows the learning curve of the FNFN controller in the water bath system.In the final experiment, the tracking capability of the proposed FNFN was proven.The ramp-reference signals are defined as follows:   The sums of the absolute error (E) of the FNFN controller with software implementation and with hardware implementation, the Takagi-Sugeno-Kang (TSK)-type neural fuzzy network (NFN) [2], the functional link neural network (FLNN) [4], the proportional-integral-derivative (PID) controller [23], and the manually designed fuzzy controller [24] are shown in Table 2.The experimental results showed that the trained FNFN controller had better noise rejection capabilities and tracking control performance than the other methods in the temperature control of the water bath.The hardware implementation of the FNFN with four fuzzy logic rules needed to use about 507,064 logic gates.The resource requirements for the example architecture with FNFN implementation are shown in Table 3.In this table, "Available" represents the various resources of the chip present; "Used" represents the resources utilized in our implementation, and "Utilization" represents the percentage of resources utilized.Experimental results showed that the proposed method obtained perfect control capability.The sums of the absolute error (E) of the FNFN controller with software implementation and with hardware implementation, the Takagi-Sugeno-Kang (TSK)-type neural fuzzy network (NFN) [2], the functional link neural network (FLNN) [4], the proportional-integral-derivative (PID) controller [23], and the manually designed fuzzy controller [24] are shown in Table 2.The experimental results showed that the trained FNFN controller had better noise rejection capabilities and tracking control performance than the other methods in the temperature control of the water bath.The hardware implementation of the FNFN with four fuzzy logic rules needed to use about 507,064 logic gates.The resource requirements for the example architecture with FNFN implementation are shown in Table 3.In this table, "Available" represents the various resources of the chip present; "Used" represents the resources utilized in our implementation, and "Utilization" represents the percentage of resources utilized.Experimental results showed that the proposed method obtained perfect control capability.

Backing Control of a Car
As the backing control of a car is a complex control problem, the traditional control method is difficult to implement [22].The loading zone and car are presented in Figure 29.The ϕ, x, and y variables were used to decide the car position.The (x, y) denotes the center position of the car and ϕ presents the angle between the horizontal axis and the car.The steering angle of the car (θ) is the controlled variable.The objective of this control was to move the car to the desired dock (x desired , y desired ) at ϕ desired = 90

Backing Control of a Car
As the backing control of a car is a complex control problem, the traditional control method is difficult to implement [22].The loading zone and car are presented in Figure 29.The φ, x, and y variables were used to decide the car position.The (x, y) denotes the center position of the car and φ presents the angle between the horizontal axis and the car.The steering angle of the car (θ) is the controlled variable.The objective of this control was to move the car to the desired dock (xdesired, ydesired) at φdesired = 90°.A fixed distance (db) of the car was moved backwards at each time step.The plane [0, 100] × [0, 100] represents the limited loading region.
where dl denotes the car length.
After BP training processing, four fuzzy rules were generated and the parameters of the membership functions also determined in the FNFN controller.Therefore, the total number of adjustable parameters was 44.The training process continued for 500 iterations.The RMS errors of The angle ϕ and cross position x of the car are two inputs of the proposed FNFN, whereas the steering angle θ presents the output of the proposed FNFN.The ranges of parameters (x, ∅, and θ) are described as follows.
0 ≤ x ≤ 100 (26) The dynamical equations of the car are where d l denotes the car length.After BP training processing, four fuzzy rules were generated and the parameters of the membership functions also determined in the FNFN controller.Therefore, the total number of adjustable parameters was 44.The training process continued for 500 iterations.The RMS errors of the FNFN were approximately 0.0329.There were four fuzzy rules generated, which are shown as follows. Rule Figure 30 presents the FNFN controller learning curve and Figure 31a-d show the moving car trajectories of the trained FNFN controller with the four different initial positions.The RMSE is adopted to evaluate the FNFN performance; Table 4 shows the RMSE of the four different initial positions.Experimental results showed that the trained FNFN could be successfully reloaded from different initial positions.
the FNFN were approximately 0.0329.There were four fuzzy rules generated, which are shown as follows.
Rule 1: Figure 30 presents the FNFN controller learning curve and Figure 31a-d show the moving car trajectories of the trained FNFN controller with the four different initial positions.The RMSE is adopted to evaluate the FNFN performance; Table 4 shows the RMSE of the four different initial positions.Experimental results showed that the trained FNFN could be successfully reloaded from different initial positions.The hardware implementation of the FNFN with four fuzzy logic rules needed to use about 507,064 logic gates.Table 5 illustrates the resource requirements for the control architecture of backing up the truck with FNFN implementation.In this table, "Available" represents the various resources of the chip present; "Used" represents the resources utilized in our implementation, and "Utilization" represents the percentage of resources utilize.Experimental results showed that the proposed method obtained perfect control capability.The hardware implementation of the FNFN with four fuzzy logic rules needed to use about 507,064 logic gates.Table 5 illustrates the resource requirements for the control architecture of backing up the truck with FNFN implementation.In this table, "Available" represents the various resources of the chip present; "Used" represents the resources utilized in our implementation, and "Utilization" represents the percentage of resources utilize.Experimental results showed that the proposed method obtained perfect control capability.

Conclusions and Future Works
This study presented the hardware implementations of FNFN using Xilinx FPGAs for solving nonlinear control problems.The proposed FNFN uses a functional link neural network as the conclusion part of a fuzzy rule, which has a nonlinear combination of inputs.In addition, an efficient learning algorithm was proposed to construct the architecture of the FNFN using structure learning, and adjust the parameters using parameter learning.The main advantage of the FNFN is a reduced computational cost in the training stage, while maintaining the approximation performance of the multi-layer perceptron network.However, FNFN has the disadvantage of an increased number of rules as it cannot automatically generate the optimum rule numbers and merge similar rules.In order to have high speed processing and real-time operating capability, using hardware implementation is necessary.
To evaluate the control performance of the proposed FNFN, two experiments including the temperature control of a water bath and the backing control of a car were performed.Finally, the performances of the experimental results successfully confirmed the validity of using FPGA implementation for the FNFN controller.In future work, we will decrease the resource requirements for designing the model and implement the learning algorithm in FPGA.In addition, a type-2 fuzzy set incorporates uncertainty about the membership function into fuzzy set theory.Many reported results have shown that the type-2 neural fuzzy network is better able to handle uncertainties than the type-1 neural fuzzy network.In future work, we will consider type-2 fuzzy sets to improve the structure of the type-1 neural fuzzy network and implement this using hardware.

Figure 2 .
Figure 2. Flowchart of the proposed learning algorithm.

Figure 2 .
Figure 2. Flowchart of the proposed learning algorithm.

Figure 3 .
Four main parts can be described as follows: (A) Input fuzzifier; (B) Inference processing unit; (C) Consequent unit; and (D) Output defuzzifier.Finally, the FPGA development platform is introduced.

Figure 3 .
Four main parts can be described as follows: (A) Input fuzzifier; (B) Inference processing unit; (C) Consequent unit; and (D) Output defuzzifier.Finally, the FPGA development platform is introduced.

Figure 5 .
Figure 5. Block diagram of exponential function with Taylor expansion.

Figure 5 .
Figure 5. Block diagram of exponential function with Taylor expansion.

Figure 5 .
Figure 5. Block diagram of exponential function with Taylor expansion.

Figure 6 .
Figure 6.Block diagram of Gaussian function.Figure 6. Block diagram of Gaussian function.

Figure 6 .
Figure 6.Block diagram of Gaussian function.Figure 6. Block diagram of Gaussian function.

Figure 7 .
Figure 7. Taylor approximation of a Gaussian function.

Figure 9 .
Figure 9. Block diagram of Gaussian function with Taylor expansion and LUT.

Figure 10 .
Figure 10.Taylor and LUT approximation of a Gaussian function.

Figure 8 . 22 Figure 8 .
Figure 8. Block diagram of exponential function with Taylor expansion and LUT.

Figure 9 .
Figure 9. Block diagram of Gaussian function with Taylor expansion and LUT.

Figure 10 .
Figure 10.Taylor and LUT approximation of a Gaussian function.

Figure 9 . 22 Figure 8 .
Figure 9. Block diagram of Gaussian function with Taylor expansion and LUT.

Figure 9 .
Figure 9. Block diagram of Gaussian function with Taylor expansion and LUT.

Figure 10 .
Figure 10.Taylor and LUT approximation of a Gaussian function.

Figure 10 .
Figure 10.Taylor and LUT approximation of a Gaussian function.

Figure 10 .
Figure 10.Taylor and LUT approximation of a Gaussian function.

Figure 11 .
Figure 11.Block diagram of the sine function with the Taylor expansion.

Figure 11 . 22 Figure 12 .
Figure 11.Block diagram of the sine function with the Taylor expansion.Electronics 2018, 7, x FOR PEER REVIEW 10 of 22

Figure 13 .
Figure 13.Taylor approximation of the sine function.

Figure 14 .
Figure 14.Taylor approximation of the cosine function.

Figure 15 .
Figure 15.Block diagram of the sine function with the Taylor expansion and LUT.

Figure 12 . 22 Figure 12 .
Figure 12.Block diagram of the cosine function with the Taylor expansion.

Figure 13 .
Figure 13.Taylor approximation of the sine function.

Figure 14 .
Figure 14.Taylor approximation of the cosine function.

Figure 15 .
Figure 15.Block diagram of the sine function with the Taylor expansion and LUT.

Figure 13 .
Figure 13.Taylor approximation of the sine function.

Figure 14 .
Figure 14.Taylor approximation of the cosine function.

Figure 14 .
Figure 14.Taylor approximation of the cosine function.

Figure 14 .
Figure 14.Taylor approximation of the cosine function.

Figure 15 .
Figure 15.Block diagram of the sine function with the Taylor expansion and LUT.

Figure 15 . 22 Figure 16 .Figure 17 .Figure 18 .
Figure 15.Block diagram of the sine function with the Taylor expansion and LUT.Electronics 2018, 7, x FOR PEER REVIEW 11 of 22

Figure 16 . 22 Figure 16 .Figure 17 .Figure 18 .
Figure 16.Block diagram of the cosine function with the Taylor expansion and LUT.

Figure 17 . 22 Figure 16 .Figure 17 .Figure 18 .
Figure 17.(a) The Taylor and LUT approximation of the sine function.(b) The error of the sine function and Taylor and LUT approximation.

Figure 18 .
Figure 18.(a) Taylor and LUT approximation of the cosine function.(b) The error of the cosine function and Taylor and LUT approximation.

u
value was divided by the sum of the (3) j u value.
divided by the sum of the (3) j u value.

( 4 )
j value was divided by the sum of the u (3) j value.
shows the SMIMS VeriEnterprise ® FPGA development platform with a Xilinx Virtex 4-XC4VLX60 chip.This development platform is a high-speed PC-based FPGA platform.The key features of the platform are: (1) On-board 128 MB DDR; (2) data transfer with USB 2.0 interface; (3) On-board 16 MB Flash; (4) Up to 168 additional available I/Os; (5) two independent banks-on-board 16 MB Pseudo SRAMs; (6) Download FPGA configuration through USB; (7) Two External Clock SMA Connector; and (8) Maximum data transmit rate between PC and FPGA was 211 Mbps [12].
shows the SMIMS VeriEnterprise ® FPGA development platform with a Xilinx Virtex 4-XC4VLX60 chip.This development platform is a high-speed PC-based FPGA platform.The key features of the platform are: (1) On-board 128 MB DDR; (2) data transfer with USB 2.0 interface; (3) On-board 16 MB Flash; (4) Up to 168 additional available I/Os; (5) two independent banks-on-board 16 MB Pseudo SRAMs; (6) Download FPGA configuration through USB; (7) Two External Clock SMA Connector; and (8) Maximum data transmit rate between PC and FPGA was 211 Mbps [12].

Figure 23 .
Figure 23.Schematic of the temperature control of the water bath system.

Figure 23 .
Figure 23.Schematic of the temperature control of the water bath system.

Figure 24 .
Figure 24.(a) The regulation outputs and (b) errors of FNFN controller in the water bath system.Figure 24.(a) The regulation outputs and (b) errors of FNFN controller in the water bath system.

Figure 24 .
Figure 24.(a) The regulation outputs and (b) errors of FNFN controller in the water bath system.Figure 24.(a) The regulation outputs and (b) errors of FNFN controller in the water bath system.

Figure 25 .
Figure 25.(a) Outputs and (b) errors of the FNFN controller with impulse noise in the water bath system.

Figure 26 .
Figure 26.(a) Outputs and (b) errors of the FNFN controller with the plant dynamics altered in the water bath system.

Figure 27 .
Figure 27.Learning curve of the FNFN controller in the water bath system.

Figure 25 .
Figure 25.(a) Outputs and (b) errors of the FNFN controller with impulse noise in the water bath system.

Figure 25 .
Figure 25.(a) Outputs and (b) errors of the FNFN controller with impulse noise in the water bath system.

Figure 26 .
Figure 26.(a) Outputs and (b) errors of the FNFN controller with the plant dynamics altered in the water bath system.

Figure 27 .
Figure 27.Learning curve of the FNFN controller in the water bath system.

Figure 26 .
Figure 26.(a) Outputs and (b) errors of the FNFN controller with the plant dynamics altered in the water bath system.

Figure 26 .
Figure 26.(a) Outputs and (b) errors of the FNFN controller with the plant dynamics altered in the water bath system.

Figure 27 .
Figure 27.Learning curve of the FNFN controller in the water bath system.

Figure 27 .Figure 28 .
Figure 27.Learning curve of the FNFN controller in the water bath system.

Figure 28 .
Figure 28.(a) Tracking outputs and (b) errors of the FNFN controller when a change occurs in the water bath system.
• .A fixed distance (d b ) of the car was moved backwards at each time step.The plane [0, 100] × [0, 100] represents the limited loading region.Electronics 2018, 7, x FOR PEER REVIEW 18 of 22

Figure 29 .
Figure 29.The simulated car and desired dock.The angle φ and cross position x of the car are two inputs of the proposed FNFN, whereas the steering angle θ presents the output of the proposed FNFN.The ranges of parameters (x, ∅, and θ) are described as follows.0 100 x  

Figure 29 .
Figure 29.The simulated car and desired dock.

Figure 30 .
Figure 30.FNFN controller learning curve in the backing control of the car.

Figure 30 .
Figure 30.FNFN controller learning curve in the backing control of the car.

Table 1 .
The error value of exponential function and Taylor approximation.

Table 2 .
Results of the comparison of various controllers.

Table 2 .
Results of the comparison of various controllers.

Table 3 .
Resources requirements of FNFN implementation for the temperature control.

Table 3 .
Resources requirements of FNFN implementation for the temperature control.

Table 4 .
The RMSE of a car backing control.

Table 5 .
Resource requirements of FNFN implementation the backing control of a car.

Table 5 .
Resource requirements of FNFN implementation the backing control of a car.