Comparison of T-Norms and S-Norms for Interval Type-2 Fuzzy Numbers in Weight Adjustment for Neural Networks

: A comparison of different T-norms and S-norms for interval type-2 fuzzy number weights is proposed in this work. The interval type-2 fuzzy number weights are used in a neural network with an interval backpropagation learning enhanced method for weight adjustment. Results of experiments and a comparative research between traditional neural networks and the neural network with interval type-2 fuzzy number weights with different T-norms and S-norms are presented to demonstrate the beneﬁts of the proposed approach. In this research, the deﬁnitions of the lower and upper interval type-2 fuzzy numbers with random initial values are presented; this interval represents the footprint of uncertainty (FOU). The proposed work is based on recent works that have considered the adaptation of weights using type-2 fuzzy numbers. To conﬁrm the efﬁciency of the proposed method, a case of data prediction is applied, in particular for the Mackey-Glass time series (for τ = 17). Noise of Gaussian type was applied to the testing data of the Mackey-Glass time series to demonstrate that the neural network using a interval type-2 fuzzy numbers method achieves a lower susceptibility to noise than other methods.


Introduction
In the literature there exists research based on a similar idea to this paper, but with different approaches and implementations, such as the adjustment of fuzzy number weights in the input and output layer in the training process for the neural network [1], or the proposal of fuzzy number operations in a fuzzy neural network [2], and also because the proposed work operates with interval type-2 fuzzy numbers weights using different T-norm and S-norm in the adaptation of the weights, which represent the contribution and main difference with respect to the methods in the literature [3][4][5][6].
The proposed method in the present research is different to other papers, such as in Gaxiola et al. [7,8], where the fuzzy weights are obtained using interval type-2 fuzzy inference systems in [7] and generalized type-2 fuzzy inference systems in [8] for the connections between the layers, and without any changes for obtaining the change of the weights for each epoch of the backpropagation algorithm.
In the present approach, the use of interval type-2 fuzzy number weights is proposed.The lower and upper fuzzy numbers are obtained with the Nguyen-Widrow algorithm for the initial weights, modifying the internal calculations of the neurons by performing the multiplications of the inputs for

Related Work
In the neural networks area, the backpropagation algorithm and variations of it is the training method that researchers use in literature [21][22][23].In the bibliography, several papers have proposed different methods to improve the convergence of the backpropagation training algorithm [3,4,6].In this paper, the most significant and essential works about the representation or managing of fuzzy numbers will be reviewed [9][10][11].
Dunyak et al. [1] presented a new algorithm for obtaining new weights (inputs and outputs) in the phase of training with any type of fuzzy numbers for a fuzzy neural network.Fard et al. [24] presented the sum and the product of two interval type-2 triangular fuzzy numbers and, basing it on the Stone-Weierstrass theorem, a fuzzy neural network working with interval type-2 fuzzy logic is developed.Li Z. et al. [2] described a fuzzified neural network computing the results of operations in two fuzzy numbers such as addition, subtraction, multiplication and division.
Asady B. [25] outlined a method for approximating trapezoidal fuzzy numbers in comparison with other methods of approximation.Coroianu L. et al. [26] described the inverse F-transform to accomplish optimal fuzzy numbers, maintaining the support and the convergence of the core.
Yang D. et al. [27] presented an interval and modified interval neuron perceptron with interval weights and biases, and modified the learning algorithm for this approach.
Requena et al. [28] presented trapezoidal fuzzy numbers with the conventional parameters (a, b, c, d) in an artificial neural network, and also proposed a decision personal index (DPI) to obtain the distance between the numbers.Kuo et al. [29] described a fuzzy neural network using the real-coded genetic algorithm to generate the initial fuzzy weights, and using the extension principle, the fuzzy operations are determined.Molinari [30] presented generalized triangular fuzzy numbers and a comparison with others fuzzy numbers.Chai et al. [31] described a representation of fuzzy numbers, establishing the theorem "that for each fuzzy number there exists a unique skew fuzzy number and a unique symmetric fuzzy number".
Figueroa-García et al. [32] made a comparison between interval type-2 fuzzy numbers using distance measures.Ishibuchi et al. [33] presented a comparison between real numbers, and different fuzzy numbers such as symmetric triangular, asymmetric triangular and symmetric trapezoidal as weights in the connections between layers in a neural network.Karnik et al. [34] presented mathematical operations of type-2 fuzzy sets for obtaining the join and meet under t-norm.Raj et al. [35] described fuzzy ranking alternatives for fuzzy numbers as linguistic variables for fuzzy weights.Chu et al. [36] proposed a ranking of fuzzy numbers with a zone between the original point and the centroid point and making numerical examples with triangular fuzzy numbers.
Ishibuchi et al. [37,38] proposed to use the weights for a fuzzy neural network like triangular or trapezoidal fuzzy numbers.Feuring [39] presented a new backpropagation algorithm for learning in the neural network, in which the new lower and upper limits of weights are computed.Castro et al. [40] proposed a type-2 fuzzy neurons model, in which the rules used interval type-2 fuzzy neurons in the antecedents and an interval of type-1 fuzzy neuron in the consequents.
In the area of time series prediction there exists in the literature many recent works developed by and based on the use of type-2 fuzzy logic, like Castro et al. [41], and other researchers [42][43][44].

Proposed Methodology
The proposed method in this work has the goal of generalizing the backpropagation learning algorithm by using interval type-2 fuzzy numbers in the calculations, and this approach gives the neural network less susceptibility to data with uncertainty.In interval type-2 fuzzy numbers, it will be necessary to obtain the interval of the fuzzy numbers, which consists in the footprint of uncertainty (FOU), and the calculations in the neurons was obtained with the T-norms and S-norms of sum-product, Dombi, Hamacher and Frank for the corresponding applications [45][46][47][48].
The method of adjustment of weights for each neuron in the connections between the layers in the backpropagation learning algorithm is modified from the original adjustment (Figure 1).D. et al. [27] presented an interval and modified interval neuron perceptron with interval weights and biases, and modified the learning algorithm for this approach.Requena et al. [28] presented trapezoidal fuzzy numbers with the conventional parameters (a, b, c, d) in an artificial neural network, and also proposed a decision personal index (DPI) to obtain the distance between the numbers.Kuo et al. [29] described a fuzzy neural network using the real-coded genetic algorithm to generate the initial fuzzy weights, and using the extension principle, the fuzzy operations are determined.Molinari [30] presented generalized triangular fuzzy numbers and a comparison with others fuzzy numbers.Chai et al. [31] described a representation of fuzzy numbers, establishing the theorem "that for each fuzzy number there exists a unique skew fuzzy number and a unique symmetric fuzzy number".
Figueroa-García et al. [32] made a comparison between interval type-2 fuzzy numbers using distance measures.Ishibuchi et al. [33] presented a comparison between real numbers, and different fuzzy numbers such as symmetric triangular, asymmetric triangular and symmetric trapezoidal as weights in the connections between layers in a neural network.Karnik et al. [34] presented mathematical operations of type-2 fuzzy sets for obtaining the join and meet under t-norm.Raj et al. [35] described fuzzy ranking alternatives for fuzzy numbers as linguistic variables for fuzzy weights.Chu et al. [36] proposed a ranking of fuzzy numbers with a zone between the original point and the centroid point and making numerical examples with triangular fuzzy numbers.
Ishibuchi et al. [37,38] proposed to use the weights for a fuzzy neural network like triangular or trapezoidal fuzzy numbers.Feuring [39] presented a new backpropagation algorithm for learning in the neural network, in which the new lower and upper limits of weights are computed.Castro et al. [40] proposed a type-2 fuzzy neurons model, in which the rules used interval type-2 fuzzy neurons in the antecedents and an interval of type-1 fuzzy neuron in the consequents.
In the area of time series prediction there exists in the literature many recent works developed by and based on the use of type-2 fuzzy logic, like Castro et al. [41], and other researchers [42][43][44].

Proposed Methodology
The proposed method in this work has the goal of generalizing the backpropagation learning algorithm by using interval type-2 fuzzy numbers in the calculations, and this approach gives the neural network less susceptibility to data with uncertainty.In interval type-2 fuzzy numbers, it will be necessary to obtain the interval of the fuzzy numbers, which consists in the footprint of uncertainty (FOU), and the calculations in the neurons was obtained with the T-norms and S-norms of sum-product, Dombi, Hamacher and Frank for the corresponding applications [45][46][47][48].
The method of adjustment of weights for each neuron in the connections between the layers in the backpropagation learning algorithm is modified from the original adjustment (Figure 1).The method in this paper consists of utilizing interval type-2 fuzzy number weights in neurons.This development modifies the internal calculation for the neurons and the adjustment of the weights to allow handle fuzzy numbers (Figure 2) [49].The method in this paper consists of utilizing interval type-2 fuzzy number weights in neurons.This development modifies the internal calculation for the neurons and the adjustment of the weights to allow handle fuzzy numbers (Figure 2) [49].We modified the operations in the neurons and the backpropagation learning to adjust the fuzzy numbers weights and accomplish the desired result, working to find the optimal process to operate with interval type-2 fuzzy number weights [50,51].
To determine the appropriate activation function f(-) to utilize, the linear and secant hyperbolic functions were considered in this approach.

Architecture of the Traditional Neural Network
The architecture of the neural network used in this work (see Figure 3) consists of a hidden layer with 16 neurons and of an output layer with 1 neuron, and a training data of the Mackey-Glass time series to the input data in the input layer.

Architecture of the Fuzzy Neural Network with Interval Type-2 Fuzzy Numbers Weights
In Figure 4 a scheme of the proposed methodology of the fuzzy neural network with interval type-2 fuzzy numbers weights (FNNIT2FNW) is presented.We modified the operations in the neurons and the backpropagation learning to adjust the fuzzy numbers weights and accomplish the desired result, working to find the optimal process to operate with interval type-2 fuzzy number weights [50,51].
To determine the appropriate activation function f(-) to utilize, the linear and secant hyperbolic functions were considered in this approach.

Architecture of the Traditional Neural Network
The architecture of the neural network used in this work (see Figure 3) consists of a hidden layer with 16 neurons and of an output layer with 1 neuron, and a training data of the Mackey-Glass time series to the input data in the input layer.We modified the operations in the neurons and the backpropagation learning to adjust the fuzzy numbers weights and accomplish the desired result, working to find the optimal process to operate with interval type-2 fuzzy number weights [50,51].
To determine the appropriate activation function f(-) to utilize, the linear and secant hyperbolic functions were considered in this approach.

Architecture of the Traditional Neural Network
The architecture of the neural network used in this work (see Figure 3) consists of a hidden layer with 16 neurons and of an output layer with 1 neuron, and a training data of the Mackey-Glass time series to the input data in the input layer.

Architecture of the Fuzzy Neural Network with Interval Type-2 Fuzzy Numbers Weights
In Figure 4 a scheme of the proposed methodology of the fuzzy neural network with interval type-2 fuzzy numbers weights (FNNIT2FNW) is presented.

Architecture of the Fuzzy Neural Network with Interval Type-2 Fuzzy Numbers Weights
In Figure 4a scheme of the proposed methodology of the fuzzy neural network with interval type-2 fuzzy numbers weights (FNNIT2FNW) is presented.The architecture of fuzzy neural network with interval type-2 fuzzy number weights (see Figure 5) is explained as follow: Phase 0: equation of the inputs data.
[ , ] where and are generated with the Nguyen-Widrow algorithm [52] for the initial weights.Phase 2: Calculation of the output of the hidden neurons with interval type-2 fuzzy number weights.

=
(3) We used the secant hyperbolic as the activation function for the hidden neurons.Subsequently, we applied the S-Norm and T-Norm for calculating the lower and upper outputs of the hidden neurons, respectively.
where: we used the T-norms and S-norms.Sum-Product: The architecture of fuzzy neural network with interval type-2 fuzzy number weights (see Figure 5) is explained as follow: Phase 0: equation of the inputs data.
where w and w are generated with the Nguyen-Widrow algorithm [52] for the initial weights.Phase 2: Calculation of the output of the hidden neurons with interval type-2 fuzzy number weights.
We used the secant hyperbolic as the activation function for the hidden neurons.Subsequently, we applied the S-Norm and T-Norm for calculating the lower and upper outputs of the hidden neurons, respectively.y = TNorm Net, Net where: we used the T-norms and S-norms.Sum-Product: Dombi: for γ > 0.
Phase 3: Calculation of the output for the output neuron with interval type-2 fuzzy number weights.
In the output neuron the linear activation function is utilized.Subsequently, we applied the S-Norm and T-Norm for the lower and upper outputs of the output neurons, respectively.
Phase 4: The single output of the neural network is obtained:

Proposed Adjustment for Interval Type-2 Fuzzy Numbers with Backpropagation Learning
The backpropagation learning algorithm is performed by the adjustment of the interval type-2 fuzzy number weights, described as follows: Stage 1: The Nguyen-Widrow algorithm is utilized to initialize the lower and upper values of the interval type-2 fuzzy numbers weights for the neural network.Stage 2: The input pattern and the wanted output for the neural network is established.Stage 3: The output of the neural network is calculated.In the first instance, the inputs for the network are introduced and the output of the network is obtained performing the calculations of the outputs from the input layer until the output layer.Stage 4: Determine the error terms for the neurons of the layers.In the output layer, the calculation of lower ( ) and upper ( ) delta for each neuron "k" is performed with the follow equations:

Proposed Adjustment for Interval Type-2 Fuzzy Numbers with Backpropagation Learning
The backpropagation learning algorithm is performed by the adjustment of the interval type-2 fuzzy number weights, described as follows: In the hidden layer, the calculation of lower (δ h pj ) and upper (δ h pj ) delta for each neuron "j" is perform with the follow equations: Stage 5: The utilization of a recursive algorithm allows the actualization of the interval type-2 fuzzy number weights, beginning from the output neurons and updating backwards until the neurons in the input layer.The adjustment is described as follows: The calculation of the change of interval type-2 fuzzy number weights is achieved with the equations described as follows: Calculations of the output neurons: Calculations of the hidden neurons: Stage 6: The method is recurrent until for each of the learned patterns the error terms are small enough.
Alternatively, we have the option of working with fuzzy inputs and fuzzy targets.In this case, the modification of the proposed neural network must be in phase 2, multiplying by the lower input in Equation ( 2) and the upper input in Equation (3); besides, in phase 4, we can maintain the lower and upper final outputs of the output neuron because there is no need for performing the average.
The neural network used in this research consists of 16 neurons for the hidden layer, based on a study of the performance of the neural networks modifying the numbers of neurons in the hidden layer, starting with 5 neurons and increasing one by one, until reaching 120 neurons; we are presenting this study in the following section.To test the proposed method, experiments in the time series prediction are performed.A benchmark and chaotic time series such as the Mackey-Glass time series (for τ = 17) is used in this study.
Based on previous work, in the experiments, the backpropagation algorithm applying gradient descent and adaptive learning rate is utilized.
The neural networks manage interval type-2 fuzzy number weights in the hidden and output layer [53,54].In the hidden and output layer of the network we used the backpropagation algorithm modified for working with interval type-2 fuzzy numbers to achieve new weights for the next epochs of the network [55][56][57].

Simulation Results
We achieved the experiments for the Mackey-Glass time series, and for this we used 1000 data points.In this case, 500 data points are considered for the training stage and 500 data points for the testing stage.We performed an experiment for determining manually the optimal number of neurons in the hidden layer of the fuzzy neural network with the interval type-2 fuzzy numbers; we increase the number of neurons by one unit at a time in the interval of 5 to 120 neurons.The acquired results from the experiments are presented in Table 1.The fuzzy neural network with 16 neurons in the hidden layer obtained the best result with 0.0149 for the best prediction error, and 0.0180 for the average error (MAE).This experiment was realized with the T-norm and S-norm of sum-product.
The mean absolute error (MAE) is considered in obtaining the results of the experiments.The average error was obtained by taking 30 experiments with the equal parameters and conditions for all experiments.
The parameters for the fuzzy neural network with interval type-2 fuzzy numbers are of 500 epochs and 0.0000001 of error for the training phase.
We observe from Table 1 that the fuzzy neural network with interval type-2 fuzzy numbers and T-norm and S-norm of sum-product with 16 neurons in the hidden layer (FNNIT2FNSp) shows better results than the others; so, based on this fact, in the following experiments we work with this architecture for the neural network.
In Figure 6, we are presenting the plot of the real data of the Mackey-Glass time series against the predicted data of the interval type-2 fuzzy neural network (FNNIT2FNSp) with 16 neurons in the hidden layer.In Figure 7, an illustration of the convergence curves in the training process is presented.We observe from Table 1 that the fuzzy neural network with interval type-2 fuzzy numbers and T-norm and S-norm of sum-product with 16 neurons in the hidden layer (FNNIT2FNSp) shows better results than the others; so, based on this fact, in the following experiments we work with this architecture for the neural network.
In Figure 6, we are presenting the plot of the real data of the Mackey-Glass time series against the predicted data of the interval type-2 fuzzy neural network (FNNIT2FNSp) with 16 neurons in the hidden layer.In Figure 7, an illustration of the convergence curves in the training process is presented.We performed the same experiment that we presented before with the T-norm and S-norm of Dombi, Hamacher and Frank.

NNIT2FNW for T-Norm and S-Norm of Dombi
The architecture for the fuzzy neural network with T-norm of Dombi (FNNIT2FND) has 4 neurons in the hidden layer and = 0.8, the best results with 0.0457 for the best result for the prediction error, and 0.0622 for the average error.We show some results for this architecture in Table 2, Figures 8 and 9.We performed the same experiment that we presented before with the T-norm and S-norm of Dombi, Hamacher and Frank.

NNIT2FNW for T-Norm and S-Norm of Dombi
The architecture for the fuzzy neural network with T-norm of Dombi (FNNIT2FND) has 4 neurons in the hidden layer and γ = 0.8, the best results with 0.0457 for the best result for the prediction error, and 0.0622 for the average error.We show some results for this architecture in Table 2, Figures 8 and 9.We performed the same experiment that we presented before with the T-norm and S-norm of Dombi, Hamacher and Frank.

NNIT2FNW for T-Norm and S-Norm of Dombi
The architecture for the fuzzy neural network with T-norm of Dombi (FNNIT2FND) has 4 neurons in the hidden layer and = 0.8, the best results with 0.0457 for the best result for the prediction error, and 0.0622 for the average error.We show some results for this architecture in Table 2, Figures 8 and 9.

NNIT2FNW for T-Norm and S-Norm of Hamacher
The architecture for the fuzzy neural network with the T-norm of Hamacher (FNNIT2FNH) has 39 neurons in the hidden layer and = 1, the best result with 0.0130 for the best prediction error, and 0.0164 for the average error.We show some results for this architecture in Table 3, Figures 10 and 11

NNIT2FNW for T-Norm and S-Norm of Hamacher
The architecture for the fuzzy neural network with the T-norm of Hamacher (FNNIT2FNH) has 39 neurons in the hidden layer and γ = 1, the best result with 0.0130 for the best prediction error, and 0.0164 for the average error.We show some results for this architecture in Table 3, Figures 10 and 11.

NNIT2FNW for T-Norm and S-Norm of Hamacher
The architecture for the fuzzy neural network with the T-norm of Hamacher (FNNIT2FNH) has 39 neurons in the hidden layer and = 1, the best result with 0.0130 for the best prediction error, and 0.0164 for the average error.We show some results for this architecture in Table 3, Figures 10 and 11.

NNIT2FNW for T-Norm and S-Norm of Frank
The architecture for the fuzzy neural network with T-norm of Frank (FNNIT2FNF) has 19 neurons in the hidden layer and = 2.8, the best results with 0.0117 for the best prediction error, and 0.0167 for the average error.We show some results for this architecture in Table 4, Figures 12 and 13.

NNIT2FNW for T-Norm and S-Norm of Frank
The architecture for the fuzzy neural network with T-norm of Frank (FNNIT2FNF) has 19 neurons in the hidden layer and γ = 2.8, the best results with 0.0117 for the best prediction error, and 0.0167 for the average error.We show some results for this architecture in Table 4, Figures 12 and 13.

NNIT2FNW for T-Norm and S-Norm of Frank
The architecture for the fuzzy neural network with T-norm of Frank (FNNIT2FNF) has 19 neurons in the hidden layer and = 2.8, the best results with 0.0117 for the best prediction error, and 0.0167 for the average error.We show some results for this architecture in Table 4, Figures 12 and 13

Comparison of Traditional Neural Network Against NNIT2FNW for T-Norm and S-Norm
The acquired results in the experiments with the traditional neural network (TNN) are present on Table 5 and Figures 14 and 15, and the neural network parameters are obtained are based on empirical testing.The best result of the prediction errors is of 0.0169, and the average error is of 0.0203 (MAE).In Table 2, we present the comparison of these results against the results of the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF).

Comparison of Traditional Neural Network Against NNIT2FNW for T-Norm and S-Norm
The acquired results in the experiments with the traditional neural network (TNN) are present on Table 5 and Figures 14 and 15, and the neural network parameters are obtained are based on empirical testing.The best result of the prediction errors is of 0.0169, and the average error is of 0.0203 (MAE).In Table 2, we present the comparison of these results against the results of the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF).

Comparison of Traditional Neural Network Against NNIT2FNW for T-Norm and S-Norm
The acquired results in the experiments with the traditional neural network (TNN) are present on Table 5 and Figures 14 and 15, and the neural network parameters are obtained are based on empirical testing.The best result of the prediction errors is of 0.0169, and the average error is of 0.0203 (MAE).In Table 2, we present the comparison of these results against the results of the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF).We also implemented an experiment utilizing noisiness in the interval 0.1 to 1 in the test data to analyze the robustness of the traditional neural network (TNN) and the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF).The obtained results for these experiments are presented in Table 6.We applied the noise by using the following equations: where: "Data" are the test data points of the Mackey-Glass Time series, "NoiseLevel" is the level of noise in the range (0.1-1), "rand" is a uniformly distributed function for random numbers used for obtained the values for the noise.We also implemented an experiment utilizing noisiness in the interval 0.1 to 1 in the test data to analyze the robustness of the traditional neural network (TNN) and the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF).The obtained results for these experiments are presented in Table 6.We applied the noise by using the following equations: where: "Data" are the test data points of the Mackey-Glass Time series, "NoiseLevel" is the level of noise in the range (0.1-1), "rand" is a uniformly distributed function for random numbers used for obtained the values for the noise.We observe in Table 6 that the better performance was accomplished with the fuzzy neural network with interval type-2 fuzzy numbers with all T-norms for almost all the levels of noise in the data test.The prediction error for the traditional neural network was increasing considerably with the higher noise levels as a difference to the fuzzy neural network, which maintains the prediction error below 0.21.
We observe in Figure 16 that when at the test data is applied noise, the fuzzy neural network with interval type-2 fuzzy numbers with the Hamacher and Frank T-norms achieve minor prediction errors compared with the traditional neural network in the different levels of noise.An important fact is that the fuzzy neural network with T-norm of Dombi presents better performance than the others in the test with noise but without noise has a higher prediction error.We observe in Table 6 that the better performance was accomplished with the fuzzy neural network with interval type-2 fuzzy numbers with all T-norms for almost all the levels of noise in the data test.The prediction error for the traditional neural network was increasing considerably with the higher noise levels as a difference to the fuzzy neural network, which maintains the prediction error below 0.21.
We observe in Figure 16 that when at the test data is applied noise, the fuzzy neural network with interval type-2 fuzzy numbers with the Hamacher and Frank T-norms achieve minor prediction errors compared with the traditional neural network in the different levels of noise.An important fact is that the fuzzy neural network with T-norm of Dombi presents better performance than the others in the test with noise but without noise has a higher prediction error.A statistical test, the t-student test, was applied to perform a comparison of the performance of TNN against FNNIT2FNH, and FNNIT2FNF in the prediction error; we selected these two fuzzy neural networks because they presented a better performance than FNNIT2FNSp, and FNNIT2FND.In the statistical tests we consider 30 experiments and a 95% of reliability in the tests.
We present in Table 7 the parameters for the statistical test of the TNN and FNNIT2FNHmodel.We used a Hypothesis testing of H0: TNN = FNNIT2FNHand for the alternative hypothesis H1: TNN > FNNIT2FNH by the comparison of these models; H0: TNN = FNNIT2FNF and H1: TNN > FNNIT2FNF.
The results obtained with the statistical test for the prediction errors for TNN against FNNIT2FNH are of 0.003907 in the estimated mean difference, 0.003153 in the lower limit of the difference, a t value of 1037, p value of 0.0001 and 56 degrees of freedom.
The results obtained with the statistical test for prediction errors for TNN against FNNIT2FNFare of 0.003631 in the estimated mean difference, 0.002897 in the lower limit of the difference, a t value of 9.93, p value of 0.0001 and 54 degrees of freedom.A statistical test, the t-student test, was applied to perform a comparison of the performance of TNN against FNNIT2FNH, and FNNIT2FNF in the prediction error; we selected these two fuzzy neural networks because they presented a better performance than FNNIT2FNSp, and FNNIT2FND.In the statistical tests we consider 30 experiments and a 95% of reliability in the tests.
We present in Table 7 the parameters for the statistical test of the TNN and FNNIT2FNHmodel.We used a Hypothesis testing of H0: TNN = FNNIT2FNHand for the alternative hypothesis H1: TNN > FNNIT2FNH by the comparison of these models; H0: TNN = FNNIT2FNF and H1: TNN > FNNIT2FNF.The results obtained with the statistical test for the prediction errors for TNN against FNNIT2FNH are of 0.003907 in the estimated mean difference, 0.003153 in the lower limit of the difference, a t value of 1037, p value of 0.0001 and 56 degrees of freedom.
The results obtained with the statistical test for prediction errors for TNN against FNNIT2FNFare of 0.003631 in the estimated mean difference, 0.002897 in the lower limit of the difference, a t value of 9.93, p value of 0.0001 and 54 degrees of freedom.
The results obtained with the statistical test for prediction errors for FNNIT2FNHagainst FNNIT2FNF are of −0.00027 in the estimated mean difference, −0.000940 in the lower limit of the difference, a t value of −0.84, p value of 0.407 and 57 degrees of freedom.
The results demonstrate that there exists significant statistical evidence to affirm that the FNNIT2FNH and FNNIT2FNHF are better than the TNN, and that the FNNIT2FNH is equal to the FNNIT2FNF.

Conclusions
Based on the experiments, we have reached the conclusion that the fuzzy neural network with interval type-2 fuzzy number weights with T-norm of Sum-product (FNNIT2FNSp), Hamacher (FNNIT2FNH) and Frank (FNNIT2FNF) achieved better results than the traditional neural network for the benchmark time series used in this work, Mackey-Glass.This affirmation is based on the prediction errors of 0.0169 for TNN, and 0.0149, 0.0130 and 0.0117 for FNNIT2FNSp, FNNIT2FNH and FNNIT2FNF, respectively; and the average errors obtained of 30 experiments are of 0.0203, and 0.0180, 0.0164 and 0.0167, respectively.
The fuzzy neural network with interval type-2 fuzzy number weights with the T-norm presented a better tolerance and behavior than the traditional neural network when Gaussian noise is applied at the testing data.This inference was reached by analyzing that the fuzzy neural network with interval type-2 fuzzy number weights with T-norms show only minor prediction errors compared to the traditional neural network at increasing the levels of noise.Besides, analyzing Tables 5 and 6, and Figure 10, from which the FNNIT2FNH and FNNIT2FNF show only minor prediction errors compared to the other paradigms in this work for the Mackey-Glass time series, and furthermore observing the results for the statistical tests performed to these paradigms.
The method proposed in this work, a fuzzy neural network with interval type-2 fuzzy number weights with T-norms, presents better performance, robustness and achieves lower results of prediction errors than the traditional neural network.Furthermore, the interval type-2 fuzzy number weights allow the neural network to have a minor susceptibility to increment the results of the prediction error when the real data is affected with noise.

Figure 1 .
Figure 1.Structure and equation of a neuron with traditional numerical weights.

Figure 1 .
Figure 1.Structure and equation of a neuron with traditional numerical weights.

Figure 2 .
Figure 2. Scheme of the proposed structure and equations of the neuron with interval type-2 fuzzy number weights.

Figure 3 .
Figure 3. Illustration of the architecture of the neural network.

Figure 2 .
Figure 2. Scheme of the proposed structure and equations of the neuron with interval type-2 fuzzy number weights.

Figure 2 .
Figure 2. Scheme of the proposed structure and equations of the neuron with interval type-2 fuzzy number weights.

Figure 3 .
Figure 3. Illustration of the architecture of the neural network.

Figure 3 .
Figure 3. Illustration of the architecture of the neural network.

Figure 5 .
Figure 5. Illustration of the architecture of the Fuzzy Neural network.

Figure 5 .
Figure 5. Illustration of the architecture of the Fuzzy Neural network.

Stage 1 :
The Nguyen-Widrow algorithm is utilized to initialize the lower and upper values of the interval type-2 fuzzy numbers weights for the neural network.Stage 2: The input pattern and the wanted output for the neural network is established.Stage 3: The output of the neural network is calculated.In the first instance, the inputs for the network are introduced and the output of the network is obtained performing the calculations of the outputs from the input layer until the output layer.Stage 4: Determine the error terms for the neurons of the layers.In the output layer, the calculation of lower (δ O pk ) and upper (δ O pk ) delta for each neuron "k" is performed with the follow equations:

4. 1 .
Neural Network with Interval Type-2 Fuzzy Numbers Weights (NNIT2FNW) for T-Norm and S-Norm of Sum-Product

Figure 6 .
Figure 6.Illustration of the real data against the prediction data of the Mackey-Glass time series for the fuzzy neural network.

Figure 6 .
Figure 6.Illustration of the real data against the prediction data of the Mackey-Glass time series for the fuzzy neural network.

Figure 7 .
Figure 7. Illustration of the convergence curves in the training process for FNNIT2FNSp.

Figure 8 .
Figure 8. Illustration of the prediction data of the FNNIT2FND against the real data for the Mackey-Glass time series.

Figure 7 .
Figure 7. Illustration of the convergence curves in the training process for FNNIT2FNSp.

Information 2017, 8 , 114 12 of 21 Figure 7 .
Figure 7. Illustration of the convergence curves in the training process for FNNIT2FNSp.

Figure 8 .
Figure 8. Illustration of the prediction data of the FNNIT2FND against the real data for the Mackey-Glass time series.

Figure 8 .
Figure 8. Illustration of the prediction data of the FNNIT2FND against the real data for the Mackey-Glass time series.

Figure 9 .
Figure 9. Illustration of the convergence curves in the training process for FNNIT2FND. .

Figure 10 .
Figure 10.Illustration of the prediction data of the FNNIT2FNH against the real data for the Mackey-Glass time series.

Figure 9 .
Figure 9. Illustration of the convergence curves in the training process for FNNIT2FND.

Figure 10 .
Figure 10.Illustration of the prediction data of the FNNIT2FNH against the real data for the Mackey-Glass time series.

Figure 10 . 21 Figure 11 .
Figure 10.Illustration of the prediction data of the FNNIT2FNH against the real data for the Mackey-Glass time series.Information 2017, 8, 114 14 of 21

Figure 11 .
Figure 11.Illustration of the convergence curves in the training process for FNNIT2FNH.

Information 2017, 8 , 114 14 of 21 Figure 11 .
Figure 11.Illustration of the convergence curves in the training process for FNNIT2FNH. .

Figure 12 .
Figure 12.Illustration of the prediction data of the FNNIT2FNF against the real data for the Mackey-Glass time series.

Figure 12 .
Figure 12.Illustration of the prediction data of the FNNIT2FNF against the real data for the Mackey-Glass time series.

Figure 13 .
Figure 13.Illustration of the convergence curves in the training process for FNNIT2FNF.

Figure 14 .
Figure 14.Illustration of the prediction data against for the traditional neural network the real data of the Mackey-Glass time series.

Figure 13 .
Figure 13.Illustration of the convergence curves in the training process for FNNIT2FNF.

Information 2017, 8 , 114 15 of 21 Figure 13 .
Figure 13.Illustration of the convergence curves in the training process for FNNIT2FNF.

Figure 14 .
Figure 14.Illustration of the prediction data against for the traditional neural network the real data of the Mackey-Glass time series.

Figure 14 .
Figure 14.Illustration of the prediction data against for the traditional neural network the real data of the Mackey-Glass time series.

Figure 15 .
Figure 15.Illustration of the convergence curves in the training process for traditional neural network.

Figure 15 .
Figure 15.Illustration of the convergence curves in the training process for traditional neural network.

Figure 16 .
Figure16.Illustration of the results of prediction error of the TNN against the results of FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, and FNNIT2FNF for data with Gaussian noise of the Mackey-Glass time series for MAE.

FNNIT2FNFFigure 16 .
Figure16.Illustration of the results of prediction error of the TNN against the results of FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, and FNNIT2FNF for data with Gaussian noise of the Mackey-Glass time series for MAE.

Table 1 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of sum-product in time series prediction using Mackey-Glass time series.

Table 2 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Dombi in time series prediction using Mackey-Glass time series.

Table 2 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Dombi in time series prediction using Mackey-Glass time series.

Table 3 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Hamacher in time series prediction using Mackey-Glass time series.

Table 3 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Hamacher in time series prediction using Mackey-Glass time series.

Table 3 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Hamacher in time series prediction using Mackey-Glass time series.

Table 4 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Frank in time series prediction using Mackey-Glass time series.

Table 4 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Frank in time series prediction using Mackey-Glass time series.

Table 4 .
Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Frank in time series prediction using Mackey-Glass time series.

Table 5 .
Results for the traditional neural network (TNN) in the Mackey-Glass time series and the comparison against the FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, and FNNIT2FNF.

Table 6 .
Results for the traditional neural network and fuzzy neural networks with all T-norms in the Mackey-Glass time series under different noise levels (n).

Table 5 .
Results for the traditional neural network (TNN) in the Mackey-Glass time series and the comparison against the FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, and FNNIT2FNF.

Table 6 .
Results for the traditional neural network and fuzzy neural networks with all T-norms in the Mackey-Glass time series under different noise levels (n).

Table 7 .
Parameters used in the t-student statistical test for the TNN against FNNIT2FNH and FNNIT2FNF.