Constructing a Precise Fuzzy Feedforward Neural Network Using an Independent Fuzzification Approach

Abstract

This study discusses how to fuzzify a feedforward neural network (FNN) to generate a fuzzy forecast that contains the actual value, while minimizing the average range of fuzzy forecasts. This topic has rarely been investigated in past studies, but is an essential step to constructing a precise fuzzy FNN (FFNN). Existing methods fuzzify all parameters at the same time, which re-sults in a nonlinear programming (NLP) problem that is not easy to solve. In contrast, in this study, the parameters of a FNN are fuzzified independently. In this way, the optimal values of fuzzy parameters can be derived theoretically. An illustrative example is used to illustrate the ap-plicability of the proposed methodology. According to the experimental results, fuzzifying the thresholds on hidden-layer nodes or the connection weights between input and hidden layers may not guarantee that all fuzzy forecasts contain the corresponding actual values. In contrast, fuzzi-fying the threshold on the output node and the connection weights between the hidden and out-put layers is more likely to achieve a 100% hit rate. The results lay a foundation for establishing a precise deep FFNN in the future.

1. Introduction

Fuzzy feedforward neural networks (FFNNs) combines the advantages of fuzzy logic (in uncertainty modelling) and feedforward neural networks (FNNs) (in nonlinear approximation) [1], and have been widely applied to forecasting in many fields [2,3,4,5]. There are various types of FFNNs with fuzzy or crisp inputs, parameters, and outputs. The numbers of layers and activation (or transformation) functions in these FFNNs are also different [6]. A recent review on FFNNs refers to de Campos Souza [7]. At present, the most commonly applied FFNNs are the variants of adaptive network-based fuzzy inference system (ANFIS) [8,9,10,11,12,13]. Past studies have shown that FFNNs can improve the forecasting accuracy, that is, each forecast is close to the actual value [14,15,16]. However, the present study aims to construct an FFNN to improve the forecasting precision, that is, every actual value is included in the narrowest possible fuzzy forecast. This topic has rarely been discussed in the past, which constitutes the motivation of this research.

However, even if a sophisticated FFNN is applied, the network output is rarely equal to the actual value, especially when the FFNN is applied to unlearned data. To address this issue, an alternative is to estimate the range of the actual value [17]. In other words, a fuzzy forecast that contains the actual value needs to be generated by an FFNN, at least for the training data. However, it is not easy since there are no actual values of the lower and upper bounds of the range. In addition, a fuzzy forecast needs to be as narrow as possible to have a reference value [18]. This is also a challenging task because a narrower range is less likely to contain the actual value. Some of the relevant literature are reviewed as follows.

In an ANFIS, the network output before defuzzification cannot guarantee the inclusion of the actual value [19]. The problem is even more complicated for an FFNN in which all parameters are fuzzy and nonlinear transformation functions (such as sigmoid or tansig functions) are applied. For example, Chen and Wang [20] showed that the problem of deriving the values of fuzzy parameters in an FFNN was a nonlinear programming (NLP) problem that was difficult to solve. A branch-and-bound algorithm can be applied to find a solution to the NLP problem [21], but the solution may be far from (global) optimal. Instead, Chen and Wang established goals for the lower and upper bounds of the actual value to simplify the NLP problem to a goal programming (GP) problem. However, a number of goals needed to be tried to improve the solution, which was time-consuming. A similar method was proposed in Chen and Lin [18], in which the membership of an actual value in the fuzzy forecast had to be greater than a specified level. If only the threshold on the output node was fuzzy, the optimal value of the fuzzy threshold can be derived by solving two linear equations [22]. Similar treatments have been taken by Chen and Wu [17] and Chen [23]. Wang et al. [24] randomized the values of fuzzy thresholds on hidden-layer nodes and then optimized the fuzzy threshold on the output node. After a few replications, these fuzzy thresholds could be optimized. However, connection weights in the FFNN were still crisp.

This study considers an FFNN with a single hidden layer in which all network parameters can be fuzzified and nonlinear transformation functions (i.e., sigmoid functions) are adopted. We aim to optimize the values of fuzzy parameters theoretically without solving an NLP problem, while guaranteeing that all actual values are contained in the corresponding fuzzy forecasts. However, instead of fuzzifying all parameters at the same time, this study follows an independent fuzzification approach in which parameters are fuzzified independently. This study is important because it is a fundamental step towards the construction of a precise FFNN, which lays a foundation for establishing a precise deep FFNN with multiple or recurrent hidden layers.

The contribution of this research is to derive the formula for optimizing the value of each fuzzy parameter in an FFNN, so as to minimize the average range of fuzzy forecasts while ensuring a 100% hit rate. In contrast, existing methods need to solve an NLP problem to achieve the same goal.

The remainder of this study is organized as follows. The independent fuzzification approach is detailed in Section 2. A numerical example is given in Section 3 to illustrate the applicability of the proposed methodology. The effects of fuzzifying various parameters on the average range of fuzzy forecasts are also compared. This study is concluded in Section 4. Some directions for future investigation are also provided.

2. Independent Fuzzification Approach

All parameters and variables in the proposed methodology are given in or approximated by triangular fuzzy numbers (TFNs).

2.1. FFNN Configuration

The FFNN considered in this study is an FFNN that has three layers: the input layer, a single hidden layer, and the output layer. Inputs to the FFNN are indicated with {$z_{jp}$ | p = 1~P; j = 1~n}. $z_{jp}$ is the normalized value of decision variable $x_{jp}$:

$$z_{jp}=\frac{x_{jp}-\underset{v}{\min }{x}_{vp}}{\underset{v}{\max }{x}_{vp}-\underset{v}{\min }{x}_{vp}}$$

(1)

To convert back to the original value,

$$\begin{array}{cc}\hfill x_{jp}& =U(z_{jp})\hfill \\ & =z_{jp}(\underset{v}{\max }{x}_{vp}-\underset{v}{\min }{x}_{vp})+\underset{v}{\min }{x}_{vp}\hfill \end{array}$$

(2)

These inputs are propagated through the FFNN as follows. First, from the input layer to the hidden layer, the following operations are performed:

$${\tilde{I}}_{jl}^h={\displaystyle \sum _{p=1}^P({\tilde{w}}_{pl}^h{z}_{jp})}$$

(3)

$$\begin{array}{cc}\hfill {\tilde{n}}_{jl}^h& ={\tilde{I}}_{jl}^h(-){\tilde{\theta }}_l^h\hfill \\ & =(I_{jl1}^h-{\theta }_{l3}^h,I_{jl2}^h-{\theta }_{l2}^h,I_{jl3}^h-{\theta }_{l1}^h)\hfill \end{array}$$

(4)

$${\tilde{h}}_{jl}=\frac{1}{1+e^{-{\tilde{n}}_{jl}^h}}$$

(5)

where ${\tilde{w}}_{pl}^h$ is the connection weight between input node p and hidden-layer node l; l = 1~L. ${\tilde{\theta }}_l^h$ is the threshold on hidden-layer node l. ${\tilde{h}}_{jl}$ is the output from hidden-layer node l. (−) denotes fuzzy subtraction. In Equation (5), the activation (or transformation) function is the logistic sigmoid activation function that returns a value within [0, 1].

Outputs from the hidden layer are aggregated on the output node,

$${\tilde{I}}_j^o={\displaystyle \sum _{l=1}^L({\tilde{w}}_l^o(\times ){\tilde{h}}_{jl})}$$

(6)

and then the network output ${\tilde{o}}_j=(o_{j1},o_{j2},o_{j3})$ is generated as

$${\tilde{o}}_j=\frac{1}{1+e^{-{\tilde{n}}_j^o}}$$

(7)

where

$${\tilde{n}}_j^o={\tilde{I}}_j^o(-){\tilde{\theta }}^o$$

(8)

${\tilde{w}}_l^o$ is the connection weight between hidden-layer node l and the output node. ${\tilde{\theta }}^o$ is the threshold on the output node. ${\tilde{o}}_j$ is unnormalized according to Equation (2) and then compared with the actual value $y_j$.

2.2. Deriving the Cores of Fuzzy Parameters

The training of the FFNN is composed of two stages. First, the cores of fuzzy parameters are derived by training the FFNN as a crisp FNN using the Levenberg–Marquardt (LM) algorithm [25], so as to minimize the mean squared error (MSE). The optimal solution is indicated with {$w_{pl2}^{h*}$, ${\theta }_{l2}^{h*}$, $w_{l2}^{o*}$, ${\theta }_2^{o*}$ | p = 1~P; l = 1~L}.

Subsequently, the lower and upper bounds of fuzzy parameters are to be determined, so as to minimize the average range (AR) of fuzzy forecasts:

$$\mathrm{Min} AR=\frac{1}{n}{\displaystyle \sum _{j=1}^n(o_{j3}-o_{j1})}$$

(9)

However, deriving the optimal values of all fuzzy parameters at the same time is a computationally intensive task [20]. As an alternative, the optimal values of fuzzy parameters are derived independently as follows.

2.3. Deriving the Optimal Value of ${\tilde{\theta }}^o$

The optimal value of ${\tilde{\theta }}^o$ is to be derived. First, substituting Equation (8) into Equation (7) gives

$${\tilde{o}}_j=\frac{1}{1+e^{({\tilde{\theta }}^o(-){\tilde{I}}_j^o)}}$$

(10)

which can be decomposed into

$$o_{j1}=\frac{1}{1+e^{({\theta }_3^o-I_{j1}^o)}}$$

(11)

$$o_{j3}=\frac{1}{1+e^{({\theta }_1^o-I_{j3}^o)}}$$

(12)

The other parameters are not fuzzified, so $I_{j1}^o$ and $I_{j3}^o$ are equal to $I_{j2}^{o*}$ that is a fixed value:

$$o_{j1}=\frac{1}{1+e^{({\theta }_3^o-I_{j2}^{o*})}}$$

(13)

$$o_{j3}=\frac{1}{1+e^{({\theta }_1^o-I_{j2}^{o*})}}$$

(14)

To minimize AR, $o_{j1}$ and $o_{j3}$ should be maximized and minimized, respectively, which correspond to the minimization of ${\theta }_3^o$ and the maximization of ${\theta }_1^o$. However, ${\tilde{o}}_j$ should include $a_j$ (the actual value), therefore

$$o_{j1}\le a_j; j = 1 ~ n$$

(15)

$$o_{j3}\ge a_j; j = 1 ~ n$$

(16)

In addition, $o_{j1}\le o_{j2}^{*}\le o_{j3}$, so

$$o_{j1}\le \min (o_{j2}^{*},\hspace{0.33em}{a}_j); j = 1 ~ n$$

(17)

$$o_{j3}\ge \max (o_{j2}^{*},\hspace{0.33em}{a}_j); j = 1 ~ n$$

(18)

Substituting Equation (13) into Constraint (17) gives

$${\theta }_3^o\ge I_{j2}^{o*}+\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(19)

Therefore,

$${\theta }_3^o\ge \underset{j}{\max }\left(I_{j2}^{o*}+\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)\right)$$

(20)

To minimize ${\theta }_3^o$,

$${\theta }_3^{o*}=\underset{j}{\max }\left(I_{j2}^{o*}+\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)\right)$$

(21)

Similarly, by substituting Equation (14) into Constraint (18), the following result can be obtained:

$${\theta }_1^{o*}=\underset{j}{\min }\left(I_{j2}^{o*}+\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)\right)$$

(22)

Most past studies [17,22,23,24] stop at this step. The following discussion is new to the body of knowledge.

2.4. Deriving the Optimal Value of ${\tilde{w}}_{l_f}^o$

The optimal value of ${\tilde{w}}_l^o$ is to be derived when $l=l_f$. Equation (6) can be decomposed into

$$I_{j1}^o=\min \left({\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl1})},\hspace{0.33em}{\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl3})}\right)$$

(23)

$$I_{j3}^o=\max \left({\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl1})},\hspace{0.33em}{\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl3})}\right)$$

(24)

because ${\tilde{h}}_{jl}$ is positive, while ${\tilde{w}}_l^o$ may be negative. Substituting Equations (23) and (24) into Equations (13) and (14) gives

$$o_{j1}=\frac{1}{1+e^{\left({\theta }_3^o-\min ({\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl1})},\hspace{0.33em}{\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl3})})\right)}}$$

(25)

$$o_{j3}=\frac{1}{1+e^{\left({\theta }_1^o-\max ({\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl1})},\hspace{0.33em}{\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl3})})\right)}}$$

(26)

Only connection weights are fuzzified. The other fuzzy parameters are set to their optimized cores as

$$\begin{array}{cc}\hfill o_{j1}=& \frac{1}{1+e^{\left({\theta }_2^{o*}-\min ({\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl2}^{*})},\hspace{0.33em}{\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl2}^{*})})\right)}}\hfill \\ & =\frac{1}{1+e^{\left({\theta }_2^{o*}-{\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl2}^{*})}\right)}}\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill o_{j3}=& \frac{1}{1+e^{\left({\theta }_2^{o*}-\max ({\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl2}^{*})},\hspace{0.33em}{\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl2}^{*})})\right)}}\hfill \\ & =\frac{1}{1+e^{\left({\theta }_2^{o*}-{\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl2}^{*})}\right)}}\hfill \end{array}$$

(28)

Substituting Equations (27) and (28) respectively into Constraints (17) and (18), we obtain

$${\displaystyle \sum _{l=1}^L(w_{l1}^o{h}_{jl2}^{*})}\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(29)

$${\displaystyle \sum _{l=1}^L(w_{l3}^o{h}_{jl2}^{*})}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(30)

Only $w_{l_f}^o$ is fuzzified, the other connection weights are equal to their optimized cores:

$$w_{l_{f}1}^o{h}_{j{l}_{f}2}^{*}+{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(31)

$$w_{l_{f}3}^o{h}_{j{l}_{f}2}^{*}+{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(32)

As a result,

$$w_{l_{f}1}^o\le \frac{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}}{h_{j{l}_{f}2}^{*}}; j = 1 ~ n$$

(33)

$$w_{l_{f}3}^o\ge \frac{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}}{h_{j{l}_{f}2}^{*}}; j = 1 ~ n$$

(34)

Therefore,

$$w_{l_{f}1}^o\le \underset{j}{\min }\left(\frac{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}}{h_{j{l}_{f}2}^{*}}\right)$$

(35)

$$w_{l_{f}3}^o\ge \underset{j}{\max }\left(\frac{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}}{h_{j{l}_{f}2}^{*}}\right)$$

(36)

Increasing the fuzziness (i.e., width) of ${\tilde{w}}_{l_f}^o$ will make ${\tilde{o}}_j$ wider. Therefore, it is reasonable to set

$$w_{l_{f}1}^{o*}=\underset{j}{\min }\left(\frac{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}}{h_{j{l}_{f}2}^{*}}\right)$$

(37)

$$w_{l_{f}3}^{o*}=\underset{j}{\max }\left(\frac{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}(w_{l2}^{o*}{h}_{jl2}^{*})}}{h_{j{l}_{f}2}^{*}}\right)$$

(38)

2.5. Deriving the Optimal Value of ${\tilde{\theta }}_l^h$

Substituting Equation (4) into Equation (5) gives

$${\tilde{h}}_{jl}=\frac{1}{1+e^{({\tilde{\theta }}_l^h(-){\tilde{I}}_{jl}^h)}}$$

(39)

which can be decomposed into

$$h_{jl1}=\frac{1}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}$$

(40)

$$h_{jl3}=\frac{1}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}$$

(41)

Substituting Equations (40) and (41) into Equations (23) and (24) leads to

$$I_{j1}^o=\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)$$

(42)

$$I_{j3}^o=\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)$$

(43)

that are substituted into Equations (13) and (14):

$$o_{j1}=\frac{1}{1+e^{\left({\theta }_3^o-\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)\right)}}$$

(44)

$$o_{j3}=\frac{1}{1+e^{\left({\theta }_1^o-\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)\right)}}$$

(45)

The following requirements should be met:

$$\frac{1}{1+e^{\left({\theta }_3^o-\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)\right)}}\le \min (o_{j2}^{*},\hspace{0.33em}{a}_j); j = 1 ~ n$$

(46)

$$\frac{1}{1+e^{\left({\theta }_1^o-\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)\right)}}\ge \max (o_{j2}^{*},\hspace{0.33em}{a}_j); j = 1 ~ n$$

(47)

that are equivalent to

$$\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)\le {\theta }_3^o-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(48)

$$\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-I_{jl1}^h)}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-I_{jl3}^h)}}}\right)\ge {\theta }_1^o-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(49)

Only ${\theta }_{l_f}^h$ is fuzzified, the other fuzzy parameters are set to their optimized cores:

$$\min \left(\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}3}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}},\hspace{0.33em}\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}1}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}\right)\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(50)

$$\max \left(\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}3}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}},\hspace{0.33em}\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}1}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}\right)\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(51)

If $w_{l_{f}2}^{o*}\ge 0$,

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}3}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(52)

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}1}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(53)

As a result,

$${\theta }_{l_{f}3}^h\ge \ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}; j = 1 ~ n$$

(54)

$${\theta }_{l_{f}1}^h\le \ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}; j = 1 ~ n$$

(55)

Therefore,

$${\theta }_{l_{f}1}^h\le \underset{j}{\min }\left(\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}\right)$$

(56)

$${\theta }_{l_{f}3}^h\ge \underset{j}{\max }\left(\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}\right)$$

(57)

To minimize the fuzziness of ${\tilde{\theta }}_{l_f}^h$,

$${\theta }_{l_{f}1}^{h*}=\underset{j}{\min }\left(\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}\right)$$

(58)

$${\theta }_{l_{f}3}^{h*}=\underset{j}{\max }\left(\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}\right)$$

(59)

Otherwise,

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}1}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(60)

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}3}^h-I_{j{l}_{f}2}^{h*})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(61)

As a result,

$${\theta }_{l_{f}1}^h\ge \ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}; j = 1 ~ n$$

(62)

$${\theta }_{l_{f}3}^h\le \ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}; j = 1 ~ n$$

(63)

Therefore,

$${\theta }_{l_{f}1}^h\ge \underset{j}{\max }\left(\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}\right)$$

(64)

$${\theta }_{l_{f}3}^h\le \underset{j}{\min }\left(\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-I_{jl2}^{h*})}}}}-1)+I_{j{l}_{f}2}^{h*}\right)$$

(65)

To minimize the fuzziness of ${\tilde{\theta }}_{l_f}^h$, ${\theta }_{l_{f}1}^{h*}$ and ${\theta }_{l_{f}3}^{h*}$ should be maximized and minimized, respectively, but they are still bounded by ${\theta }_{l_{f}2}^{h*}$. Therefore,

$${\theta }_{l_{f}1}^{h*}={\theta }_{l_{f}2}^{h*}$$

(66)

$${\theta }_{l_{f}3}^{h*}={\theta }_{l_{f}2}^{h*}$$

(67)

2.6. Deriving the Optimal Value of ${\tilde{w}}_{pl}^h$

Equation (3) can be decomposed into

$$I_{jl1}^h={\displaystyle \sum _{p=1}^P(w_{pl1}^h{z}_{jp})}$$

(68)

$$I_{jl3}^h={\displaystyle \sum _{p=1}^P(w_{pl3}^h{z}_{jp})}$$

(69)

Substituting Equations (68) and (69) into Equations (46) and (47) gives

$$o_{j1}=\frac{1}{1+e^{\left({\theta }_3^o-\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-{\displaystyle \sum _{p=1}^P(w_{pl1}^h{z}_{jp})})}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-{\displaystyle \sum _{p=1}^P(w_{pl3}^h{z}_{jp})})}}}\right)\right)}}$$

(70)

$$o_{j3}=\frac{1}{1+e^{\left({\theta }_1^o-\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-{\displaystyle \sum _{p=1}^P(w_{pl1}^h{z}_{jp})})}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-{\displaystyle \sum _{p=1}^P(w_{pl3}^h{z}_{jp})})}}}\right)\right)}}$$

(71)

The following requirements need to be met:

$$\frac{1}{1+e^{\left({\theta }_3^o-\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-{\displaystyle \sum _{p=1}^P(w_{pl1}^h{z}_{jp})})}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-{\displaystyle \sum _{p=1}^P(w_{pl3}^h{z}_{jp})})}}}\right)\right)}}\le \min (o_{j2}^{*},\hspace{0.33em}{a}_j); j = 1 ~ n$$

(72)

$$\frac{1}{1+e^{\left({\theta }_1^o-\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-{\displaystyle \sum _{p=1}^P(w_{pl1}^h{z}_{jp})})}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-{\displaystyle \sum _{p=1}^P(w_{pl3}^h{z}_{jp})})}}}\right)\right)}}\ge \max (o_{j2}^{*},\hspace{0.33em}{a}_j); j = 1 ~ n$$

(73)

that are equivalent to

$$\min \left({\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l3}^h-{\displaystyle \sum _{p=1}^P(w_{pl1}^h{x}_{jp})})}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l1}^o}{1+e^{({\theta }_{l1}^h-{\displaystyle \sum _{p=1}^P(w_{pl3}^h{x}_{jp})})}}}\right)\le {\theta }_3^o-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(74)

$$\max \left({\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l3}^h-{\displaystyle \sum _{p=1}^P(w_{pl1}^h{x}_{jp})})}}},\hspace{0.33em}{\displaystyle \sum _{l=1}^L\frac{w_{l3}^o}{1+e^{({\theta }_{l1}^h-{\displaystyle \sum _{p=1}^P(w_{pl3}^h{x}_{jp})})}}}\right)\ge {\theta }_1^o-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(75)

Only $w_{p_f{l}_f}^h$ is fuzzified, the other fuzzy parameters are set to their optimized cores:

$$\min \left(\begin{array}{l}\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}1}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}},\\ \hspace{0.33em}\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}3}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\end{array}\right)\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(76)

$$\max \left(\begin{array}{l}\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}1}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}},\\ \hspace{0.33em}\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}3}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\end{array}\right)\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(77)

If $w_{l_{f}2}^{o*}\ge 0$,

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}3}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(78)

As a result,

$$w_{p_f{l}_{f}1}^h\le \frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}; j = 1 ~ n$$

(79)

$$w_{p_f{l}_{f}3}^h\ge \frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}; j = 1 ~ n$$

(80)

Therefore,

$$w_{p_f{l}_{f}1}^h\le \underset{j}{\min }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(81)

$$w_{p_f{l}_{f}3}^h\ge \underset{j}{\max }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(82)

To minimize the fuzziness of ${\tilde{w}}_{p_f{l}_f}^h$,

$$w_{p_f{l}_{f}1}^{h*}=\underset{j}{\min }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(83)

$$w_{p_f{l}_{f}3}^{h*}=\underset{j}{\max }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(84)

Otherwise,

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}3}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(85)

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}1}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(86)

As a result,

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}3}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\le {\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(87)

$$\frac{w_{l_{f}2}^{o*}}{1+e^{({\theta }_{l_{f}2}^{h*}-w_{p_f{l}_{f}1}^h{x}_{j{p}_f}-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})})}}+{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}\ge {\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1); j = 1 ~ n$$

(88)

Therefore,

$$w_{p_f{l}_{f}3}^h\ge \underset{j}{\max }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(89)

$$w_{p_f{l}_{f}1}^h\le \underset{j}{\min }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(90)

To minimize the fuzziness of ${\tilde{w}}_{p_f{l}_f}^h$,

$$w_{p_f{l}_{f}3}^{h*}=\underset{j}{\max }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\min (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(91)

$$w_{p_f{l}_{f}1}^{h*}=\underset{j}{\min }\left(\frac{{\theta }_{l_{f}2}^{h*}-\ln (\frac{w_{l_{f}2}^{o*}}{{\theta }_2^{o*}-\ln (\frac{1}{\max (o_{j2}^{*},\hspace{0.33em}{a}_j)}-1)-{\displaystyle \sum _{l\ne l_f}\frac{w_{l2}^{o*}}{1+e^{({\theta }_{l2}^{h*}-{\displaystyle \sum _{p=1}^P(w_{pl2}^{h*}{x}_{jp})})}}}}-1)-{\displaystyle \sum _{p\ne p_f}(w_{p{l}_{f}2}^{h*}{x}_{jp})}}{x_{j{p}_f}}\right)$$

(92)

3. An Illustrative Case Using FFNN(3, 6, 1)

The problem of predicting the time to replace a computer numeric control (CNC) tool based on the monitoring results of three sensors is adopted to illustrate the ap-plicability of the proposed methodology. Therefore, P = 3. A small-scale problem is used, so that the constructed FFNN will not be too large, and the effect of fuzzifying each parameter can be more obvious. The collected data in-cludes ninety records, as shown in

Table 1. Collected data.

j	${\mathit{x}}_{\mathit{j}1}$	${\mathit{x}}_{\mathit{j}2}$	${\mathit{x}}_{\mathit{j}3}$	${\mathit{y}}_{\mathit{j}}$
1	265	30	2028	468
2	224	40	2018	507
3	173	52	2641	811
4	151	36	1837	468
5	322	55	2274	776
6	167	56	2508	926
90	311	39	2170	468
Min	125	29	1173	463
Max	364	57	3269	967

. The collected data are first normalized (see

Table 2. Normalized data.

j	${\mathit{z}}_{\mathit{j}1}$	${\mathit{z}}_{\mathit{j}2}$	${\mathit{z}}_{\mathit{j}3}$	${\mathit{a}}_{\mathit{j}}$
1	0.585	0.036	0.408	0.010
2	0.415	0.392	0.403	0.087
3	0.199	0.835	0.700	0.691
4	0.111	0.267	0.317	0.010
5	0.825	0.942	0.525	0.621
6	0.174	0.962	0.637	0.919
90	0.780	0.340	0.476	0.010

).

The FFNN has a hidden layer with six nodes. Therefore, it is indicated with FFNN(3, 6, 1) afterwards. There is no absolute rule for determining the optimal num-ber of nodes in the hidden layer. In many studies, it has been shown that a hidden layer with twice the number of inputs is sufficient to fit a complex nonlinear relationship [26,27,28]. The first sixty records are used to train the FFNN, while the rest records are left for evaluating the forecasting performance.

First, the FFNN(3, 6, 1) is regarded as a crisp FNN(3, 6, 1) and trained using the LM algorithm to derive the cores of fuzzy parameters. Other training algorithms, such as the gradient descent (GD) algorithm, the Broyden−Fletcher−Goldfarb−Shanno (BFGS) quasi-Newton algorithm, the GD algorithm with momentum and adaptive learning rate (GDX), and the resilient backpropagation (RP) algorithm [25], are also applicable. However, this research aims to improve the forecasting precision, rather than the forecasting accuracy. The choice of the training algorithm does not affect the application of the proposed methodology.

The optimal values of cores are summarized in

Table 3. Optimized cores of fuzzy parameters.

$w_{112}^{h*}$	$w_{122}^{h*}$	$w_{132}^{h*}$	$w_{142}^{h*}$	$w_{152}^{h*}$	$w_{162}^{h*}$	$w_{212}^{h*}$	$w_{222}^{h*}$	$w_{232}^{h*}$	$w_{242}^{h*}$
2.034	3.410	−0.349	−2.615	−2.557	1.213	3.912	−1.746	1.813	3.187
$w_{252}^{h*}$	$w_{262}^{h*}$	$w_{312}^{h*}$	$w_{322}^{h*}$	$w_{332}^{h*}$	$w_{342}^{h*}$	$w_{352}^{h*}$	$w_{362}^{h*}$	${\theta }_{12}^{h*}$	${\theta }_{22}^{h*}$
−2.090	3.082	3.733	−0.424	3.964	3.831	−4.754	4.911	3.222	5.251
${\theta }_{32}^{h*}$	${\theta }_{42}^{h*}$	${\theta }_{52}^{h*}$	${\theta }_{62}^{h*}$	$w_{12}^{o*}$	$w_{22}^{o*}$	$w_{32}^{o*}$	$w_{42}^{o*}$	$w_{52}^{o*}$	$w_{62}^{o*}$
9.183	2.293	2.569	4.785	4.844	4.740	−2.641	3.727	−4.905	3.668
${\theta }_2^{o*}$
9.475

. The crisp forecasts for the training data based on the cores of fuzzy parameters are shown in Figure 1. The forecasting accuracy, measured in terms of root mean squared error (RMSE), is 0.084 (normalized value). Although the forecasting accuracy is satisfactory, there are many records with considerable deviations between actual values and crisp forecasts, showing the necessity of estimating the range of the actual value. To this end, the ef-fects of fuzzifying four parameters, ${\theta }^o$, $w_1^o$, ${\theta }_1^h$, and $w_{11}^h$, are compared. There are four types of parameters in the FNN(3, 6, 1): the connection weights between the input layer and the hidden layer, the thresholds on the nodes of the hidden layer, the connection weights between the hidden layer and the output lay-er, and the threshold on the node of the output layer. In this way, the effects of fuzzi-fying all types of parameters can be observed and compared.

First, ${\theta }^o$ is fuzzified to minimize the average range of fuzzy forecasts. To this end, the lower and upper bounds of ${\tilde{\theta }}^o$ are to be derived. For this purpose, Equations (21) and (22) are applied. As a result, ${\theta }_1^{o*}=4.646$ and ${\theta }_3^{o*}=11.690$. Therefore, ${\tilde{\theta }}^o=(4.646,\hspace{0.33em}9.475,\hspace{0.33em}11.690)$. The estimated ranges by fuzzifying this parameter are shown in Figure 2. In this case, all ranges contain the corresponding actual values for the training data by fuzzifying ${\theta }^o$. The average range of fuzzy forecasts is 303.9 (unnormalized value).

The second fuzzified network parameter is $w_1^o$. To this end, Equations (37) and (38) are applied to derive the lower and upper bounds of ${\tilde{w}}_1^o$. The result is ${\tilde{w}}_1^o$ = (1.892, 4.844, 66.88). The ranges estimated by fuzzifying $w_1^o$ are shown in Figure 3. All actual values in the training data are contained in the corresponding fuzzy forecasts. However, the average range widens to 455.4.

Subsequently, ${\theta }_1^h$ is fuzzified by deriving its lower and upper bounds according to Equations (58) and (59), since $w_{12}^{o*}$ is positive. The optimal solution is ${\tilde{\theta }}_1^h$ = (−6.477, 3.222, 5.798). The estimated ranges of actual values are shown in Figure 4. However, it is not possible for all fuzzy forecasts to contain the corresponding actual values solely by fuzzifying ${\theta }_1^h$. The hit rate is only 67%. With such a low hit rate, the average range is narrowed to only 93.5.

The last parameter fuzzified in the experiment is $w_{11}^h$. Since $w_{12}^{o*}\ge 0$, Equations (84) and (85) are applied to derive the lower and upper bounds of ${\tilde{w}}_{11}^h$. The result is ${\tilde{w}}_{11}^h$ = (−84.42, 2.034, 247.4). The ranges of actual values estimated by fuzzifying this parameter are summarized in Figure 5. The hit rate is only 65%, accompanied by an average range of 160.4.

From the experimental results, the following discussion is made:

• Fuzzifying some network parameters may not guarantee that all actual values are contained in the estimated ranges.
• In contrast, fuzzifying a network parameter closer to the output node is more like to ensure a 100% hit rate.
• Both the ranges estimated by fuzzifying ${\theta }^o$ and $w_1^o$ contain the actual value. Therefore, the fuzzy intersection (FI) of the ranges also contain the actual value, which further narrows the range of the actual value.
• After applying the trained FFNN(3, 6, 1) to the test/unlearned data, the fore-casting precision levels achieved by fuzzifying various network parameters are evaluated and compared in

  Table 4. Forecasting precision levels achieved by fuzzifying various network parameters.

  	$\mathbf{Fuzzifying} {\mathit{\theta }}^{\mathit{o}}$	$\mathbf{Fuzzifying} {\mathit{w}}_1^{\mathit{o}}$	$\mathbf{Fuzzifying} {\mathit{\theta }}_1^{\mathit{h}}$	$\mathbf{Fuzzifying} {\mathit{w}}_{11}^{\mathit{h}}$
  Hit Rate	97%	100%	63%	63%
  Average Range	367.3	457.9	122.7	228.5

  . As expected, the hit rate has decreased compared to the results when applied to the training data, but is still acceptable. Fuzzifying $w_1^o$ achieves the highest hit rate, while fuzzifying ${\theta }_1^h$ minimizes the average range of fuzzy forecasts.
• The effectiveness (i.e., forecasting precision) and efficiency of the proposed methodology is compared with those of some existing methods in

  Table 5. Comparing the effectiveness and efficiency of the proposed methodology with those of existing methods.

  Method	Hit Rate	Average Range	Execution Time (sec)
  FFNN-NLP [18]	82%	407	168
  FFNN-GP [20]	76%	482	46
  FFNN (only ${\tilde{\theta }}^o$ fuzzified) [22]	97%	367	<1
  The proposed methodology $w_1^o$ fuzzified)	100%	458	<1

  . All methods are implemented using MATLAB 2017a on a PC with an i7-7700 CPU of 3.6 GHz and 8 GB of RAM. Obviously, the proposed methodology maximized the hit rate for the test data without considerably widening the average range. In addition, the proposed methodology is also the most effi-cient method.

4. Conclusions and Future Research Directions

Many complex FFNNs have been constructed to improve the forecasting accuracy. Even so, a forecast is rarely equal to the actual value. In addition, the range of a fuzzy forecast generated by prevalent FFNNs does not necessarily include the actual value. In order to solve these problems, this research explores how to fuzzify the parameters of a FNN so that every fuzzy forecast generated by the FFNN contains the actual value. To achieve this goal, most previous studies have solved an NLP problem, which was computationally challenging. In contrast, this research proposes an independent fuzzification approach to fuzzify the parameters of a FNN independently. In this way, the optimal value of each fuzzy parameter can be derived theoretically, thereby enabling the construction of a precise FFNN.

After applying the proposed methodology to an illustrative case FFNN(3, 6, 1), the following conclusions are drawn:

• Fuzzifying ${\theta }_1^h$ and $w_{11}^h$ alone cannot guarantee that all fuzzy forecasts contain corresponding actual values.
• Fuzzifying ${\theta }^o$ and $w_1^o$ has a higher chance of achieving a 100% hit rate.
• Parameters closer to the output node have a greater impact on the forecast-ing precision, and should be fuzzified earlier.
• Fuzzifying parameters far away from the output node cannot guarantee a 100% hit rate. Therefore, multiple such parameters should be fuzzified at the same time.

The FFNN discussed in this study is an FFNN with a single hidden layer. The pro-posed methodology can be extended to deal with a deep FFNN with multiple hidden layers [29,30,31] or recurrent layers [32]. In this case, the parameters of the output layer will be fuzzified first, then the parameters of the hidden layer closest to the output layer, and so on. In addition, FI can also be applied to aggregate the estimated ranges by fuzzifying various parameters (with 100% hit rates) to further enhance the fore-casting precision. These constitute some direction for future research.