Robust Process Parameter Design Methodology: A New Estimation Approach by Using Feed-Forward Neural Network Structures and Machine Learning Algorithms

Abstract

In robust design (RD) modeling, the response surface methodology (RSM) based on the least-squares method (LSM) is a useful statistical tool for estimating functional relationships between input factors and their associated output responses. Neural network (NN)-based models provide an alternative means of executing input-output functions without the assumptions necessary with LSM-based RSM. However, current NN-based estimation methods do not always provide suitable response functions. Thus, there is room for improvement in the realm of RD modeling. In this study, a new NN-based RD modeling procedure is proposed to obtain the process mean and standard deviation response functions. Second, RD modeling methods based on the feed-forward back-propagation neural network (FFNN), cascade-forward back-propagation neural network (CFNN), and radial basis function network (RBFN) are proposed. Third, two simulation studies are conducted using a given true function to verify the proposed three methods. Fourth, a case study is examined to illustrate the potential of the proposed approach. In conclusion, a comparative analysis of the three feed-forward NN structure-based modeling methods and conventional LSM-based RSM proposed in this study showed that the proposed methods were significantly lower in the expected quality loss (EQL) and various variability indicators.

1. Introduction

In recent decades, robust design (RD) has been considered essential for improvement of product quality, as the primary purpose of RD is to seek a set of parameters that make a product insensitive to various sources of noise factors. In other words, RD attempts to minimize the variability of quality characteristics while ensuring that the process mean meets the target value. To solve the RD problem, Taguchi [1] considered both the process mean and variance as a single performance measure and defined a number of signal-to-noise ratios to obtain the optimal factor settings. Unfortunately, the orthogonal arrays (OAs), statistical analysis, and signal-to-noise ratios associated with this technique were criticized by Box et al. [2], Leon et al. [3], Box [4], and Nair [5]. Therefore, Vining and Myers [6] proposed the dual response (DR) approach based on response surface methodology (RSM), in which the process mean and variance are estimated separately as functions of control factors. The result is an RD optimization model in which the process mean is prioritized by setting it as a constraint and the process variability is set as an objective function. From this starting point, the three sequential steps of the RD procedure were generated: design of experiment (DoE), estimation, and optimization. The DoE step exploits information about the relationship between the input and output variables. In the second step, the functional form of this relationship is defined by estimating the model parameters. Ultimately, the optimal settings for the input factors are identified in the third step.

As several RD optimization models have been proposed and modified according to various criteria, the optimization step is relatively well developed. The priority criterion was used in the DR models proposed by Vining and Myers [6], Copeland and Nelson [7], and Del Castillo and Montgomery [8], whereas the process mean and variance were considered simultaneously in the mean squares error (MSE) model developed by Lin and Tu [9]. The weight criterion was used to consider the trade-off between the mean and variance in the weighted sum models reported by Cho et al. [10], Ding et al. [11], and Koksoy and Doganaksoy [12], whereas Ames et al. [13] used the weight criterion in a quality loss function model. Shin and Cho [14,15] extended the DR model by integrating the customized maximum value on the process bias and variance. Based on the MSE concept, Robinson et al. [16] and Truong and Shin [17] proposed generalized linear mixed models and inverse problem models. Using the goal programming approach, Kim and Cho [18] and Tang and Xu [19] introduced prioritized models, whereas Borror [20] and Fogliatto [21] both considered the output responses on the same scale as the desirability functions. In an attempt to identify the Pareto efficient solutions, Kim and Lin [22] and Shin and Cho [23] developed a fuzzy model and a lexicographical weighted Tchebycheff model for the RD multiple-objective optimization problem, respectively. Furthermore, Goethals and Cho [24] integrated the economic factor in the economic time-oriented model to handle time-oriented dynamic characteristics, while Nha et al. [25] introduced the lexicographical dynamic goal programming model.

The DoE is a systematic method that aims to identify the effects of controllable factors on the quality characteristics of interest. DOE has been developed since the 1920s. Montgomery [26] reviewed the history of DoE in the published literature. Several DoE techniques were developed and intensively researched as a means of conducting experiments in industrial applications; these include full factorial designs, fractional factorial designs (screening designs), mixture designs, Box-Benken designs, central composite designs (CCD), Taguchi array designs, Latin square designs and other non-conventional methods (D-optimal designs).

Although many techniques for the estimation stage of an RD process are reported in literature, there is room for improvement. Indeed, the accuracy and reliability of prediction and optimization depend directly on the estimation results. Additionally, most regression methods are RSM-based approaches which commonly rely on assumptions, such as normality and homogeneous variance of the response data. However, in practice, these assumptions may not be maintained.

Along with the development of RD, RSM has been widely applied in various fields of applied statistics. The most extensive applications of RSM are in the industrial world, particularly in situations where several input variables may influence some performance measure or quality characteristic of the product or process [27]. RSM uses mathematical and statistical techniques to explore the functional relationship between input control variables and an output response variable of interest, with the unknown coefficients in this functional relationship typically estimated by the least-squares method (LSM). The usual assumptions behind LSM-based RSM are that the experimental data and error terms must be normally distributed, and the distribution of error terms must have constant variance and zero mean. When one of these assumptions is violated, the Gauss-Markov theory no longer holds. Instead, alternative techniques can be applied, such as the maximum likelihood estimation (MLE), weighted least-squares (WLS), and Bayesian methods. From the viewpoint of MLE, the model parameters are regarded as fixed and unknown quantities, and the observed data are considered as random variables [28]. Truong and Shin [17,29] showed that LSM-based RSM does not always estimate the input-output functions effectively, so they developed a procedure to estimate these unknown coefficients using an inverse problem.

In recent decades, neural networks (NNs) have become a hot topic of research; NNs are now widely used in various fields, including speech recognition, multi-objective optimization, function estimation, and classification. NNs can model linear and nonlinear relationships between inputs and outputs without any assumptions based on the activation function’s generalization capacity. NNs are universal functional approximators, and Irie and Miyake [30], Funahashi [31], Cybenko [32] and Hornik et al. [33], and Zainuddin and Pauline [34] showed that NNs are capable of approximating any arbitrary nonlinear function to the desired accuracy without the knowledge of predetermined models, as NNs are data-driven and self-adaptive. Therefore, a NN provides a powerful regression method to model the functional relationship between RD input factors and output responses without making any assumptions. In RD settings, Rowlands et al. [35] integrated an NN into RD by using the NN to conduct the DoE stage. Su and Hsieh [36] applied two NNs to train the data to obtain the optimal parameter sets and predict the system response value. Cook et al. [37] developed an NN model to forecast a set of critical process parameters and employed a genetic algorithm to train the NN model to achieve the desired level of efficiency. The integration of NNs into RD has also been discussed by Chow et al. [38], Chang [39], and Chang and Chen [40]. Arungpadang and Kim [41] developed a feed-forward NN-based RSM to model the functional relationship between input variables and output responses to improve the precision of estimation without increasing the number of experimental runs. Sabouri et al. [42] proposed an NN-based method for function estimation and optimization by adjusting weights until the response reached the target conditions. With regard to input-output relationship modeling, Hong and Satriani [43] also discussed a convolutional neural network (CNN) with an architecture determined by Taguchi’s orthogonal array to predict renewable power. Recently, Arungpadang et al. [44] proposed a hybrid neural network-genetic algorithm to predict process parameters. Le et al. proposed NN-based response function estimation (NRFE) identifies a new screening procedure to obtain the best transfer function in an NN structure using a desirability function family while determining its associated weight parameters [45]. Le and Shin propose an NN-based estimation method as a RD modeling approach. The modeling method based on a feedback NN approach is first integrated in the RD response functions estimation. Two new feedback NN structures are then proposed. Next, the existing recurrent NNs (i.e., Jordan-type and Elman-type NNs) and the proposed feedback NN approaches are suggested as an alternative RD modeling method [46].

The primary motive of this research is to establish feed-forward NN structure-based estimation methods as alternative RD modeling approaches. First, an NN-based estimation framework is incorporated into the RD modeling procedure. Second, RD modeling methods based on the feed-forward back-propagation neural network (FFNN), cascade-forward back-propagation neural network (CFNN), and radial basis function network (RBFN) are proposed. These are applied to estimate the process mean and standard deviation response functions. Third, the efficiency of the proposed modeling methods is illustrated through simulation studies with a given real function. Fourth, the efficacy of the proposed modeling methods is illustrated through a printing case study. Finally, the results of comparative studies show that the proposed methods obtain better optimal solutions than conventional LSM-based RSM. The proposed estimation methods based on feed-forward NN structures are illustrated in Figure 1. From the experimental data, the optimal numbers of hidden neurons in the FFNN and CFNN structures and the dispersion constant “spread” in the RBFN are identified to finalize the optimal structures of the corresponding NNs. The DR functions can be separately estimated using the proposed estimation methods from the optimal NN structures with their control factors and output responses.

The statistical estimation method of RSM based on conventional LSM, as introduced by Box and Wilson [47], generates the response surface approximation using linear, quadratic, and other functions, while the coefficients are estimated by minimizing the total error between the actual and estimated values. For a more comprehensive understanding of RSM, Myers [48] and Khuri and Mukhopadhyay [49] discuss the various development stages and future directions of RSM. When the exact functional relationship is unknown or very complicated, conventional LSM is typically used to estimate the input-output functional responses in RSM [50,51]. In general, the output response $y$ can be identified as a function of input factors $x$ as follows:

$$y=x\mathit{\beta }+\epsilon$$

(1)

where $x$ is a vector of the control factors, $\mathit{\beta }$ is a column vector of the estimated parameters, and $\epsilon$ is the random error. The estimated second-order models for the process mean ${\widehat{\mu }}_{LSM}$ and standard deviation ${\widehat{\sigma }}_{LSM}$ are represented as

$${\widehat{\mu }}_{LSM}\left(x\right)={\widehat{\beta }}_0+{\displaystyle \sum }_{i=1}^p{\widehat{\beta }}_i{x}_i+{\displaystyle \sum }_{i=1}^p{\widehat{\beta }}_i{x}_i^2+{\displaystyle \sum }_{\begin{array}{c}i=1\\ i<j\end{array}}^p{\displaystyle \sum }_{j=1}^p{\widehat{\beta }}_{ij}{x}_i{x}_j$$

(2)

$${\widehat{\sigma }}_{LSM}\left(x\right)={\widehat{\delta }}_0+{\displaystyle \sum }_{i=1}^p{\widehat{\delta }}_i{x}_i+{\displaystyle \sum }_{i=1}^p{\widehat{\delta }}_i{x}_i^2+{\displaystyle \sum }_{\begin{array}{c}i=1\\ i<j\end{array}}^p{\displaystyle \sum }_{j=1}^p{\widehat{\delta }}_{ij}{x}_i{x}_j$$

(3)

where $\widehat{\beta }$ and $\widehat{\delta }$ are the estimators of unknown parameters in the mean and standard deviation functions, respectively. These coefficients are estimated using LSM as

$$\hat{\beta }={\left({\mathbf{X}}^{\mathrm{T}}\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\mathrm{T}}\overline{y} \mathrm{and} \hat{\delta }={\left({\mathbf{X}}^{\mathrm{T}}\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\mathrm{T}}s$$

(4)

where $\overline{y}$ and $s$ are the average and standard deviation values for the experimental data, respectively.

2. Proposed Feed-Forward NN Structure-Based Estimation Methods

Abbreviations and main variables used in this study are summarized in Appendix D.

2.1. NN Structures

An artificial neuron is a computational model inspired by biological neurons [52]. A NN consists of numerous artificial neurons or nodes and the connections between them, with weight coefficients applied to each connection. Based on the pattern of connection, NNs typically fall into two distinct categories: feed-forward networks, in which the connection flows unidirectionally from input to output, and recurrent networks, in which the connections among layers run in both directions. Feed-forward networks are the most popular type for function approximation [34]. Therefore, this study uses feed-forward back-propagation networks and RBFN structures to estimate the desired functions.

2.2. Proposed NN-Based Estimation Method 1: FFNN Structure

2.2.1. Response Function Estimation Using FFNN

According to Taguchi’s philosophy, there are two output responses of interest, namely, the process mean and standard deviation. These output responses can be estimated simultaneously from the output layer in a single FFNN. Besides, as demonstrated by Cybenko [32], any continuous function of n input variables can be approximated by an FFNN with only one hidden layer. A similar notion is also discussed by Funahashi [31], Hornik et al. [33], and Hartman et al. [53]. The proposed FFNN-based RD modeling method with one hidden layer is illustrated in Figure 2.

The input layer has $k$ control factors, $x_1$, …, $x_i$, …, $x_k$. The input for each hidden neuron is the weighted sum of these $k$ factors plus their associated bias, expressed as ${\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+a_j^{hid}$. This calculated value will be transformed by a transformation function ($f_1$), also called an activation function (expressed by Equation (5)). The transformed value $y_j^{hid}$ is both the output of the hidden neuron and the input for the next layer (in our case, the output layer). Subsequently, the ultimate transformed outcome ${\widehat{y}}_{FFNN}$ is the final result. The outputs of the hidden neuron and FFNN can then be expressed as

$$y_j^{hid}{=f}_1\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i{+a}_j^{hid}\right)$$

(5)

where $w_{ji}$ is the weight connecting input factor $x_i$ to hidden node j; $a_j^{hid}$ is the associated bias of the hidden layer node j, and $f_1$ is the transfer function of the hidden layer.

$${\widehat{y}}_{FFNN}=g_1\left\{{\displaystyle \sum }_{j=1}^h{w}_j{y}_j^{hid}+a^{out}\right\}=g_1\left\{{\displaystyle \sum }_{j=1}^h{t}_j\left[f_1\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+a_j^{hid}\right)\right]+b^{out}\right\}$$

(6)

where $g_1$ is the activation function of the output layer, h is the number of hidden nodes, $t_j$ is the weight connecting each hidden node $y_j^{hid}$ to the output layer, and $b^{out}$ is the output bias.

The typical transfer functions for the hidden and output layers are the hyperbolic tangent sigmoid and linear functions, respectively. The estimated mean and standard deviation functions in this case are

$$\begin{array}{cc}\hfill {\widehat{\mu }}_{FFNN}\left(x\right)={\displaystyle \sum _{j=1}^{h\_mean}}& \left\{t_j^{mean}\left[\frac{2}{1+exp\left(-2\left({{\displaystyle \sum }}_{i=1}^k{w}_{ij}^{mean}{x}_i+a_j^{mean}\right)\right)}-1\right]\right\}\hfill \\ & +b_{mean}^{FFNN}\hfill \end{array}$$

(7)

$${\widehat{\sigma }}_{FFNN}\left(x\right)={\displaystyle \sum }_{j=1}^{h\_std}\left\{t_j^{std}\left[\frac{2}{1+exp\left(-2\left({{\displaystyle \sum }}_{i=1}^k{w}_{ij}^{std}{x}_i+a_j^{std}\right)\right)}-1\right]\right\}+b_{std}^{FFNN}$$

(8)

where $h\_mean$ and $h\_std$ represent the number of hidden neurons in the mean and standard deviation estimation model, respectively. Moreover, $a_j^{mean}$ and $a_j^{std}$ represent the bias for hidden node $j$ in both models, while $b_{mean}^{FFNN}$ and $b_{std}^{FFNN}$ denote the bias of the output neuron. Similarly, $t_j^{mean}$ and $t_j^{std}$ signify the weight connecting a hidden node $j$ to the output for both models, while $w_{ij}^{mean}$ and $w_{ij}^{std}$ refer to the weight connecting the input factors to a hidden node $j$ of the FFNN estimation model for mean and standard deviation, respectively.

2.2.2. Number of Hidden Neurons

For a NN model with a single hidden layer, the number of neurons in the hidden layer must be chosen carefully since use of a larger number of hidden neurons can generate a model that more accurately reflects the training data. However, use of a large number of hidden neurons makes the model increasingly complex and may lead to overfitting. On the flip side, use of only a few hidden neurons may cause underfitting. Several researchers have proposed approaches to determine the ideal number of hidden neurons. A review that discusses how to fix the number of hidden neurons in NNs was presented by Sheela and Deepa [54], but no single method is effective in every circumstance. In this paper, the well-established Schwartz’s Bayesian criterion, or the Bayesian information criterion (BIC), is used to determine the number of hidden neurons. This criterion is defined as

$$BIC=n\ast ln\left(\frac{1}{n}{\displaystyle \sum }_{i=1}^n{E}^2\right)+p\ast ln \left(n\right)$$

(9)

where n and $p$ denote the sample size and the number of parameters, respectively. In BIC, the term $ln \left(n\right)$ generally penalizes free parameters strongly. In addition, the accuracy of the NN model will increase as the sample size increases.

2.2.3. Integration into a Learning Algorithm: Back-Propagation

NN-based estimation requires a learning algorithm, such as error correction, perception learning, Boltzmann learning, Hebbian rules, or back-propagation (BP). Among these, BP is one of the most popular network training algorithms because it is both simple and generally applicable [55]. The back-propagation algorithm trains a NN by applying the chain rule method. The weights of the network are randomly initialized, then repeatedly adjusted through minimization of the cost function, which is calculated by measuring the difference between the actual output ${\widehat{y}}_{out}$ and the desired value $y_{out}$. One of the most common error measures is the root mean square (RMS) error, defined by

$$E=\frac{1}{2}\sum {\left(y_{out}-{\widehat{y}}_{out}\right)}^2$$

(10)

During the training process, the RMS error (cost function) is minimized as much as possible. The iterative step of the gradient descent algorithm then changes the weights $w_j$ according to

$$w_j\to w_j+\Delta w_j \mathrm{where} \Delta w_j=-\eta \frac{\partial E\left(w\right)}{\partial w_j}.$$

(11)

Here, $\eta$ is the learning rate, which can determine the effect of the gradient. There are a few alternative optimization techniques for finding the local minimum, such as conjugate gradient, steepest descent, and Newton’s method.

• Steepest descent is an iterative method that finds the local minimum by moving in the direction opposite to the one implied by the gradient; the learning rate (such as traingda or traingdx) indicates how quickly it moves. Usually, the smaller the learning rate, the slower the process of convergence to a local minimum. In standard steepest descent, the learning rate remains constant throughout the training stage while the weight and bias parameter values are interactively updated.
• The resilientBP training algorithm (i.e., trainrp) is a local adaptive learning scheme for supervised batch learning in an FFNN. Resilient back-propagation is similar to the standard BP algorithm, but it is capable of training the NN model faster than the regular method without the need to specify any free parameter values. Additionally, since trainrp merely considers the size of the partial derivatives, the direction of the weight update is only affected by the sign of the derivative.
• The Levenberg-Marquardt algorithm (i.e., trainlm) is a standard method for solving nonlinear least-squares minimization problems without computing the Hessian matrix. It can be thought of as a middle ground between the steepest descent and Gauss-Newton methods.

2.2.4. Generalization and Overfitting Issues

NN models are often used in order to generalize the information learned from the training data to any unseen data so that it is possible to make inferences or predictions for the problem domain. Nevertheless, overfitting can occur, in which case a trained NN model will work correctly on the training data but perform terribly on test data. Overfitting can happen when the model is too complicated or has too many parameters. To avoid this problem, the additional technique of “early stopping” is often used to improve the generalizability of the model. With an early stopping mechanism, the training process can be halted in an early iteration to prevent overfitting. Normally, to ensure the accuracy and efficiency of the NN model, the whole dataset should be divided into three subsets: the training, validation, and test datasets. Training data is used for NN model construction, while validation and test data are applied to check the model’s error and efficiency, respectively. The exact proportions of the training, test, and validation datasets are determined by the designer; the most widely used ratios are 50:25:25 or 60:20:20. In this technique, the optimal point at which to stop training is indicated by the minimum estimated true error, as shown in Figure 3 [56].

2.3. Proposed NN-Based Estimation Method 2: CFNN Structure

An alternative NN-based model, the cascade forward neural network (CFNN), has been shown to boost model accuracy and learning speed in cases with complicated relationships. CFNN is similar to FFNN, except that the former also includes a weighted link between the input and output layers. The additional connection means that the inputs can directly affect the output, which increases the complexity of the model but also increases its accuracy. Likewise, the CFNN model uses the BP algorithm to update the weights but has a drawback in that each layer of neurons is related to all previous layers of neurons [57]. The proposed CFNN-based modeling method is illustrated in Figure 4. The output of CFNN is expressed as follows:

$${\widehat{y}}_{CFNN}=g_2\left\{{\displaystyle \sum }_{o=1}^l{q}_o\left[f_2\left({\displaystyle \sum }_{i=1}^k{r}_{oi}{x}_i+c_o^{hid}\right)\right]+{\displaystyle \sum }_{i=1}^k{z}_{1i}{x}_i+d\right\}$$

(12)

Again, the hyperbolic tangent sigmoid and linear functions are usually used as the transfer functions in the hidden and output layers, respectively. In this case, the derived mean and standard deviation functions are expressed as

$${\widehat{\mu }}_{CFNN}\left(x\right)={\displaystyle \sum }_{o=1}^{l\_mean}\left\{q_o^{mean}\left[\frac{2}{1+exp\left(-2\left({{\displaystyle \sum }}_{i=1}^k{r}_{oi}^{mean}{x}_i+c_o^{mean}\right)\right)}-1\right]\right\}+{\displaystyle \sum }_{i=1}^k{z}_{1i}^{mean}{x}_i+d_{mean}^{CFNN}$$

(13)

$${\widehat{\sigma }}_{CFNN}\left(x\right)={\displaystyle \sum }_{o=1}^{l\_std}\left\{q_o^{std}\left[\frac{2}{1+exp\left(-2\left({{\displaystyle \sum }}_{i=1}^k{r}_{oi}^{std}{x}_i+c_o^{std}\right)\right)}-1\right]\right\}+{\displaystyle \sum }_{i=1}^k{z}_{1i}^{std}{x}_i+d_{std}^{CFNN}$$

(14)

where $l\_mean$ and $l\_std$, $c_o^{mean}$ and $c_o^{std}$, $d_{mean}^{CFNN}$ and $d_{std}^{CFNN}$, $q_o^{mean}$ and $q_o^{std}$, $r_{oi}^{mean}$ and $r_{oi}^{std}$, $z_{1i}^{mean}$, and $z_{1i}^{std}$ denote the number of hidden neurons, bias at hidden node $o$, bias at the output neuron, weight connecting hidden node $o$ to the output neuron, weight connecting input factor $x_i$ to hidden node $o$, and weights directly linking input factor $x_i$ to the output neuron of the CFNN for the mean and standard deviation, respectively.

2.4. Proposed NN-Based Estimation Method 3: RBFN

RBFN generally gives excellent, fast approximations for curve-fitting problems. This multilayer feed-forward NN has a single hidden layer and uses radial basis activation functions in the hidden neurons. The radial basis function (kernel) is a function whose value depends on the distance between the input value and center points, where the centers and width can be determined by methods such as random selection, k-means clustering, supervised selection, unsupervised learning, etc. [58]. Unlike in an FFNN, the output of the hidden neuron $y_{hid}$ is given by the product of the bias and the Euclidean distance between the input and weight vectors. Figure 5 shows the RBFN structure used to estimate the process mean and standard deviation functions.

In this essay, the design of an RBFN requires a two-step training procedure. First, the kernel positions, kernels widths, and weights that link the hidden layer inputs to the output layer nodes are estimated using an unsupervised LMS algorithm. After this initial solution has been obtained, a supervised gradient-based algorithm can then be used to refine the network parameters. The dispersion constant, or spread, exhibits a strong relationship with the output of the network. Specifically, the input region covers a large area when the spread is vast; if the spread is relatively narrow, the radial basis function curve is steeper, and the neuron output is much more significant relative to the weighted input vector approach. As a result, the network output is closer to the expected output [59]. The output of RBFN is expressed as

$${\widehat{y}}_{out}={\displaystyle \sum }_{p=1}^m\left\{u_{ip}\left[exp-{\left(\frac{\left|\right|{{\displaystyle \sum }}_{i=1}^k\left(v_{ip}-x_i\right)\left|\right|e_p}{2{\sigma }^2}\right)}^2\right]\right\}+f$$

(15)

In this case, the estimated mean and standard deviation functions are

$${\widehat{\mu }}_{RBFN}\left(x\right)={\displaystyle \sum }_{p=1}^{m\_mean}\left\{u_p^{mean}\left[exp-{\left(\frac{\left|\right|{{\displaystyle \sum }}_{i=1}^k\left(v_{pi}^{\mathrm{mean}}-x_i\right)\left|\right|e_p^{mean}}{2{\sigma }^2}\right)}^2\right]\right\}+f_{mean}^{RBFN}$$

(16)

$${\widehat{\sigma }}_{RBFN}\left(x\right)={\displaystyle \sum }_{p=1}^{m\_std}\left\{u_p^{std}\left[exp-{\left(\frac{\left|\right|{{\displaystyle \sum }}_{i=1}^k\left(v_{pi}^{\mathrm{std}}-x_i\right)\left|\right|e_p^{std}}{2{\sigma }^2}\right)}^2\right]\right\}+f_{std}^{RBFN}$$

(17)

where $m\_mean$, $m\_std$, $e_p^{mean}$, $e_p^{std}$, $f_{mean}^{RBFN}$, $f_{std}^{RBFN}$, $u_p^{mean}$, $u_p^{std}$, $v_{pi}^{mean}$, and $v_{pi}^{std}$ denote the number of hidden neurons, bias at hidden node $p$, bias at the output neuron, weight connecting hidden node $p$ to the output, and weight connecting input factor $x_i$ to hidden node $p$ of RBFN for the mean and standard deviation, respectively.

3. Simulation Studies

A variety of simulation-based examples are presented to demonstrate the efficacy of the NN-based RD modeling methods. Assume that a given exact function representing the true relationship between the input factors and output response is represented as

$$y=\left(x_2^2-4x_2\right)exp\left(-2x_1^2-3x_2^2-x_1{x}_2+4\right)+110$$

(18)

The factorial design is used to evaluate how two factors with five levels impact one response. The related experimental data are exhibited in

Table 1. Experimental data and true function response value.

Run	${\mathit{x}}_1$	${\mathit{x}}_2$	${\mathit{y}}_{\mathit{t}\mathit{r}\mathit{u}\mathit{e}}$
1	0.5	−1.0	113.5914
2	1.0	−0.5	112.9478
3	−1.0	−0.5	104.7632
4	1.0	0.0	100.0000
5	0.5	0.5	78.6806
6	0.0	0.0	100.0000
7	0.5	0.0	100.0000
8	−1.0	0.5	89.9294
9	−1.0	1.0	97.0000
10	1.0	−1.0	105.0000
11	0.0	1.0	91.8451
12	−1.0	0.0	100.0000
13	1.0	1.0	99.5939
14	−0.5	0.5	64.8503
15	0.0	−0.5	158.0282
16	−0.5	−1.0	105.0000
17	−0.5	−0.5	127.4106
18	0.5	1.0	97.0000
19	−1.0	−1.0	100.6766
20	0.0	−1.0	113.5914
21	0.0	0.5	54.8669
22	1.0	0.5	96.2952
23	−0.5	0.0	100.0000
24	−0.5	1.0	91.8451
25	0.5	−0.5	145.1924

, and the actual relationship function is shown in Equation (18). Figure 6 shows a plot of the experimental data and true response value. The replicated response value is created at each treatment level by randomly adding some deviation from the true response value. In this paper, two different simulation studies are conducted to check the efficiency of our proposed model.

In RD, the expected quality loss (EQL) is often used as a critical optimization criterion to compare different methods. The EQL is given by

$$EQL=\theta \left[{\left(\widehat{\mu }\left(x\right)-\tau \right)}^2+{\widehat{\sigma }}^2\left(x\right)\right]$$

(19)

where $\theta$ denotes the loss coefficient, normally $\theta =1$, and $\widehat{\mu }\left(x\right)$, $\widehat{\sigma }\left(x\right)$, and $\tau$ represent the estimated mean function, approximated standard deviation function, and target value, respectively. In both simulation studies, the target value is defined as 128 (${\tau }_1=128$).

3.1. Simulation Study 1

Simulation study 1 was conducted by randomly adding a number of small standard deviation values ${\sigma }_{noise,i\_small}=\sqrt{e^{0.35x_1-0.35x_2+2}}$ to the true response to give 50 replicates of the output response. Experimental data from simulation study 1 are presented in

Table 2. Experimental data from simulation study 1.

Run	${\mathit{x}}_1$	${\mathit{x}}_2$	$\overline{\mathit{y}}$	$\mathit{s}$	${\mathit{s}}^2$
1	0.50	−1.00	112.5910	5.9420	35.3060
2	1.00	−0.50	111.1480	6.3660	40.5310
3	−1.00	−0.50	105.1830	4.9330	24.3300
4	1.00	0.00	100.6400	5.2010	27.0510
5	0.50	0.50	77.4810	5.3530	28.6530
6	0.00	0.00	99.7600	5.1250	26.2680
7	0.50	0.00	101.0600	5.0650	25.6490
8	−1.00	0.50	89.9690	3.9950	15.9580
9	−1.00	1.00	96.6600	2.9110	8.4740
10	1.00	−1.00	105.8000	6.6580	44.3270
11	0.00	1.00	91.7450	4.2200	17.8060
12	−1.00	0.00	99.7800	3.7970	14.4200
13	1.00	1.00	100.3540	5.0330	25.3290
14	−0.50	0.50	66.1700	3.4490	11.8960
15	0.00	−0.50	158.7880	4.8550	23.5740
16	−0.50	−1.00	104.5000	5.0920	25.9290
17	−0.50	−0.50	127.9310	5.3540	28.6630
18	0.50	1.00	96.5800	4.4950	20.2080
19	−1.00	−1.00	100.8770	4.3940	19.3060
20	0.00	−1.00	113.8910	5.2110	27.1530
21	0.00	0.50	54.8070	4.0020	16.0170
22	1.00	0.50	95.8950	5.4510	29.7140
23	−0.50	0.00	100.8800	4.0340	16.2710
24	−0.50	1.00	91.2050	3.4390	11.8270
25	0.50	−0.50	144.5120	5.3620	28.7530

.

The approximated mean and standard deviation functions shown in Equations (20) and (21) were estimated using the LSM-based RSM method in MATLAB software.

$${\widehat{\mu }}_{LSM}\left(x\right)=103.5282+3.3709x_1-15.4188x_2-2.7589x_1^2-0.4410x_2^2-0.6280x_1{x}_2$$

(20)

$${\widehat{\sigma }}_{LSM}\left(x\right)=4.7239+0.8883x_1-0.7607x_2+0.1752x_1^2-0.0442x_2^2+0.0713x_1{x}_2$$

(21)

Information about the trained multilayer feed-forward NN model, including the transfer function and NN architecture (number of inputs, number of hidden neurons, number of outputs), is given in

Table 3. Summary of multilayer feed-forward NN information for simulation 1.

Model		Transfer Function	Training Function	Architecture	#Epoch
FFNN	Mean	Tansig-Purelin	Trainlm	2-17-1	3
	Standard deviation	Tansig-Purelin	Trainrp	2-6-1	44
CFNN	Mean	Tansig-Purelin	Trainlm	2-17-1	5
	Standard deviation	Tansig-Purelin	Trainrp	2-5-1	28

and

Table 4. Summary of RBFN information for simulation 1.

Model		Transfer Function	Goal	Spread
RBFN	Mean	Radbas-Purelin	1 × 10−7	0.6
	Standard deviation	Radbas-Purelin	1 × 10−7	0.6

. Furthermore, the weight and bias values associated with each layer of the three proposed NN models (FFNN, CFNN, and RBFN) for both the process mean and standard deviation function estimations are exhibited in Appendix A (

Table A1. Estimated parameter values (Weight and bias) of the proposed FFNN for simulation study 1.

a. Mean Function
Weights			Biases
${\mathbf{W}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$		${\left({\mathbf{t}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{a}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{b}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
3.6550	4.7000	−0.5320	−5.6150	−0.3370
4.6370	3.2890	0.7650	−5.1200
−3.6880	4.6390	0.3910	4.1080
5.4630	1.9730	−0.1880	−3.5370
1.3880	−5.7220	0.2690	−2.5120
−4.3340	3.7490	−0.2760	2.2050
−2.5860	5.1880	−0.1570	1.3530
1.4470	5.5880	−0.3560	−0.7180
4.9970	2.8280	0.1800	−0.0290
5.1210	2.5860	−0.1410	0.8320
−5.5550	−1.5900	−0.4130	−1.4570
5.4750	1.8190	−0.2640	2.1750
4.6360	−3.3980	0.0280	2.9310
−0.4150	5.7380	0.4640	−3.6420
3.1110	−4.8490	0.0690	4.3330
−4.8490	−2.9730	−0.0830	−5.0940
−1.0380	−5.9250	−0.5680	−5.5340
b. Standard Deviation Function
Weights			Biases
$W_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$		${\left(t_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	$a_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	$b_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
2.3640	−2.2900	0.6380	−3.2240	0.5100
3.1150	3.1890	0.2420	−1.9530
−1.9210	1.7710	0.1340	2.8660
0.7590	5.2410	−0.0100	1.6240
1.4680	−3.0400	0.4510	1.2370
0.8010	2.7570	0.1990	−3.3430

,

Table A2. Estimated parameter values (Weight and bias) of the proposed CFNN for mean function in simulation study 1.

Weights				Biases
${\mathbf{R}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$		${\left({\mathbf{z}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\left({\mathbf{q}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{c}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{d}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
3.4680	4.5940		0.5890	−5.7880	−0.3310
4.7000	3.4200		−0.1830	−5.0120
−3.5790	4.9050		0.5820	3.9430
5.2270	2.2140		−0.1130	−3.7440
1.5730	−5.5360		0.2110	−2.9200
−4.3610	3.8520		−0.5380	2.0050
−2.6110	5.1060		−0.2640	1.4290
1.3240	5.6320	−0.2540	−0.3850	−0.8230
5.0350	2.7780	0.3700	−0.2030	0.3890
5.1120	2.6770		0.0370	0.7280
−5.4880	−1.8040		−0.0510	−1.4370
5.4600	1.8770		0.1170	2.1230
4.5330	−3.4080		0.3510	3.1740
−0.6110	5.8300		0.0350	−3.4760
3.1690	−4.8040		−0.3720	4.4170
−4.9220	−3.0140		−0.0030	−5.0410
−0.9670	−5.6650		−0.5560	−5.7940

,

Table A3. Estimated parameter values (Weight and bias) of the proposed CFNN for standard deviation function in simulation study 1.

Weights				Biases
${\mathbf{R}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$		${\left({\mathbf{z}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\left({\mathbf{q}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{c}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{d}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
1.9650	−1.2350	−0.1600 0.1370	0.9570	−3.1820	0.9310
2.8420	−1.4330		0.3030	1.1660
−2.7860	−0.2660		−0.3060	0.4420
1.7340	3.2990		−0.4380	1.9590
0.6130	3.8920		0.1590	3.1660

and

Table A4. Estimated parameter values (Weight and bias) of the proposed RBFN in simulation study 1.

a. Mean Function					b. Standard Deviation Function
Weights			Biases		Weights			Biases
${\mathbf{V}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$		${\left({\mathbf{u}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{e}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{f}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{V}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$		${\left({\mathbf{u}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{e}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{f}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
0.0	−0.5	126.7450	1.3870	31.5280	0.5	−0.5	6.8760	1.3870	33.6620
0.5	1.0	−29.6660	1.3870		0.5	1.0	−3.6160	1.3870
−1.0	0.5	9.1310	1.3870		−1.0	−0.5	3.9420	1.3870
1.0	−0.5	−35.0720	1.3870		−1.0	1.0	−28.7180	1.3870
−1.0	−1.0	55.8470	1.3870		1.0	−1.0	−24.2000	1.3870
1.0	1.0	72.6050	1.3870		1.0	0.5	0.1440	1.3870
−0.5	1.0	7.0470	1.3870		−0.5	−0.5	8.5470	1.3870
−0.5	0.5	−52.0380	1.3870		0.0	0.5	−11.9050	1.3870
−1.0	0.0	31.4830	1.3870		−0.5	−1.0	0.5760	1.3870
1.0	−1.0	77.7170	1.3870		−1.0	0.5	5.4580	1.3870
1.0	0.0	79.6550	1.3870		0.5	0.5	11.8210	1.3870
−1.0	1.0	56.6220	1.3870		0.0	0.0	−4.1700	1.3870
0.5	0.5	43.0670	1.3870		0.0	1.0	−16.5340	1.3870
0.0	1.0	92.4890	1.3870		1.0	1.0	−23.5990	1.3870
−1.0	−0.5	13.2020	1.3870		1.0	0.0	−21.1190	1.3870
−0.5	−0.5	−60.8050	1.3870		0.0	−1.0	−18.9870	1.3870
0.0	0.5	−59.6330	1.3870		0.5	0.0	−9.2400	1.3870
−0.5	0.0	65.2090	1.3870		−1.0	0.0	−25.1250	1.3870
−0.5	−1.0	11.5900	1.3870		0.0	−0.5	−9.8510	1.3870
0.0	0.0	−2.2410	1.3870		−0.5	0.0	−7.7680	1.3870
0.5	−1.0	−31.3730	1.3870		1.0	−0.5	2.7330	1.3870
1.0	0.5	−27.4060	1.3870		−1.0	−1.0	−27.3400	1.3870
0.5	−0.5	51.3100	1.3870		0.5	−1.0	0.6880	1.3870
0.5	0.0	−51.9210	1.3870		−0.5	0.5	6.3400	1.3870

, respectively).

The conventional LSM-based RSM method is compared to the proposed FFNN-, CFNN-, and RBFN-based RD modeling methods in Figure 7, Figure 8, Figure 9 and Figure 10 in contour plot form. Furthermore,

Table 5. Comparative results in simulation study 1.

Estimation Model	Optimal Factor Settings		Process Mean	Process Bias	Process Variance	EQL
	${\mathit{x}}_1$	${\mathit{x}}_2$
LSM	−0.5870	−1.0000	119.9030	8.0960	35.7730	101.3270
FFNN	−0.5230	−0.5110	127.9900	0.0090	23.0860	23.0860
CFNN	−0.3280	−0.9290	127.9830	0.0160	27.0260	27.0260
RBFN	0.6860	−0.7690	127.9410	0.0580	24.3250	24.3280

shows the ultimate solution set and corresponding best response values, while the comparison is done in terms of EQL.

The results in Table 5 clearly show that the proposed RD modeling approach with the three NN architectures produces much smaller EQL values than the approach using LSM-based RSM. Specifically, both the process bias and variance values obtained from the proposed NN-based model are significantly lower than those of LSM-based RSM. The squared process bias vs. variance results of the conventional LSM-based RSM and proposed FFNN-, CFNN-, and RBFN-based modeling methods in simulation study 1 are illustrated in Figure 11. The optimal settings are marked with green stars in each figure.

3.2. Simulation Study 2

Simulation study 2 was conducted by randomly adding a number of large standard deviation values ${\sigma }_{noise,i\_large}=\sqrt{e^{0.35x_1-0.35x_2+5}}$ to the true response to give 50 replicates. The experimental data of simulation study 2 are presented in

Table 6. Experimental data for simulation 2.

Run	${\mathit{x}}_1$	${\mathit{x}}_2$	$\overline{\mathit{y}}$	$\mathit{s}$	${\mathit{s}}^2$
1	0.5	−1.0	117.0310	28.4560	809.7210
2	1.0	−0.5	108.9480	28.5790	816.7760
3	−1.0	−0.5	107.1230	17.1750	294.9700
4	1.0	0.0	100.5800	22.3820	500.9420
5	0.5	0.5	83.9410	22.0650	486.8490
6	0.0	0.0	98.3400	20.3830	415.4530
7	0.5	0.0	98.0000	23.8250	567.6330
8	−1.0	0.5	90.1890	16.3130	266.1150
9	−1.0	1.0	92.2200	14.7930	218.8280
10	1.0	−1.0	100.1200	26.8040	718.4340
11	0.0	1.0	91.2050	17.0730	291.5000
12	−1.0	0.0	97.5800	18.1380	328.9830
13	1.0	1.0	104.2340	21.8010	475.2960
14	−0.5	0.5	68.7300	18.4750	341.3320
15	0.0	−0.5	157.4680	23.2770	541.8020
16	−0.5	−1.0	115.1800	21.6910	470.5180
17	−0.5	−0.5	128.1910	21.9950	483.7670
18	0.5	1.0	98.3600	18.7680	352.2350
19	−1.0	−1.0	104.5970	20.9930	440.6870
20	0.0	−1.0	114.6310	26.1640	684.5700
21	0.0	0.5	53.3870	20.1200	404.8260
22	1.0	0.5	93.0950	20.7210	429.3470
23	−0.5	0.0	97.8600	19.4930	379.9600
24	−0.5	1.0	95.2650	16.5790	274.8610
25	0.5	−0.5	147.8720	25.1310	631.5690

. The estimated mean and standard deviation functions given by LSM-based RSM in simulation study 2 are:

$${\widehat{\mu }}_{LSM}\left(x\right)=103.7401+2.8205x_1-16.0324x_2-4.7497x_1^2+2.4012x_2^2+2.6455x_1{x}_2$$

(22)

$${\widehat{\sigma }}_{LSM}\left(x\right)=21.5666+3.4305x_1-3.5459x_2-0.8637x_1^2+0.2259x_2^2-0.7160x_1{x}_2$$

(23)

Similarly, detailed information about the multilayer FFNN is given in

Table 7. Summary of the multilayer FFNN in simulation 2.

Model		Transfer Function	Training Function	Architecture	#Epoch
FFNN	Mean	Tansig-Purelin	Trainlm	2-11-1	5
	Standard deviation	Tansig-Purelin	Trainrp	2-3-1	23
CFNN	Mean	Tansig-Purelin	Trainlm	2-13-1	6
	Standard deviation	Tansig-Purelin	Trainrp	2-1-1	29

and

Table 8. Summary of the RBFN in simulation 2.

Model		Transfer Function	Goal	Spread
RBFN	Mean	Radbas-Purelin	1e-20	0.5
	Standard deviation	Radbas-Purelin	1e-20	0.6

, while the weight and bias values of all proposed models for both the process mean and standard deviation functions are summarized in Appendix B (

Table A5. Estimated parameter values (Weight and bias) of the proposed FFNN in simulation study 2.

a. Mean Function					b. Standard Deviation Function
Weights			Biases		Weights			Biases
${\mathbf{W}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$		${\left({\mathbf{t}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{a}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{b}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{W}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$		${\left({\mathbf{t}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{a}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{b}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
2.0570	3.9280	0.7630	−4.7780	0.0530	1.7820	1.8550	−0.3890	−1.6150	0.6640
2.4000	3.9130	0.4830	−3.4680		2.2310	−0.7030	0.5740	0.0480
−4.5010	−0.4190	0.5650	3.0550		−0.2220	−1.8100	1.0090	−2.9250
2.9150	3.7590	0.2290	−1.3400
1.4850	−4.3180	0.9410	−1.3350
−4.6270	−0.2660	0.6220	−0.1490
−2.2500	3.9870	−0.6890	−1.0590
0.7070	4.5580	0.1550	1.5870
2.6160	3.8280	0.5310	2.4090
4.3540	1.5560	−0.9940	4.0990
−2.6170	−3.7890	−1.3930	−4.6190

,

Table A6. Estimated parameter values (Weight and bias) of the proposed CFNN for mean function in simulation study 2.

Weights				Biases
${\mathbf{R}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$		${\left({\mathbf{z}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\left({\mathbf{q}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{c}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{d}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
5.1170	−0.1120	0.3710 0.8340	−0.3840	−4.9710	0.0220
4.2260	2.9460		0.2340	−4.0130
−3.7930	−3.5990		0.0640	3.0950
4.5130	−0.8420		−0.0300	−3.2350
1.8880	4.7280		−0.2380	−1.5650
−4.2200	2.7300		0.0610	0.9880
−1.7950	4.8310		−0.4210	0.2780
1.5620	4.7170		−0.5560	0.9210
3.5040	−3.6720		0.5150	1.6860
4.1320	2.8270		0.2500	2.6160
−3.2120	4.0320		0.5630	−3.1850
4.8210	1.5590		−0.5310	4.0190
4.4480	2.5990		0.2130	4.9300

,

Table A7. Estimated parameter values (Weight and bias) of the proposed CFNN for standard deviation function in simulation study 2.

Weights				Biases
${\mathbf{R}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$		${\left({\mathbf{z}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\left({\mathbf{q}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{c}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{d}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
1.2450	0.7210	0.5230 −0.5240	−0.0310	0.2590	−0.0380

and

Table A8. Estimated parameter values (Weight and bias) of the proposed RBFN in simulation study 2.

a. Mean Function					b. Standard Deviation Function
Weights			Biases		Weights			Biases
${\mathbf{V}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$		${\left({\mathbf{u}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{e}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{f}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{V}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$		${\left({\mathbf{u}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{e}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{f}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
0.0	−0.5	64.8980	1.6650	38.201	0.5	−0.5	−28.2270	1.3870	−1.6450
0.5	1.0	−6.6220	1.6650		−0.5	0.5	8.2530	1.3870
−1.0	0.5	17.0920	1.6650		1.0	1.0	21.6710	1.3870
1.0	−0.5	−8.6860	1.6650		−1.0	−1.0	34.5170	1.3870
−1.0	−1.0	34.9310	1.6650		1.0	−1.0	9.1260	1.3870
1.0	0.5	−21.9170	1.6650		−0.5	−1.0	−17.9100	1.3870
−0.5	1.0	25.6960	1.6650		1.0	0.0	−5.4440	1.3870
0.5	−1.0	10.9180	1.6650		0.0	−1.0	22.5250	1.3870
−1.0	0.0	26.8410	1.6650		0.5	1.0	−3.2470	1.3870
−1.0	1.0	31.4250	1.6650		−1.0	0.0	32.0960	1.3870
−0.5	−1.0	34.7400	1.6650		−1.0	1.0	17.5940	1.3870
1.0	1.0	62.8200	1.6650		0.0	0.5	4.8550	1.3870
0.5	0.0	−47.0190	1.6650		−0.5	0.0	−11.1920	1.3870
1.0	−1.0	40.4510	1.6650		0.0	1.0	12.6800	1.3870
1.0	0.0	63.1560	1.6650		−1.0	−0.5	−25.7220	1.3870
0.0	1.0	55.4460	1.6650		1.0	0.5	4.0170	1.3870
−0.5	−0.5	−4.6920	1.6650		0.5	0.5	−7.4010	1.3870
0.5	−0.5	62.5500	1.6650		−0.5	−0.5	20.5440	1.3870
−0.5	0.0	17.0590	1.6650		1.0	−0.5	26.7630	1.3870
−0.5	0.5	−11.3100	1.6650		0.5	0.0	27.5990	1.3870
−1.0	−0.5	21.5900	1.6650		0.5	−1.0	13.4280	1.3870
0.5	0.5	52.6940	1.6650		−1.0	0.5	−12.6070	1.3870
0.0	0.5	−59.7970	1.6650		−0.5	1.0	−3.6890	1.3870
0.0	0.0	35.7290	1.6650		0.0	−0.5	3.9230	1.3870

). Contour plots of the process mean and standard deviation response values for each model are displayed in Figure 12, Figure 13, Figure 14 and Figure 15, respectively.

Table 9. Comparative results of simulation study 2.

Estimation Model	Optimal Factor Settings		Process Mean	Process Bias	Process Variance	EQL
	${\mathit{x}}_1$	${\mathit{x}}_2$
LSM	−1.000	−1.000	117.2490	10.7500	413.2320	528.8130
FFNN	−0.804	−0.561	127.2800	0.7190	359.3420	359.8600
CFNN	−0.383	−0.274	127.2120	0.7870	445.7840	446.4040
RBFN	−0.141	−0.190	127.0680	0.9310	422.7920	423.6600

presents the optimal input factor settings, associated process mean, process bias, process variance, and EQL values obtained from the proposed NN-based modeling methods and LSM-based RSM in simulation study 2. The proposed NN-based modeling methods again produced significantly smaller EQL values than the LSM-based RSM. The plot of squared process bias vs. the variance for LSM-based RSM and the proposed FFNN-, CFNN-, and RBNN-based modeling methods are illustrated in Figure 16. The optimal settings are marked with green stars in each figure.

Clearly, the proposed NN-based modeling methods produced better solutions than LSM-based RSM in both simulation studies. Whereas RSM is generally used to estimate second-order functions, the proposed modeling methods can effectively estimate nonlinear functions.

4. Case Study

The printing data example used by Vining and Myers [6] and Lin and Tu [9] was selected to demonstrate the application of the proposed methods. The printing experiment investigates the effects of speed ($x_1$), pressure ($x_2$), and distance ($x_3$) on a printing machine’s ability to add colored ink to a package (y). The case study has a factorial design with three level three factors, so in total, the number of experimental runs is $3^3$. To ensure the accuracy of the experiment, each treatment combination is repeated three times. In this case study, the target ${\tau }_2=500$. The experimental data are given in Vining and Myers [6]. The estimated mean and standard deviation functions given by LSM-based RSM are

$${\widehat{\mu }}_{\mathrm{LSM}}\left(x\right)=327.6296+x^T{a}_{LSM}+x^T{A}_{LSM}x$$

(24)

$${\widehat{\sigma }}_{\mathrm{LSM}}\left(x\right)=34.8832+x^T{b}_{LSM}+x^T{B}_{LSM}x$$

(25)

where $x=\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]$, $a_{LSM}=\left[\begin{array}{c}177.0001\\ 109.4259\\ 131.4630\end{array}\right]$, $A_{LSM}=\left[\begin{array}{ccc}32.0001& 66.0278& 75.4721\\ 66.0278& -6.02780& 43.5833\\ 75.4721& 43.5833& -3.58330\end{array}\right]$, $b_{LSM}=\left[\begin{array}{c}11.5268\\ 15.3231\\ 29.1904\end{array}\right]$, and $B_{LSM}=\left[\begin{array}{ccc}4.2038& 7.7195& 5.1093\\ 7.7195& -0.71954& 14.082\\ 5.1093& 14.082& 16.778\end{array}\right]$.

The information used for the RD modeling methods after training with the associated transfer functions, training functions, NN architectures (number of inputs, number of hidden neurons, number of outputs), and number of epochs in the case study are summarized in

Table 10. Summary of the multilayer FFNN used in the case study.

Model		Transfer Function	Training Function	Architecture	#Epoch
FFNN	Mean	Tansig-Purelin	Traingdx	3-9-1	105
	Standard deviation	Tansig-Purelin	Traingda	3-10-1	19
CFNN	Mean	Tansig-Purelin	Traingda	3-12-1	107
	Standard deviation	Tansig-Purelin	Trainrp	3-11-1	9

and

Table 11. Summary of the RBFN used in the case study.

Model		Transfer Function	Goal	Spread
RBFN	Mean	Radbas-Purelin	1 × 10−27	0.5
	Standard deviation	Radbas-Purelin	1 × 10−27	0.8

. The specified weight and bias values of the proposed FFNN, DFNN, and RBFN used to approximate the process mean and standard deviation functions in the case study are shown in Appendix C (

Table A9. Estimated parameter values (Weight and bias) of the FFNN for mean function in the case study.

Weights				Biases
${\mathbf{W}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$			${\left({\mathbf{t}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{a}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{b}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
1.5660	1.9450	1.1790	0.3610	−3.0880	−0.1520
1.7450	−1.3140	2.0400	0.4270	−2.0620
−1.6810	2.2590	0.6690	0.2180	1.4730
1.5770	1.7570	−1.5510	0.1660	−0.9160
1.1460	−0.0730	2.6140	0.0510	0.1540
−1.7480	1.4660	1.7830	−0.0630	−0.4130
−1.3760	−2.3660	0.9550	−0.1300	−1.4310
0.5280	−0.8930	2.4800	0.2700	2.4590
1.9980	1.7550	1.2700	−0.0220	2.8680

,

Table A10. Estimated parameter values (Weight and bias) of the FFNN for standard deviation function in the case study.

Weights				Biases
${\mathbf{W}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$			${\left({\mathbf{t}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{a}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{b}}_{\mathit{F}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
1.9520	−2.0810	0.9550	−0.2470	−3.0060	−0.3080
1.4520	1.6590	−1.9370	−0.5730	−2.5620
−1.6750	1.9270	1.5750	−0.0660	1.6940
1.9030	−0.2630	2.2310	0.1620	−1.0300
0.9430	2.3320	1.6200	0.1020	−0.3810
−2.0160	−1.7170	1.3500	−0.2570	−0.1860
−1.9130	−0.4820	2.2770	0.0650	−0.9540
0.2000	2.8330	−0.8230	0.1420	1.7220
2.7580	1.5070	0.6520	0.3000	1.9940
1.497	1.726	−1.141	−0.938	3.480

,

Table A11. Estimated parameter values (Weight and bias) of the CFNN for mean function in the case study.

Weights					Biases
${\mathbf{R}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$			${\left({\mathbf{z}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\left({\mathbf{q}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{c}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{d}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
1.7480	2.4100	1.0820	0.5820 0.1600 0.6080	−0.0060	−3.2030	0.1530
2.6590	−0.1570	1.7430		−0.1080	−2.6440
−2.1980	1.7500	1.4620		−0.2100	2.0920
2.3100	−1.9110	−0.7440		0.0770	−1.6370
1.9620	−0.9270	2.3510		−0.0110	−0.8430
−1.9680	1.8610	−1.6700		−0.0560	0.2480
−1.5170	2.3360	1.6270		0.4240	−0.2960
0.2480	2.1750	−2.3370		0.0730	0.8240
2.7660	0.9810	−1.3500		−0.0010	1.3920
1.8210	−1.8120	−1.8440		0.2630	2.0910
−1.6870	1.7400	−2.0190		0.2820	−2.6670
2.1030	1.8880	1.2460		−0.5660	3.3270

,

Table A12. Estimated parameter values (Weight and bias) of the CFNN for standard deviation function in the case study.

Weights					Biases
${\mathbf{R}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$			${\left({\mathbf{z}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\left({\mathbf{q}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{c}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{d}}_{\mathit{C}\mathit{F}\mathit{N}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
2.0800	2.0450	1.7150	−0.4480 −0.6100 0.7170	0.8060	−2.7520	−0.0920
1.3360	2.1850	1.1350		0.0190	−3.1570
−2.1980	−0.2670	1.5240		0.0550	2.4750
1.6410	1.7640	2.2460		−0.5140	−1.7790
0.9380	−2.5720	1.7050		0.1360	−0.7120
−3.1130	−0.7290	0.9550		−0.4490	0.6660
0.8410	2.7640	1.8200		0.4000	−1.2890
0.9370	2.2670	−1.8640		0.8030	0.9310
1.7920	2.1840	1.0390		−0.0180	1.7790
2.2030	0.7220	−2.3830		0.0520	2.2580
−1.8960	−2.6360	−0.6560		0.0710	−2.9400

,

Table A13. Estimated parameter values (Weight and bias) of the RBFN for mean function in the case study.

Weights				Biases
${\mathbf{V}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$			${\left({\mathbf{u}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}\right)}^{\mathit{T}}$	${\mathbf{e}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathbf{f}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$
1.000	1.000	1.000	804.3300	1.6650	124.7800
1.000	1.000	0.000	513.9900	1.6650
1.000	0.000	1.000	442.760	1.6650
0.000	0.000	1.000	288.7100	1.6650
1.000	0.000	0.000	274.6200	1.6650
0.000	1.000	0.000	219.6200	1.6650
1.000	−1.000	1.000	250.5800	1.6650
0.000	1.000	1.000	287.6200	1.6650
1.000	0.000	−1.000	185.9900	1.6650
1.000	−1.000	0.000	195.1100	1.6650
0.000	0.000	0.000	193.3600	1.6650
−1.000	1.000	0.000	121.6500	1.6650
0.000	1.000	−1.000	110.810	1.6650
−1.000	−1.000	1.000	89.9600	1.665
1.000	1.000	−1.000	94.2940	1.6650
1.000	−1.000	−1.000	64.2240	1.6650
0.000	−1.000	1.000	75.5390	1.6650
−1.000	0.000	1.000	45.0000	1.6650
−1.000	0.000	0.000	27.0270	1.6650
−1.000	1.000	1.000	21.3410	1.6650
0.000	−1.000	−1.000	0.0000	1.6650
−1.000	1.000	−1.000	−25.8380	1.6650
0.000	0.000	−1.000	−18.8420	1.6650
−1.000	0.000	−1.000	−33.2240	1.6650
−1.000	−1.000	0.000	−42.9520	1.6650
0.000	−1.000	0.000	−53.0300	1.6650
−1.000	−1.000	−1.000	−95.8940	1.6650

and

Table A14. Estimated parameter values (Weight and bias) of the RBFN for standard deviation function in the case study.

Weights				Biases
${\mathbf{V}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$			${\left({\mathbf{u}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}\right)}^{\mathit{T}}$	${\mathbf{e}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$	${\mathbf{f}}_{\mathit{R}\mathit{B}\mathit{F}\mathit{N}}^{\mathit{s}\mathit{t}\mathit{d}}$
1.000	0.000	1.000	158.1600	2.7750	−0.0810
1.000	1.000	1.000	142.3900	2.7750
0.000	1.000	1.000	138.8700	2.7750
−1.000	−1.000	1.000	133.8800	2.7750
1.000	0.000	0.000	92.4780	2.7750
0.000	1.000	0.000	88.5840	2.7750
0.000	0.000	−1.000	80.4730	2.7750
−1.000	1.000	0.000	63.4950	2.7750
−1.000	1.000	1.000	55.4860	2.7750
0.000	0.000	1.000	44.5590	2.7750
1.000	−1.000	−1.000	42.8840	2.7750
1.000	−1.000	0.000	32.9120	2.7750
−1.000	0.000	1.000	29.4130	2.7750
−1.000	1.000	−1.000	27.6230	2.7750
1.000	1.000	−1.000	23.6910	2.7750
0.000	−1.000	1.000	23.4430	2.7750
1.000	1.000	0.000	21.0030	2.7750
1.000	−1.000	1.000	18.5040	2.7750
0.000	−1.000	0.000	17.7250	2.7750
1.000	0.000	−1.000	16.1390	2.7750
−1.000	0.000	0.000	15.0480	2.7750
−1.000	−1.000	−1.000	12.5660	2.7750
0.000	−1.000	−1.000	8.3970	2.7750
0.000	1.000	−1.000	4.6000	2.7750
−1.000	0.000	−1.000	3.4830	2.7750
000	0.000	0.000	−0.0720	2.7750

).

The contour plots of the response functions for the process mean and standard deviation estimated by the LSM-based RSM, FFNN, CFNN, and RBFN robust design modeling methods are demonstrated in Figure 17, Figure 18, Figure 19 and Figure 20, respectively.

Table 12. Comparative results of the case study.

Estimation Model	Optimal Factor Settings			Process Mean	Process Bias	Process Variance	EQL
	${\mathit{x}}_1$	${\mathit{x}}_2$	${\mathit{x}}_3$
LSM	1.000	0.071	−0.2500	494.6720	5.3270	1977.5320	2005.9170
FFNN	0.999	0.999	−0.6230	499.8690	0.1310	7.7630	7.7800
CFNN	0.792	−0.752	0.9990	499.6870	0.3120	2.8540	2.9520
RBFN	0.999	0.999	−0.4300	500.0520	0.0520	23.2190	23.2210

presents the optimal factor settings, associated process mean, process bias, process variance, and EQL values obtained from the proposed NN-based modeling methods and LSM-based RSM in the case study. The proposed RD modeling methods produced significantly smaller EQL values than LSM-based RSM.

The process variance obtained from the proposed NN-based estimation methods is markedly lower than that of conventional RSM. The scatter plots demonstrate the value of squared process bias vs. variance for the traditional LSM-based RSM model and the three proposed NN-based modeling methods as illustrated in Figure 21. The optimal settings are marked with green stars in each figure.

5. Conclusions and Further Studies

This paper identified three NN-based modeling approaches (FFNN, CFNN, and RBFN) that obviate the need for the assumptions required when LSM-based RSM is used to approximate the mean and standard deviation functions. The feed-forward NN structure-based RD modeling methods are alternative options for identifying a functional relationship between input factors and process mean and standard deviation in RD. Compared with the conventional RD modeling method, the proposed approach has significant advantages with regard to accuracy and efficiency. The proposed RD modeling methods can easily be implemented using existing software such as MATLAB. The results of both the two types of simulation studies and case study show that the proposed RD modeling methods provide better optimal solutions than conventional LSM-based RSM. The main results are summarized, and the variability index, EQL, and R² are central to the model validation criteria. (i) In simulation study 1, the proposed RD modeling methods showed, on average, 139 times lower process bias and 1.5 times lower process variance than LSM-based RSM in terms of variability, and about four times smaller in EQL. Moreover, among the three NN-Based modelings, FFNN showed the lowest value in terms of EQL. (ii) In simulation study 2, the proposed RD modeling methods showed 11 times lower process bias and one time lower process variance than LSM-based RSM on average in terms of variability and about 1.2 times smaller in EQL. Among the three NN-Based modelings, FFNN also showed the lowest value. (iii) In the case study, the proposed RD modeling methods showed significantly lower results than LSM-based RSM, with an average process bias of 102 times and process variance of 85 times in terms of variability, and about 86 times smaller in EQL. Among the three NN-Based modelings, CFNN and FFNN showed much lower values than RBFN. (iv) Specifically, in the printing machine case study, the R² values for the estimated standard deviation functions are 45.452%, 73.51%, 69.47%, and 99.99%, when the conventional LSM-based RSM, FFNN-based modeling, CFNN-based modeling and RBFN-based modeling methods are applied respectively.

In future work, the proposed NN structures-based RD methods could be used to estimate multiple responses (RD multi-objective optimization problem), time-series data, big data, and simulation data [60]. But the activation and transfer function in the hidden and output layers should be carefully investigated and selected separately when applied. In addition, in this study, a classical case study was performed to verify the proposed methodology, but in the future study, a field-based case study that suggests optimal process conditions for productivity improvement in a smart factory will be performed.