This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (

An innovative classification and back-propagation-network tree (CABPN tree) approach is proposed in this study to estimate the cycle time of a job in a wafer fabrication factory, which is one of the most important tasks in controlling the wafer fabrication factory. The CABPN tree approach is an extension from the traditional classification and regression tree (CART) approach. In CART, the cycle times of jobs of the same branch are estimated with the same value, which is far from accurate. To tackle this problem, the CABPN tree approach replaces the constant estimate with variant estimates. To this end, the cycle times of jobs of the same branch are estimated with a BPN, and may be different. In this way, the estimation accuracy can be improved. In addition, to determine the optimal location of the splitting point on a node, the symmetric partition with incremental re-learning (SP-IR) algorithm is proposed and illustrated with an example. The applicability of the CABPN tree approach is shown with a real case. The experimental results supported its effectiveness over several existing methods.

The cycle time (flow time, manufacturing lead time) of a job is the time required for the job to go through the factory. Therefore, it is subject to capacity constraints, the factory congestion level, the quality of job scheduling, and many other factors. As a result, the cycle time of a job is highly uncertain. Cycle time management activities include cycle time estimation, internal due date assignment, job sequencing and scheduling, and cycle time reduction (see

In the literature, various types of methods have been proposed to estimate the cycle time of a job in a factory. For example, probability-based statistical methods, such as queuing theory and regression, have been proposed. In [

The application of artificial neural networks (ANNs) is also a mainstream in this field. For example, a self-organization map (SOM) was developed in Chen [

An innovative classification and back-propagation-network tree (CABPN tree) approach is proposed in this study to estimate the cycle time of a job in a wafer fabrication factory. The CABPN tree approach is an extension from the traditional classification and regression tree (CART) approach. The significance of doing so is fourfold:

In CART, the cycle times of jobs of a branch are estimated with the same value, which is not accurate. In the proposed CABPN tree approach, a BPN is constructed for each branch to estimate the cycle times of jobs, which is expected to enhance the estimation accuracy;

Although clusterwise models such as SUPPORT and treed Gaussian process models have been used in various fields, they have not been applied to estimating the cycle time of a job in a manufacturing system;

Compared with the existing methods, the proposed CABPN tree approach classifies jobs based on fewer job attributes. On one hand, it is easier to implement. On the other hand, it is possible to assign more jobs to a branch;

The existing classifiers in this field, such as kM, FCM, and SOM, classify jobs based on their attributes rather than their compatibilities with the estimation mechanism. However, the compatibility with the estimation mechanism is important, and may be more influential to the estimation performance. In this regard, CART considers the estimation performance in classifying jobs, which makes it more suitable for the same purpose.

In addition, to determine the optimal location of the splitting point on a node, the symmetric partition with incremental re-learning (SP-IR) algorithm is proposed. The proposed CABPN tree approach is a hybrid approach that fulfills both the classification and regression tasks.

The proposed CABPN tree approach is introduced in Section 2. In addition, the SP-IR algorithm is illustrated with a small example. Subsequently, in Section 3, a real case containing the data from a wafer fabrication factory was used to evaluate the effectiveness of the proposed methodology. Several existing methods in this field have been applied to the same case to make a comparison. The advantages and/or disadvantages of each method were discussed. Finally, Section 4 concludes this paper and puts forward some directions that can be explored in future studies.

There are five steps in the proposed methodology: data normalization, cycle time estimation using the BPN approach, CABPN tree growing, stopping, and pruning, as illustrated in

In the traditional CART approach, the cycle times of jobs assigned to a branch (_{(}_{b}_{)} that is equal to the average of the historical cycle times:
_{j}_{b}_{1} > Δ_{1} and _{3} > Δ_{3} are classified into the same category. The cycle times of these jobs are estimated with _{1}. In the proposed methodology, jobs satisfying _{2} > Δ_{2} and _{5} > Δ_{5} are classified into the same category. A BPN is then constructed to estimate the cycle times of these jobs. Obviously, the conditions established by the proposed methodology are different from those established by CART. The differences between CART and the proposed CABPN tree are summarized in

First, the collected data are normalized into [0.1, 0.9]:

The normalized value can be converted back to

Conversely, the ranges of the inputs and output of the BPN are set to [0, 1]. In this way, the future value may be greater than the historical maximum or less than the historical minimum [

Assume max _{f}

The historical values of _{f}_{f}_{f}

It is also acceptable to normalize an input into a narrower range, such as [0.2, 0.8] or [0.3, 0.7], to allow for a larger or smaller future value. However, the fluctuation in the input will be further smoothened in this way; that is not beneficial to fit an exact relationship between the input and the output.

Subsequently, a BPN is constructed to estimate the cycle times of jobs. The BPN is configured as follows. There are

Inputs to the BPN are multiplied by the weights of the connections between the input and hidden layers, then are summed on each neuron in the hidden layer. After being compared with the threshold on the neuron, only significant signals will be transformed and outputted as:
_{jl}

Signals outputted from the hidden-layer neurons are transmitted to the neuron in the output layer in the same manner. Finally, the output from the BPN is generated as:
^{o}_{j}

Subsequently, a lot of algorithms can be applied to train a BPN, such as the gradient descent (GD) algorithm, the conjugate gradient algorithm, the Levenberg-Marquardt (LM) algorithm, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, and others. For a recent comparison of these algorithms, refer to [

First, placing the inputs and the BPN parameters in vectors
_{j}

Specify the initial values of

Replace

Approximating the right-hand side by its linearization gives

The optimal value of

Return to Step (2).

Artificial neural networks such as BPN, radial basis function (RBF) network, and support vector regression (SVR) are effective methods to carry out nonlinear approximation. However, the dynamic environment in a wafer fabrication factory is quite complex. A number of studies (e.g., [

This step is to grow the CABPN tree using a recursive partitioning technique that selects variables and split points according to a pre-specified criterion. In the traditional CART approach, such criteria include Gini, towing, ordered towing, and maximum deviance reduction [

In the beginning, all jobs are assigned to the current node. Assume jobs of the current node are put in a set _{(}_{p}_{)} | ^{k}_{1}(_{1}_{2}(_{2}

To help solve this problem, the symmetric partition with incremental re-learning (SP-IR) algorithm is proposed as follows:

^{k}

_{1} by adding the data of the new job;

_{2} by adding the data of the new job;

The retraining process uses the parametric values at the end of the previous training as the initial setting, and train the BPN with fewer epochs. In this way, it is likely that the new optimal solution will be close to that in the previous training.

An example is provided in ^{1} = {_{1}, _{2}, _{7}, _{5}, _{6}, _{4}, _{3}, _{10}, _{8}, _{9}}. Set _{1} and BPN_{2}, were constructed to estimate the cycle times of jobs in the two subsets, respectively. The impurity measure Imp(^{−4}. The improvement in Imp(_{1}, _{2}, _{7}, _{5}} (_{j}_{1} ≤ 0.304) and {_{6}, _{4}, _{3}, _{10}, _{8}, _{9}} (_{j}_{1} > 0.304). The minimum impurity was 1.21 × 10^{−4}. Subsequently, the data were partitioned along the second attribute in the same way, and the results were summarized in _{5}, _{10}, _{1}, _{3}} (_{j}_{2} ≤ 0.440) and {_{6}, _{9}, _{2}, _{7}, _{8}, _{4}} (_{j}_{2} > 0.440). The minimum impurity was 6.04 × 10^{−6}. After comparing the two results, the better way for the current node is to partition the jobs along the second attribute, because the impurity can be reduced to a smaller value (see

The growing of a CABPN tree stops if any of the following conditions has been satisfied:

The improvement in the performance measure,

A certain number of nodes have been generated;

The depth of the tree has reached a certain level.

A large CABPN tree may overfit, while a small CABPN tree may not reflect the inherent structure of the data. Cost-complexity pruning is usually used to tackle this issue. The cost-complexity of a CABPN tree

The results of cost-complexity pruning are a nested subset of CABPN trees starting from the largest tree and ending with the smallest tree (

A real case containing the data of 200 jobs from a dynamic random access memory (DRAM) manufacturing factory was used to evaluate the effectiveness of the proposed methodology (see ^{2}) is used to evaluate the effectiveness of deleting a variable. Finally, four attributes are chosen: _{j}_{1} (the job size), _{j}_{2} (the queue length before the bottleneck), _{j}_{3} (the average waiting time of jobs already finished), and _{j}_{4} (factory utilization). Jobs in the collected data were of the same product type and priority. The average and standard deviation of the cycle times were 1225 and 206 h. Obviously, there was a considerable fluctuation in the cycle times. The data of the first 150 jobs were used as the training examples, while the remaining data were reserved for testing.

The collected data were normalized into [0.1, 0.9], as shown in

A single hidden layer with eight nodes;

Number of epochs per replication: 1000;

Training algorithm: the LM algorithm;

Stop training if the mean squared error (MSE) < 10^{−4} or 1000 epochs have been run;

Early stopping: After each epoch of training, the BPN is applied to the testing data. If the performance, measured in terms of MSE, begins to deteriorate, the training process stops to avoid overfitting.

The CABPN tree was grown using the proposed SP-IR algorithm. The criteria for stopping the CABPN tree growing include:

The improvement in the SSE was less than 5%;

More than 10 nodes have been generated;

The depth of the CABPN tree has reached the third level.

These stopping criteria are also beneficial to avoid overfitting.

Then, the CABPN tree was pruned by minimizing the cost-complexity measure, for which ^{2}.

For a comparison, the traditional CART method was also applied to this case, for which MATLAB 2006a was used. The criterion for choosing a split was Gini. In addition, impure nodes must have 10 or more jobs to be split. Subsequently, the CART tree was pruned based on an optimal pruning scheme that first prunes branches contributing to the least improvement in SSE. Finally, the results were shown in ^{2}, much greater than that obtained using the CABPN tree approach. However, such a difference was for the training data.

The two approaches were applied to the testing data. The estimation results using the proposed methodology were shown in

The results are summarized in

Obviously, the estimation accuracy, measured in terms of MAE, MAPE, or RMSE, using the proposed CABPN tree approach was much better than that by the traditional CART approach. The advantage was the most obvious (up to 23%) in the RMSE regard.

On the other hand, the numbers of jobs of nodes in the CABPN tree were summarized in

Starting with different initial conditions, a BPN will converge to different results, so does the CABPN tree approach, which means it is likely to further improve the estimation accuracy of the CABPN tree approach in an iterative way or with carefully chosen initial conditions.

However, the training time required for the CABPN tree approach was 297 s, much longer than that required for the CART approach (less than 5 s). Obviously, the efficiency of the proposed methodology needs to be improved somehow.

A sensitivity analysis was performed to assess the effects of

The CABPN tree approach is computationally more intensive than its counterpart CART. The computation time required for the CABPN tree approach depends on the number of jobs, the size of each BPN (including the number of hidden layers and the number of nodes in each hidden layer), and the stopping criteria of the CABPN tree. It is almost instantaneous to find a feasible solution by changing the weights of the BPN, but it takes time to evolve the solution to the optimal one. In this case, the CABPN tree approach was implemented on a PC with an Intel Dual CPU E2200 2.2 GHz and 2.0G RAM. It took about 3 min to estimate the cycle times of jobs. In contrast, the CART approach took only 15 s. Obviously, there is considerable room for improving the efficiency of the CABPN tree approach.

To further elaborate the effectiveness of the proposed methodology, another existing method—the hybrid kM and BPN approach [

An innovative combination of CART and BPN is formed in this study to estimate the cycle time of a job in a wafer fabrication factory, which is always considered as one of the most important tasks in controlling a wafer fabrication factory. After embedding an individual BPN into each node of a classification tree, the CABPN tree approach is proposed, so that the cycle times of jobs of the same node will be estimated with the same BPN, instead of a single value as in the traditional CART approach. Such a treatment is expected to improve the estimation accuracy. However, to determine the optimal location of the splitting point on a node is a challenging task. To tackle this issue, the SP-IR algorithm is proposed to generate quite symmetric partitioning results. In this way, most BPNs can be trained with enough examples.

The effectiveness of the proposed CABPN tree approach was evaluated with a real case from a DRAM manufacturing factory. According to the experimental results,

The estimation accuracy, measured in terms of MAE, MAPE, or RMSE, was significantly improved after applying the CABPN tree approach;

On the other hand, there is considerable room for improving the efficiency of the CABPN tree approach, owing to the extensive BPN learning;

The estimation accuracy of the CABPN tree approach can be further improved in an iterative way or with carefully chosen initial conditions.

The symmetric property of the SP-IR algorithm aims to reserve enough examples for each node, however, may cause the CABPN tree to be trapped in the local optimum. This issue can be investigated in a future study.

This work was supported by National Science Council of Taiwan.

The author declares no conflict of interest.

Cycle time management activities.

The five steps of the CABPN tree approach.

Comparison of (

The results of moving the splitting point along the first attribute.

The results of moving the splitting point along the second attribute.

The splitting result of the current node.

The built CABPN tree.

The CART tree.

The estimation results of the testing data.

The symmetric property of the SP-IR algorithm.

The effects of

The effects of the maximum number of nodes (the stopping criterion).

A comparison of the classification and back-propagation-network (CABPN) tree approach and some existing methods.

BPN [ |
– | BPN | Easy | Moderate |

SOM-BPN [ |
SOM | BPN | Moderate | High |

Pearn |
– | Gamma Distribution Fitting | Easy | Low |

SOM-FBPN [ |
SOM | FBPN | Difficult | High |

kM-FBPN [ |
kM | FBPN | Difficult | High |

PCA-CART [ |
PCA-CART | CART | Easy | Moderate |

CABPN tree | SP-IR | BPN | Easy | High |

The differences between CART and the proposed methodology.

CART | the same | Gini, |
Linear | LR |

CABPN tree | Different | the SP-IR algorithm | Nonlinear | BPN |

An illustrative example.

_{j}_{1} |
_{j}_{2} |
_{j} | |
---|---|---|---|

1 | 0.100 | 0.391 | 0.100 |

2 | 0.112 | 0.677 | 0.176 |

3 | 0.575 | 0.440 | 0.582 |

4 | 0.546 | 0.900 | 0.546 |

5 | 0.304 | 0.100 | 0.288 |

6 | 0.305 | 0.466 | 0.436 |

7 | 0.232 | 0.716 | 0.267 |

8 | 0.861 | 0.895 | 0.795 |

9 | 0.900 | 0.672 | 0.900 |

10 | 0.720 | 0.286 | 0.511 |

The collected data.

_{j1} |
_{j2} |
_{j3} |
_{j4} |
_{j} | |
---|---|---|---|---|---|

1 | 24 | 175 | 441 | 97% | 863 |

2 | 23 | 154 | 256 | 97% | 1342 |

3 | 23 | 171 | 45 | 87% | 956 |

4 | 23 | 183 | 466 | 85% | 1391 |

5 | 23 | 163 | 90 | 100% | 1153 |

6 | 24 | 181 | 332 | 89% | 1321 |

… | |||||

198 | 23 | 181 | 68 | 98% | 1099 |

199 | 22 | 168 | 206 | 89% | 1314 |

200 | 23 | 164 | 261 | 91% | 1575 |

The normalized data.

_{j1} |
_{j2} |
_{j3} |
_{j4} |
_{j} | |
---|---|---|---|---|---|

1 | 0.670 | 0.609 | 0.853 | 0.786 | 0.105 |

2 | 0.448 | 0.128 | 0.500 | 0.795 | 0.501 |

3 | 0.431 | 0.504 | 0.100 | 0.449 | 0.181 |

4 | 0.475 | 0.788 | 0.900 | 0.386 | 0.541 |

5 | 0.424 | 0.324 | 0.184 | 0.897 | 0.345 |

6 | 0.562 | 0.749 | 0.645 | 0.525 | 0.483 |

… | |||||

198 | 0.545 | 0.737 | 0.144 | 0.817 | 0.300 |

199 | 0.348 | 0.450 | 0.406 | 0.536 | 0.477 |

200 | 0.437 | 0.346 | 0.511 | 0.584 | 0.692 |

Comparison of the performances of the two approaches.

CART | 97 | 8.0% | 126 |

CABPN Tree | 75 | 5.7% | 97 |

The number of jobs of each node in the CABPN tree.

No. of jobs | 40 | 17 | 21 | 20 | 14 | 21 | 17 |

The number of jobs of each node in the CART tree.

No. of Jobs | 27 | 16 | 37 | 18 | 10 | 29 | 2 | 5 | 6 |

Comparison with the hybrid k-means and back propagation networks (BPN) approach.

kM-BPN | 82 | 6.3% | 101 |

The proposed methodology | 75 | 5.7% | 97 |