Parallel Simplex, an Alternative to Classical Experimentation: A Case Study

Abstract

Experimentation is a strong methodology that improves and optimizes processes. Nevertheless, in many cases, real-life dynamics of production demands and other restrictions inhibit the use of these methodologies because their use implies stopping production, generating scrap, jeopardizing demand accomplishments, and other problems. Proposed here is an alternative methodology to search for the best process variable levels and optimize the response of the process without the need to stop production. This algorithm is based on the principles of the Variable Simplex developed by Nelder and Mead and the continuous iterative process of EVOPS developed by Box, which is then modified as a simplex by Spendley. It is named parallel simplex because it searches for the best response with three independent Simplexes searching for the same response at the same time. The algorithm was designed for three simplexes of two input variables each. The case study documented shows that it is efficient and effective.

1. Summary

Traditional experimentation requires designing an experimental array, stopping production, setting up the equipment/machinery for each run, and running all the runs of the array. This process inevitably results in lost production time, scrap, and the use of extraordinary resources. In many real manufacturing scenarios, this is difficult to do due to the complexity of production demand trade-offs and manufacturing process dynamics. The main idea of Box’s Evolutive Operations [1], Spendley and Himsworth’s Simplex [2], and Nelder and Mead’s Variable Simplex [3] is to find the best combination of process parameters by introducing small changes in the process variables. These small changes allow the experimenter to run the process continuously because the response variable does not move dramatically away from its target, thereby eliminating or minimizing the generation of nonconforming parts (scrap).

These direct search methods (EVOPS, Simplex, Variable Simplex, etc.) have the following disadvantages that have limited their use in a manufacturing process:

• They are very sensitive to noise (internal noise).
• The complexity of the computation increases dramatically with the number of variables considered.
• The number of iterations to find a local optimum grows exponentially with the number of variables in the algorithm.

These drawbacks make these methods not very practical to apply to a manufacturing process.

On the other hand, these procedures are based on heuristic/stochastic algorithms, i.e., a series of iterations is developed by introducing small changes in the input variables and evaluating the observed response. The process continues until the response is located on the desired nominal value. We must consider that a deterministic algorithm is defined as one that exhibits predictable behavior and a fixed sequence of steps. In other words, a deterministic algorithm always produces the same result for a given set of inputs, and its execution is not subject to variability or uncertainty. Furthermore, the process that the algorithm follows is not dependent on random factors or any probability, and it follows the same path each time it is executed, producing the same result. In regard to the heuristic and stochastic algorithms, it is important to note that their behavior is not predictable. The results produced by these algorithms may vary depending on the inputs provided, due to the inherent random or probabilistic nature of the algorithms. The algorithms do not adhere to a fixed sequence of steps in each run, thereby introducing variability.

• The utilization of randomness or approximation: Stochastic algorithms frequently employ a random component (for example, in the decisions made during their execution) or an approximation based on heuristics to identify solutions more expediently in complex problems.
• Objective: In numerous instances, these algorithms do not pursue an exact solution, but a “good” solution within a reasonable time, particularly when the search space is vast or the problem is challenging to solve exactly.

Deterministic algorithms, in contrast to heuristic or stochastic algorithms, rely on known or well-defined functions to guide their execution. These algorithms follow a set series of steps that are determined solely by the inputs they receive, without any random factor affecting their outcome. They are based on specific functions that operate on the inputs and dictate the algorithm’s steps, consistently producing the same results for the same inputs. On the other hand, when an algorithm lacks a clearly defined function or deals with ambiguous or uncertain problems, it may be heuristic or stochastic in nature. These algorithms make decisions that are not strictly predictable and may involve approximation or randomness. Examples of such algorithms are genetic algorithms, simulated annealing, and random search, which are commonly used in complex optimization or search problems with large solution spaces.

The Nelder–Mead algorithm is a heuristic and non-deterministic algorithm. Although it follows a clearly defined set of steps to identify the minimum value of a function without utilizing derivatives, the manner in which it explores the solution space incorporates elements of randomness, particularly during the reflection, expansion, and contraction phases.

The algorithm begins with a set of random initial points. The algorithm commences with an initial set of points that may be selected at random or otherwise, although the latter is more common.

Space exploration: In the reflection, expansion, and contraction stages, the decisions regarding the movement of the points of the ‘simplex’ (the set of points representing the current solutions) may be contingent upon minor variations in the configuration of the simplex, which introduces variability.

Although the process is defined by a sequence of steps, the behavior of the algorithm may vary slightly between runs due to the initial choice and the nature of the moves in the search space. Accordingly, it is not deemed a deterministic algorithm.

In conclusion, the Nelder–Mead algorithm is not deterministic due to the involvement of randomness in its decision-making processes, such as the selection of initial points. The method is heuristic in nature, seeking an approximate solution without the necessity of gradients or derivatives [3,4,5].

Logic algorithms are widely used in the engineering field, from genetic algorithms applied to analyze and improve the location, assignment, and routing of transportation systems [6], to many other applications; these authors documented another few heuristic and stochastic methods and their objectives. It can be seen that there are many designs of algorithms. Mainly, they can be classified as follows:

• Deterministic: Those that work with a known function.
• Heuristic: Algorithms based on test and error procedures.
• Stochastic: They use successive iterations, taking into account changes in the input variables and measuring the change observed in the output variables, with the help of probabilistic theories.

One of the algorithms that seem to work like the Nelder and Mead algorithm presented in this paper is the Genetic Algorithm, widely used to analyze and improve many different systems. These algorithms evolve a population of individuals by subjecting it to random actions, similar to those that act in biological evolution (mutations and genetic recombination), as well as natural selection. According to some criterion, it is decided which are the most adapted individuals, which survive, and which are the least adapted, which are discarded. An algorithm is a set of organized steps that describe the process to be followed to solve a particular problem, such as an experimental design for optimizing process parameters.

It should be noted that the primary distinction between the genetic algorithm and the Nelder–Mead algorithm lies in their respective deterministic and stochastic characteristics. As previously stated, the former is deterministic, while the latter is heuristic/stochastic. Despite this difference, both algorithms exhibit similarities in several aspects.

The optimization objective is as follows: Both algorithms are designed to solve optimization problems, that is, they seek to identify a solution (or set of solutions) that minimizes or maximizes an objective function. The objective of both methods is to identify the optimal solution within a given search space, whether that be the global minimum or a satisfactory approximation.

Solution Space Exploration: Both algorithms endeavor to explore the search space in pursuit of optimal or approximate solutions. Despite employing disparate methodologies—one via an evolutionary process and the other through the manipulation of a set of points—both approaches are predicated on the notion of directing the “search” in the space towards more favorable regions.

The adaptation or modification of solutions is a crucial aspect of the process.

A genetic algorithm is a computational method that employs the principles of genetics to search for optimal solutions. It employs a process of mutation, crossover, and selection to generate novel solutions from existing ones.

The Nelder–Mead method is a metaheuristic optimization technique that employs a simplex method to iteratively improve the quality of a solution. The process entails reflection, expansion, contraction, and reduction of a set of points (simplex) within the solution space, with the objective of moving the set towards the minimum.

Solution population:

While Nelder–Mead does not operate with “populations” of solutions in the same manner as a genetic algorithm, it nevertheless possesses a comparable collection of points (the simplex) that evolve during the optimization process. In this sense, it can be posited that, at each iteration, the algorithm is handling a set of solutions.

Trial and error methods: Both algorithms employ an approach based on the evaluation of the objective function and adjust the solutions based on the results obtained. They do not require derivatives of the objective function, which makes them useful in problems where the function is non-differentiable, noisy, or has discontinuities, or in the case of Nelder and Mead, the objective function is unknown. There is extensive literature on these algorithm applications [7,8,9,10].

An exhaustive review of the literature shows evidence of the loss of efficiency of these direct search algorithms when the number of dimensions (variables) is increased. As an example, ref. [11] compares different algorithms (heuristic and deterministic) and their efficiency; our attention is focused on the number of runs required to obtain a local or global optimum. As shown in

Table 1. Comparison of Different Algorithms for Differentiable Functions [11].

Function			VNS+NM			VNS+RNM			RMNME			EMNM
Name	n	Fopt	Feval	Succ.	Fmin	Feval	Succ.	Fmin	Feval	Succ.	Fmin	Feval	Succ.	Fmin
S1. Branin	2	0.3979	72	10		75	10		72	10		72	10
S2. Gldstein and Price	2	3.0000	150	10		211	10		109	10		109	10
S3. Hartman H34	3	−3.8626	245	10		367	10		702	10		123	4	−2.1453
S4. Hartman H64	6	−3.3224	1019	10		1375	10		1547	8	−3.2744	590	5	−2.9421
S5. Shekel 4, 10	4	−10.5364	1367	10		3131	10		1793	6	−7.3679	222	2	−4.2124
S6. Shekel 4, 5	4	−10.1532	1035	10		1318	10		1487	5	−6.4069	270	5	−6.4069
S7. Shubert	2	−186.7306	1071	10		1218	10		2228	10		73	0	−23.3780
S8. Ackley	10	0	188,726	10		324,405	10		35,706	0	0.8977	880	0	12.4700
S9. Dixon and Price	10	0	5273	10		4752	10		5138	10		5138	10
S10. Griewank	6	0	148,528	10		161,096	10		17,285	0	0.2108	601	0	53.1800
S11. Rosenbrock	10	0	9738	10		7917	10		8482	8	0.7974	6750	8	0.7974
S12. Schwefel	10	0	1,173,568	10		1,262,060	10		16,426	0	1789.88	2535	0	1818.4500
S13. Zakharov	10	0	4056	10		4218	10		4508	10		4508	10
S14. Rastrigin	10	0	130,802	10		222,881	10		32,184	0	7.5618	1147	0	88.7500
S15. Powell	12	0	6092	10		5977	10		6349	10		6237	10
Sum			1,671,742	150		2,002,001	150		134,016	97		29,255	74

, the number of runs (iterations of the algorithm) increases dramatically when the number of dimensions (variables) is increased (n).

As another example, ref. [12] compared the NMS algorithm with five typical problems.

Table 2. Efficiency of NMS of Five Typical Problems [12].

				Efficiency Parameters
Problem	Dimension	Step Size	Iterations	L	D	B	A
1	2	1	20	6.931	0.022	0.073	0.047
	10	4	250	9.66	0.325	0.195	0.042
	18	4	1250	11.340	0.797	0.096	0.023
			Average	9.31	0.381	0.121	0.037
2	2	1	20	7.149	0.013	0.105	0.070
	10	1	225	9.480	0.113	0.217	0.080
	18	4	600	10.534	0.510	0.210	0.077
			Average	9.054	0.212	0.177	0.076
3	2	1	30	7.630	0.049	0.046	0.038
	10	4	595	10.579	2.461	0.963	0.543
	18	4	2000	11.960	6.846	1.241	0.777
			Average	10.056	3.119	0.75	0.453
4	4	1	80	8.357	0.113	0.153	0.075
	8	1	240	9.519	0.306	0.254	0.093
	16	4	1520	11.663	0.547	0.432	0.122
			Average	9.846	0.322	0.280	0.097
5	2	1	20	7.172	0.028	0.084	0.060
	10	4	170	9.112	0.222	0.310	0.106
	18	4	1360	11.546	0.367	0.342	0.111
			Average	9.277	0.206	0.245	0.092

shows the comparison of results under different sample sizes and the number of variables for five typical problems; it can be seen that the number of iterations increases dramatically as the number of variables in the geometric array (dimensions) increases.

To give the reader a better idea, these algorithms are essentially different from those used to make decisions with multiple objectives, such as those used as multicriteria. To understand these, you can see [13,14,15] and others.

The algorithm we discussed here is used to find the best combination of parameters for a process, whatever it may be, from a design, a machine, or a manufacturing process. It has to be considered that the increase in the number of iterations for these algorithms is directly related to the number of variables analyzed. If two variables are considered, the geometric array corresponds to a triangle, if three variables are considered, the array corresponds to a square, if four variables are considered, the geometric array corresponds to a pentagon, and so on. As the number of variables increases, so does the number of edges and vertices, which slows down the algorithm and increases the complexity of the computation. In many cases, the high number of variables leads to a non-convergence situation (the algorithm gets “lost”).

The proposed methodology tries to overcome exactly this situation, that is, to search for the optimal response (local or global) when the number of input variables increases, trying to minimize the number of iterations without the algorithm “getting lost”, that is, it converges quickly. Again, these algorithms, by introducing small changes in the input variables, allow them to be executed during the normal production process. Every process/manufacturing engineer knows the value of this.

2. Methods

Box [1] proposed an iterative experimentation process, basically, his proposal consisted in the introduction of a two-level factorial design that is modified in search of the best response [1]. Small changes in the levels of the design are estimated to minimize the generation of nonconforming products. This way of managing the experimentation process is known as operative evolutions (EVOPs). EVOP is mainly used in the 3rd phase of experimentation (optimization) after the process has been characterized.

On the other hand [16], there are questions about the sample size. The Box algorithm does not consider whether the sample size is sufficient to generate acceptable statistical properties. It is clear that the proposal of Box has many advantages and that it is innovative because it searches for the best response with small changes during production, minimizing the nonconforming product.

Spendley [2] proposed a big change in the algorithm of Box, adding a simplex arrangement that replaces the factorial design [2]. It is through this simplex that the levels of each iteration are calculated. The algorithm of [2] is known as simplex EVOP. It iteratively moves the polyhedron (with n + 1 vertices), looking for the optimization of the response. Here is a brief review of the operations of the variable simplex of Nelder and Mead [3].

In the classical array, the levels of the variables considered are pre-designed, that is, once the array is defined, these levels remain fixed during the process of experimentation. In the Nelder and Mead Variable Simplex (NMS), the algorithm starts with pre-designed levels, then these levels are modified by the iterations of the algorithm according to a set of rules (operations of the simplex). The algorithm consists of using a simplex (a geometric arrangement), generally a polyhedron with n − 1 vertices (for two input variables, it will be a triangle). The search mechanism consists of using one of the vertices as a pivot (called the worse vertex) to estimate the next vertices (variable levels), always moving towards the best vertex (the one closest to the response objective: maximum, minimum, and target). The algorithm moves through the response, calculating new vertexes in each iteration, through four basic operations: Reflection, Expansion, Contraction, and Shrinkage. Figure 1 illustrates these operations.

There is plenty of literature available to understand the logistics of variable simplex operations. In this section, a brief explanation of the logistics of the algorithm of NMS is resumed.

2.1. Basic Operations of the Nelder and Mead Algorithm

For the first iteration, k = 0, the algorithm needs an initial simplex S₀ = {${\mathit{x}}_0^0,{\mathit{x}}_1^0,{\dots ,\mathit{x}}_{\mathit{n}}^0$} because it prefers to start with a regular simplex. This characteristic changes immediately, since the successive iterations replace the worst vertex, ${\mathit{x}}_{\mathit{h}}^{\mathit{k}}$, of the S_k-th simplex with another vertex. This worse vertex will be replaced by one of the new points (vertexes) ${\mathit{x}}_{\mathit{r}}^{\mathit{k}}$, ${\mathit{x}}_{\mathit{e}}^{\mathit{k}}$ and ${\mathit{x}}_{\mathit{c}}^{\mathit{k}}$. The selection of one of these three points, at any k ≥ 0 iteration, is defined by one of the three operations of the algorithm: Reflection, Expansion, and Contraction.

2.1.1. Reflection

Considering that ${\mathit{f}}_{\mathit{i}}>{\mathit{f}}_{\mathit{r}}>{\mathit{f}}_{\mathit{l}}$ for at least one i, where i ≠ h or i ≠ l; α is the coefficient of reflection. Figure 2 shows the reflection operation in two dimensions (two variables of the process) for the n = 2 case. The distance between the points ${\mathit{x}}_{\mathit{h}}^{\mathit{k}}$ and ${\mathit{x}}^{\mathit{k}}$ is equal to the distance between ${\overline{\mathit{x}}}^{\mathit{k}}$ and ${\mathit{x}}_{\mathit{r}}^{\mathit{k}}$. This is obtained when α = 1.

2.1.2. Expansion

The worse vertex ${\mathit{x}}_{\mathit{h}}^{\mathit{k}}$ is replaced by:

$${\mathit{x}}_{\mathit{e}}^{\mathit{k}}=\mathit{\gamma }{\mathit{x}}_{\mathit{r}}^{\mathit{k}}+(1-\mathit{\gamma }){\overline{\mathit{x}}}^{\mathit{k}}$$

(1)

Considering that (i) f_r < f_e, and (ii) f_e < f_l. If (i) is kept but (ii) is not, then the worse vertex ${\mathit{x}}_{\mathit{h}}^{\mathit{k}}$ is replaced by ${\mathit{x}}_{\mathit{r}}^{\mathit{k}}$. γ is the expansion coefficient. Figure 3 shows the expansion operation for two dimensions, n = 2. The distance between the points ${\mathit{x}}_{\mathit{h}}$ and $\overline{\mathit{x}}$ is equivalent to the distances between the points $\overline{\mathit{x}}$ and ${\mathit{x}}_{\mathit{r}}$, and ${\mathit{x}}_{\mathit{e}}$ and ${\mathit{x}}_{\mathit{r}}$. This is obtained for a γ = 2.

2.1.3. Contraction

In the contraction operation, the worse vertex ${\mathit{x}}_{\mathit{h}}^{\mathit{k}}$ is replaced by:

$${\mathit{x}}_{\mathit{c}}^{\mathit{k}}=\mathit{\beta }{\mathit{x}}_{\mathit{h}}^{\mathit{k}}+(1-\mathit{\beta }){\overline{\mathit{x}}}^{\mathit{k}}$$

(2)

Considering that f_r ≥ f_h, or:

$${\mathit{x}}_{\mathit{c}}^{\mathit{k}}=\mathit{\beta }{\mathit{x}}_{\mathit{r}}^{\mathit{k}}+(1-\mathit{\beta }){\overline{\mathit{x}}}^{\mathit{k}}$$

(3)

When f_r < f_h; β is the coefficient of contraction. In Figure 4 the operation of contraction is shown for two dimensions n = 2. The distance between the points ${\mathit{x}}_{\mathit{h}}$ and $\overline{\mathit{x}}$ corresponds to two times the distances between $\overline{\mathit{x}}$ and ${\mathit{x}}_{\mathit{c}1}$, or $\overline{\mathit{x}}$ and ${\mathit{x}}_{\mathit{c}2}$. This is obtained for a β = ½.

2.1.4. Shrinkage

A specific form of contraction can be employed within an iteration to prevent over-expansion. This operation is referred to as “shrinkage”, and it differs from the standard contraction in that it involves the retention of a smaller number of points to generate the new simplex. In regular contraction, all vertices are retained. In shrinkage, however, only one point, designated as the best vertex, ${\mathit{X}}_{\mathit{l}}^{\mathit{k}}$, is kept. Shrinkage is made if ${\mathit{f}}_{\mathit{c}}>{\mathit{f}}_{\mathit{h}}$, then the new simplex is defined by

$${\mathit{X}}_{\mathit{i}}^{\mathit{k}+1}=\mathit{\delta }{\mathit{X}}_{\mathit{i}}^{\mathit{k}}+\left(1-\mathit{\delta }\right){\mathit{X}}_{\mathit{l}}^{\mathit{k}},\forall \mathit{i} \mathit{and} \mathit{i}\ne \mathit{l}$$

(4)

where $\mathit{\delta }$ is the shrinkage coefficient and 0 < $\mathit{\delta }$ < 1. Figure 5 illustrates a two-dimensional shrinkage step with n = 2 and a shrinkage coefficient $\mathit{\delta }$ equal to 1/2.

In the case studied and presented in this paper, they were considered six variables, so the geometrical array is to be a heptagon, if the traditional NMS algorithm was used. As an alternative to these mathematical and practical problems, the named Parallel Simplex is proposed.

3. The Parallel Simplex Proposal

This document concentrates on the proposed alternative of the parallel simplex. Figure 6 shows the movements of the simplex (geometrical array), with the basic operations of Reflection, Expansion, and Contraction, always moving towards the objective of the response variable. As shown in Figure 5, the parallel simplex approaches the objective of the response variable with three simplexes, independent between them, but pursuing the same response and objective. After an exhaustive literature review, nothing similar was found. Many authors have tried to minimize the problems of non-convergency and the high number of iterations adding or modifying the original Nelder Mead simplex, nevertheless, the results are very questionable because of the following:

• The complexity of the algorithm still increases with the number of input variables.
• Almost all the evidence found is generated through known test functions (thus, deterministic approximations).
• Almost all the cases presented are computer-simulated situations.

Maintaining independence among the three simplexes assures the efficiency of the algorithm, avoiding the increase of the complexity of the calculus, thus reducing the number of iterations, as it will happen if the traditional array of a heptagon (n + 1 dimensions) is used. Because the three simplexes chase the same objective, the simplex operations are the same for every single simplex.

4. The Analyzed Process

The process analyzed corresponds to the manufacturing of plastic profiles, to be applied as seals for refrigerators, washing machines, and other domestic equipment. Plastic is melted through a simple plasticizing screw and extruded through a nozzle with the designed profile. In Figure 7, the selected profile is shown. This profile was chosen because it presented a high rejection rate (100% rejection of lots). A 100% inspection and rework were used on a regular basis.

This seal showed a very high rate of defects, mainly, incomplete shots. This defect is shown in Figure 8.

The main control variables of this process are the cubic feet per minute of air (at a constant 4 bar pressure) injected by four nozzles. This is done to keep the profile of the extrusion, as shown in Figure 9. The fifth variable is the height of the puller. This height is the distance between the rollers that pull the extruded material. The sixth variable is the speed of the extruder and the puller (that are synchronized). The output variable is measured as defects per 1000 parts as λ (the continuous extruded material is cut to the design length).

At this point, it is very important to consider that if a traditional/classic array (two-level factorial design) is used, it will imply the following:

• The design, set up, and run of 2⁶ runs, that is 64 runs.
• If more replicates are used for better-fit estimations, the number of runs will increase.
• The majority of products will not be conformant to specifications.
• Production will be lost because the process must stop (to set up each run).
• Extra resources (time, technicians, operators, materials, etc.) will be needed.

The principle of Evolutive Operations of these direct search algorithms, as the NMS, allows the investigator to search for the best response while the process is running. The small changes induced in the process allow for reduction or even elimination of the generation of non-conformant parts.

The Experimental Array

In this case, the objective of the response variable y is cero defects per 1000 parts. The control variables and the initial vertexes of the simplexes are set up in the software. This is shown in Figure 10.

The first set of geometrical arrays—three triangles, three vertexes each one—is set up; three combinations for each pair of variables, ${\mathit{X}}_1$ and ${\mathit{X}}_2$; ${\mathit{X}}_3$ and ${\mathit{X}}_4$; ${\mathit{X}}_5$ and ${\mathit{X}}_6$. The process is started and the feedback of the response is input into the software. The software calculates, on every new iteration, a new combination (vertex) for each simplex (in classical experimentation, these combinations or runs are determined in advance. Luckily some runs will be useful to estimate changes in the response, but many others will not).

Table 3. Results of the Parallel Simplex Process.

Vertice No.	Simplex No.	Air Out 01	Air Out 02	Air Out 03	Air Out 04	Puller Height	Extrusion Speed	RD	Ranking	TETA
1	1	0.2000	0.1800	0.1800	0.2200	0.6900	22.4000	0.005	B	0.0000000000
2	1	0.3000	0.2800	0.2500	0.3200	0.7200	24.0000	0.085	Nw	0.0000000000
3	1	0.1500	0.1500	0.1500	0.1700	0.6500	21.0000	0.130	W	0.0026722222
4	1	0.3500	0.3100	0.2800	0.3700	0.7600	25.4000	0.500	R	0.0470722222
5	1	0.3000	0.2700	0.2475	0.3200	0.6775	22.1000	0.099	Cw	0.0017146667
6	2	0.2000	0.1700	0.1775	0.2200	0.6475	20.5000	0.050	R	0.0014735556
7	3	0.1000	0.0800	0.1100	0.1200	0.6600	20.8000	0.075	R	0.0008388889
8	3	0.1500	0.1275	0.1444	0.1700	0.6644	21.1250	0.065	Cr	0.0006500000
9	4	0.2500	0.2175	0.2119	0.2700	0.6519	20.8250	0.060	R	0.0000389000
10	5	0.3000	0.2600	0.2450	0.3200	0.6350	20.2000	0.090	R	0.0002888889
11	5	0.2625	0.2269	0.2198	0.2825	0.6570	20.8938	0.105	Cw	0.0005722222
12	6	0.2125	0.1794	0.1855	0.2325	0.6527	20.5688	0.006	R	0.0016402222
13	6	0.1938	0.1603	0.1723	0.2138	0.6530	20.4406	0.005	E	0.0016722222
14	7	0.1313	0.1034	0.1299	0.1513	0.6435	20.0469	0.060	R	0.0005722222
15	7	0.1641	0.1343	0.1524	0.1841	0.6469	20.2586	0.056	Cr	0.0005180000
16	8	0.2328	0.2009	0.2000	0.2528	0.6509	20.7117	0.003	R	0.0005615556
17	8	0.2836	0.2496	0.2350	0.3036	0.6546	21.0441	0.101	E	0.0005180000
18	9	0.2688	0.2366	0.2251	0.2888	0.6515	20.9531	0.025	R	0.0003686667
19	10	0.3016	0.2674	0.2476	0.3216	0.6549	21.1648	0.106	R	0.0019615556
20	10	0.2762	0.2431	0.2300	0.2962	0.6493	20.6662	0.096	Cw	0.0015748889
21	11	0.2402	0.2074	0.2049	0.2602	0.6487	20.4248	0.060	R	0.0014660000
22	12	0.1969	0.1652	0.1749	0.2169	0.6503	20.4703	0.002	R	0.0007348889
23	12	0.1572	0.1262	0.1473	0.1772	0.6507	20.3724	0.060	E	0.0007220000
24	13	0.1895	0.1587	0.1699	0.2095	0.6524	20.7572	0.002	R	0.0000002222

shows the final results of the experimentation process.

The NMS algorithm categorizes the response as Best (B), Near the Worse (N_W), and Worse (W), considering the distance of the response from the objective (in this case a minimum of cero is intended). This is documented as the Ranking of the response. There are a lot of stop criteria of the algorithm in the literature, nevertheless, the Teta parameter developed [12] was included because of its simplicity and accuracy.

$$\sum _{\mathbf{i}=1}^{\mathbf{k}+1}{\displaystyle \frac{{({\mathbf{y}}_{\mathbf{i}}-\overline{\mathbf{y}})}^2}{\mathbf{k}+1}}\le \mathsf{\theta }$$

(5)

This is a comparison of the standard error of the response against a predetermined value, typically θ = 1 × 10⁻⁶, for test functions. In real practical processes, a value of Teta of θ = 1 × 10⁻³ is more than acceptable.

5. Results Analysis

In Table 3, it is shown that the algorithm stopped at the 24th iteration, obtaining a desired 0.002 on the dual response and a Teta value of 2.22 × 10⁻⁷. Before any other comparison, a two proportions analysis was made to compare the process with the new set-up. Historical data of 3 months against lots of 1000 pieces were utilized. The analysis showed a p-value of 0.0000 for the probability of the alfa error, for the hypothesis that ${\mathit{H}}_0:{\mathit{P}}_{\mathit{i}\mathit{n}\mathit{i}\mathit{t}\mathit{i}\mathit{a}\mathit{l}}={\mathit{P}}_{\mathit{f}\mathit{i}\mathit{n}\mathit{a}\mathit{l}}$; thus, the difference is statistically significant.

Utilizing the 17th version of Minitab^®, it can be seen that, for a full factorial design needed to estimate a Response Surface, 90 runs are required for six variables (with the minimum of central, axial, and radial points), and at minimum, 53 runs are required for a half design.

These runs were not executed because they implied many runs, much material, many scraps, etc. Nevertheless, data from the Parallel Simplex results were utilized to feed the Response Surface array and compare the results. This comparison is shown in

Table 4. Parallel Simplex vs. Response Surface Comparison.

Variable	Parallel Simplex	Response Surface
$x_1$ (Air 1)	0.19	0.2059
$x_2$ (Air 2)	0.16	0.1768
$x_3$ (Air 3)	0.17	0.1813
$x_4$ (Air 4)	0.21	(Out of the analysis due to lack of effect)
$x_5$ (Puller Height)	0.65	0.6514
$x_6$ (Extrusion Speed)	20.75	20.66

.

The internal operation of the Parallel Simplex is based on the continuous iteration of three independent simplexes, with two variables each in this case. As a further comparison, a Response Surface analysis was made for each pair of variables (simplex), utilizing the algorithm results (again, these runs were not executed). A full model with principal and interaction effects was designed and analyzed.

$$\mathit{f}\left(\mathit{x}\right)={\mathit{\beta }}_0+{{\mathit{\beta }}_1\mathit{x}}_1+{\mathit{\beta }}_2{\mathit{x}}_2+{\mathit{\beta }}_3{\mathit{x}}_1^2+{\mathit{\beta }}_4{\mathit{x}}_2^2+{\mathit{\beta }}_5{\mathit{x}}_1{\mathit{x}}_2+{\mathit{e}}_{\mathit{i}\mathit{j}}$$

(6)

As part of the comparison, a canonical analysis was made to ensure that the optimum found with the Poly Simplex algorithm corresponds to a local minimum. The regression analysis for the first simplex resulted in the equation:

$$R_d=0.742-16.93x_1+10.90x_2+126x_1^2+76x_2^2-190x_1{x}_2$$

(7)

With an adjusted ${\mathit{R}}^2=71.55\%$ and p-values for ${\mathit{x}}_1$ and ${\mathit{x}}_2$ of 0.000 and 0.003.

The stationary point estimated for this pair of variables is:

Considering matrix B and b

$$\mathit{B}=\left[\begin{array}{cc}126& \mathrm{-95}\\ \mathrm{-95}& 76\end{array}\right] \mathrm{and} \mathit{b}=\left[\begin{array}{c}\mathrm{-16.93}\\ 10.90\end{array}\right]$$

Thus

${\mathit{x}}_{\mathit{s}}=-{\displaystyle \frac{1}{2}}{\mathit{B}}^{\mathrm{-1}}\mathit{b}=\left[\begin{array}{c}0.227931\\ 0.213203\end{array}\right]$; the canonical conversion shows that $\mathit{\lambda }=\left[\begin{array}{c}199.234\\ 2.765\end{array}\right]$ for ${\mathit{x}}_1$ and ${\mathit{x}}_2$ as both roots are positive, this stationary point is a minimum.

The regression analysis for the second simplex resulted in the equation:

$${\mathit{R}}_{\mathit{d}}=0.075+12.4{\mathit{x}}_3-10.9{\mathit{x}}_4+454{\mathit{x}}_3^2+343{\mathit{x}}_4^2-788{\mathit{x}}_3{\mathit{x}}_4$$

(8)

With an adjusted ${\mathit{R}}^2=75.22\%$ and p-values for ${\mathit{x}}_3$ and ${\mathit{x}}_4$ of 0.000 and 0.723.

The stationary point estimated for this pair of variables is:

Considering matrix B and b

$$\mathit{B}=\left[\begin{array}{cc}454& \mathrm{-394}\\ \mathrm{-394}& 343\end{array}\right] \mathrm{and} \mathit{b}=\left[\begin{array}{c}12.40\\ \mathrm{-10.90}\end{array}\right]$$

Thus

${\mathit{x}}_{\mathit{s}}=-{\displaystyle \frac{1}{2}}{\mathit{B}}^{\mathrm{-1}}\mathit{b}=\left[\begin{array}{c}0.042593\\ 0.064815\end{array}\right]$; the canonical conversion shows that $\mathit{\lambda }=\left[\begin{array}{c}796.390\\ 0.610\end{array}\right]$ for ${\mathit{x}}_3$ and ${\mathit{x}}_4$ as both roots are positive, this stationary point is a minimum.

The canonical analysis demonstrates that the final combinations of each pair of variables is a local minimum. In

Table 5. Regression Coefficients for Each Model.

Model	X1, X2	X3, X4	X5, X6	Global (X1, X2, X3, X4, X5, X6)
Adjusted R2	71.55%	75.22%	82.52%	93.28
Stationary Point	Minimum	Minimum	Minimum	Saddle

, a resume of the regression coefficients for the six variables at the same time and for each pair of variables is shown.

6. Discussion

The case study presented pieces of evidence that the Parallel Simplex is an efficient alternative to classical experimentation. It is an experimentation process that does not require stopping production, generating scrap, or using additional resources.

For many years, these Direct Search Algorithms were concealed to theoretical approaches, with test functions, etc. One of the main reasons is because the mathematical complexity of the calculus and the sensibility of the algorithms to the internal noise is present in many industrial processes. As a matter of fact, it is difficult to find literature on successful approaches to real processes. In addition, an approach such as the one proposed here was not found in the literature reviewed; three independent simplexes seek to optimize the same response in a synchronous manner. A few examples can be revised on [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32], among many others. This alternative is a powerful way to optimize processes, overcoming the restrictions of the original algorithms.

A summary of analogous algorithms, founded upon the tenets of Nelder and Mead with select adaptations, is provided in

Table 6. Algorithms based on the proposal of Nelder and Mead.

References	Method	Application	Characteristics
Xiong and Jutan (2003) [33]	Dynamic Simplex	Chemical Processes	The conventional Nelder–Mead Method is adapted and expanded to facilitate the monitoring of evolving optima.
Luersen and Le Riche (2004) [34]	Globalized Nelder Mead	Engineering Design	Particularly adapted to tackle multimodal, discontinuous optimization problems, for which it is uncertain that a global optimization can be afforded. Different strategies for restarting the local search. Made more robust by reinitializing degenerated simplexes
Fan S. et al. (2006) [35]	Particle Swarm Optimizer Hybridized with Nelder–Mead Simplex	To locate the global optimal solutions for the nonlinear continuous variable functions mainly focusing on response surface methodology (RSM).	Both the hybrid NM–GA and NM–PSO algorithms incorporate concepts from the NM, GA or PSO, which are readily implemented in practice and the computation of functional derivatives is not necessary.
Chelouah and Siarry (2003) [36]	Continuous Tabu Search and Nelder–Mead	Combinatorial Optimization Problems	Avoid the risk of trapping into a local minimum.

below. Additionally, the table illustrates the characteristics of these algorithms.

7. Conclusions

The proposed algorithm has demonstrated its efficiency by finding the global minimum for the six variables included in the model in only 24 iterations. A traditional model using the original Nelder and Mead algorithm, which requires a heptagon for the analysis, would require a very large number of iterations, and, as already demonstrated, a traditional factorial design would also require a considerable number of experimental runs. Among the most important contributions is the fact that this process was performed during a normal production run, production was not stopped, no extraordinary resources were used, and the generation of nonconforming parts was minimal.

On the other hand, the canonical analysis shows that a local minimum for the response variable has been found for every pair of variables. It is worth mentioning that with the new operating parameters, the process currently has a quality efficiency of 98%. Prior to the implementation of the experimental parameters, the production line was capable of sorting 100% of the material produced. Following the introduction of the new parameters, a sampling procedure was initiated, which revealed a defect rate of 2%.

The objective of this research is to develop an algorithmic approach that can effectively address the complexities inherent in complex production processes, machines, and equipment. To achieve this, the proposed algorithm is designed to focus on simple production processes, where no more than eight variables can be identified. A plethora of algorithms exists, encompassing both stochastic and deterministic approaches. However, these algorithms are predominantly tailored towards intricate processes and systems, where a multitude of variables interacts simultaneously, or in systems with pre-defined functions, which also exhibit a high degree of complexity. These complexities and restrictions impede the ability of the average industrial engineer to apply them. The proposed approach offers several advantages, including the following:

The proposed algorithm is simple, effective for simple processes, does not require knowledge of the function to be optimized, runs without stopping production or processes, and does not require large computational resources. Indeed, anyone can develop a macro in commercial software and execute and interpret its analysis.

“The complexity of the solution is directly proportional to the complexity of the mind that is searching for it”.

Anonymous.

Applying the Parallel concept to a higher dimension (variables) needs to be developed to investigate the efficiency of the algorithm. Further investigations of this algorithm are being developed to accumulate evidence of its practical benefit to the average industrial Engineer. Furthermore, the objective is to develop an application analogous to the one presented for discrete variables.