Handling Constrained Optimization Problems Using the “Re-Generation of Solutions” Strategy ^†

Abstract

Optimization algorithms typically utilize unconstrained mathematical functions to evaluate their performance. Constraints are unavoidable in real-world engineering problems. These constraints should be handled efficiently to obtain the optimal/near-optimal solutions. This paper proposes a new strategy that initially checks for constraint violations, and if none are found, the cost is evaluated. Each new approximate solution generated from the present one is tested until no constraint is violated, subject to a maximum of 10 times in one case and only once in the second case. If satisfied, the cost function is evaluated. This reduces the number of actual function evaluations. Another feature is that all algorithms used for the performance analysis use the same initial candidate solutions for further iterations during each trial. Four constrained mathematical functions and 14 real-world engineering benchmark problems with up to 11 constraints are used for validation. The results are compared with the regular penalty approach. The results show that the strategy that regenerates the candidate solution up to a maximum of 10 times for a single population performs best, followed by the regular penalty strategy.

1. Introduction

Any optimization algorithm is aimed at identifying the best solution, which need not be an optimal one, from a set of feasible alternatives. The objective may be either minimizing or maximizing a particular cost function, termed the “Objective Function”, with or without constraints. Optimization algorithms iteratively explore the search space, evaluate candidate solutions, and update them based on defined rules or heuristics until convergence criteria are met. Optimization algorithms are efficient tools to achieve cost-effective solutions in all domains, including real-world engineering, economics, management, and design problems. They are classified in many ways, such as single objective/multi-objective, deterministic/stochastic, gradient-based/derivative-free, exact/heuristic, and local search/population-based, to name a few.

In recent years, population-based optimization algorithms have predominantly been used for handling real-world constrained problems. The Sine (AB) and Cosine (AB) algorithms [1], two-phase trigonometric T-Cos algorithm [2], two-phase trigonometric TP-AB algorithm [3], Hippopotamus optimization algorithm [4], Secretary bird optimization algorithm [5], and a recent enhanced LSHADE-based algorithm [6] are a few optimization algorithms that fall under this category. Real-world engineering, which includes design problems, invariably includes constraints, which can be either equal or unequal in nature. Various surveys show that constraint handling is important to define the efficacy of any optimization algorithm and hence remains an active research area. Rahimi et al. [7] reviewed a few constraint-handling techniques, like the penalty function, the separation of constraints and objectives, retaining infeasible solutions in the population, and hybrid strategies.

Adding a penalty term (usually of a very high value) to the objective function is a simple and widely used constraint-handling strategy. The penalty may be static, adaptive, or dynamic in nature. Deb [8] explored GA’s population-based approach and ability to make pair-wise comparisons in the tournament selection operator in order to devise a penalty function approach that does not require any penalty parameter. Although penalty and feasibility-based methods are most popular, advanced techniques like ε-constrained and surrogate-based approaches are gaining importance in real-world engineering problems with complex constraints.

A Levy flight is a random walk that has a heavy-tailed probability distribution and is widely used in optimization algorithms. The step-lengths mostly show small variations, and very large jumps are rare. This is different from Gaussian random walks, which have light tails. The heavy tail feature helps to avoid local optima in optimization problems and explores the global search space well while still allowing for exploitation at the local level. A comprehensive review of the statistical properties, usage, and applications of the Levy flight strategy used in metaheuristic algorithms was carried out by Li et al. [9]. It was shown that the Levy Flight strategy, if used properly, could further improve the efficiency of an optimization algorithm [10].

This paper uses this Levy Flight strategy to generate random candidate solutions and evaluates the cost function only if the constraints are not violated for a pre-defined number of trials, either once or ten times.

2. Strategy Used

In population-based optimization algorithms, the process starts with a set of random candidate solutions, usually generated using random numbers within the given bounds. The common expression used for this is

Candidate solutions = Lower Bound + Random Numbers [0, 1]. × Upper Bound

(1)

These approximate solutions (population size) are iteratively moved towards the optimal/near-optimal solution in each iteration. Since this category of algorithms is highly stochastic in nature, several trials are conducted to compute statistical metrics like the “minimum”, “mean,” and “standard deviation” of the computed cost. This procedure is repeated independently for other algorithms whose performances are to be compared with a new algorithm.

In the widely used penalty approach to handle constrained optimization problems, the violations are multiplied by a penalty parameter (which is of a very high value, in the order of 10⁹) and added to the cost. This naturally results in a higher gross cost for any infeasible solution. However, the population and its corresponding higher cost are sometimes updated. This infeasible solution will be used during the next iteration to generate another draft solution, which may be a feasible or an infeasible one.

In this work, this regular strategy is modified as follows:

• During each trial, the same initialized candidate solution set is fed to all the algorithms (in our paper, three) under consideration (Figure 1). These are then individually and iteratively refined towards an acceptable solution. This ensures a fair comparison for all algorithms.
• If the constraints are not violated, the cost function is evaluated for a candidate solution. Otherwise, the candidate solution is regenerated a few times—in our paper, this happens once (strategy “K1”) in one case and a maximum of 10 times in the other case (strategy “K10”) or until the constraints are satisfied. If the constraints are satisfied and the computed cost is less than the existing cost for that solution, then only that population (solution) and the corresponding cost are updated. If the constraints are still violated, the present population is retained, and the iteration moves to the next candidate solution (Figure 2). The reason for this is that, in some problems, evaluating the objective function is relatively complex and may take much more time than checking for a violation of the constraints.
• If the cost is less than the best cost available until that moment, the best cost and the corresponding best solution are updated.

Three strategies are considered, and the same initialized candidate solution set (in this paper, the population size is 5) is passed to all the strategies for further iteration. Initial populations are analyzed using the penalty approach (penalty parameter 10⁹); hence, in some cases, we also obtain a high cost.

Regular: The regular strategy is carried out using the penalty approach (penalty parameter 10⁹).

K1: The new solution (within the bounds) is checked for the constraints. If violated, the solution is generated once again. If the constraints are satisfied, the cost is evaluated; otherwise, the original solution is retained without function evaluation. The cost and population are updated when the cost is less than the present cost.

K10: The new solution (within the bounds) is checked for the constraints. If these are violated, the solution is regenerated repeatedly for a maximum of 10 times. If the constraints are satisfied, the cost is evaluated; otherwise, the original solution is retained without function evaluation. The cost and population are updated when the cost is less than the present cost.

The second strategy is referred to as the “K1” algorithm, and the third one as “K10”; “K” indicates the maximum number of regenerations. The important point here is that the Number of Function Evaluations (NFEs) and updates per iteration for the second algorithm (K1) are much lower than those for the regular algorithm. The NFEs for the third algorithm (K10) are less than or equal to those of the regular algorithm.

One single trial loop could be represented as follows:

• Approximate solutions are initialized;
• A cost evaluation is conducted with penalty parameters;
• The best cost and best solution are determined;
• The same initial solution set is passed to all three algorithms (Figure 1);
• The iteration loop starts;
• A new solution is generated;
• A check for constraint violations is carried out;
• Function evaluations are conducted without penalty parameters;
• An update is carried out if required;
• The best cost and best solution are determined;
• The iteration loop ends (Figure 2).

The complexity of the new strategy is as follows:

Initialization: O(N); N—population size.

Updating: O(N) + O(I × N × D); I—number of iterations; D—dimensions (variables). As one function evaluation takes place per iteration, the complexity is: O(N × (I × D + 1)).

3. Updating Expression (Using Levy Flight Strategy)

The levy function, referred to as LF, is represented in this work as

LF = Levy(Population Size, Dimension, Shape Parameter 1.5)

(2)

The updating expression used here for obtaining a new solution is

New Solution = Old Solution + Sine (LF) × [(Old Solution − 1) − (Old Solution + 1)]

(3)

In the “K1” and “K10” strategies, the objective function is evaluated only if the constraints are not violated.

“LF” generates a set of random numbers using the Levy Flight Strategy (LFT), with 1.5 as shape parameter “β”. When compared to the normal distribution, the Levy distribution can generate relatively larger steps. In such a case, the output contains a mixture of larger and smaller steps, which has the potential to better avoid the local minima in the search space being explored.

The expression used to represent an approximated Levy distribution [11] could be as follows:

L(s)~s^{(−1 − β)}

(4)

where ‘s’ is the step size.

The trigonometric “Sine” function is used in the updating expression as “Sine(LF)”.

4. Results and Discussions

An online MATLAB R2025b version is used for the analyses, and codes are run on an i5 desktop PC with 4 GB RAM. For the analyses, 30 trials and 1000 iterations per trial are carried out to estimate the statistical metrics of “minimum”, “mean”, and “standard deviation”. The population size is taken as five in all cases. In a single-phase algorithm, one function evaluation occurs per iteration, resulting in 5000 function evaluations for each trial.

Two benchmark sets are used for analysis. The first set (

Table 1. Single-objective constrained mathematical functions; dimension of 15 [12].

S.No.	Problem	Characteristic	No. of Constraints	Bounds	Optimal Value
1.	F1	Multimodal	1 Inequality	[0, 5]	4
2.	F4	Unimodal	1 Inequality	[0.001, 2]	0
3.	F15	Unimodal	1 Inequality	[0.001, 2]	1
4.	F19	Multimodal	1 Inequality	[0, 1]	1.8

) is a collection of four constrained mathematical functions proposed by George Anescu [12], and the second set contains 14 real-world constrained engineering problems. The function number refers to the corresponding function number in the original paper in Table 1. The second set (

Table 2. Single-objective constrained real-world engineering problems [16].

S.No.	Problem	No. of Variables	No. of Constraints	Optimal Value
1.	Himmelblau’s Function	5	6	−30,665.5
2.	Speed Reducer	7	11	2994.424466
3.	Helical Spring (Continuous)	3	4	0.01266051
4.	Three-Bar Truss	2	3	263.8958434
5.	Welded Beam	4	7	1.7248523
6.	Car Side Impact	11	10	22.8429695
7.	Piston Lever	4	4	8.4126983
8.	Pressure Vessel	4	4	6059.7143350
9.	I-shaped Beam	4	2	0.0130741
10.	RCC Beam	3	2	359.2080
11.	Cantilever Stepped Beam (Continuous)	10	11	63,408.9
12.	Tubular Column	2	6	26.4863615
13.	Corrugated Bulkhead	4	6	6.8429580
14.	Helical Spring (Discrete)	3	8	2.6585592

) has discrete variables, continuous variables, and a mixture of both continuous and discrete variables. The first two variables are discrete in the RCC Beam, Pressure Vessel, and Helical Spring (discrete) cases. Himmelblau’s Function is obtained from the paper of Himmelblau [13], the Helical Spring (Discrete) instance is extracted from the work of Deb and Goyal [14], and the Cantilever Stepped (Continuous) Beam is discussed in the Help Center of MathWorks [15]. The remaining problems are taken from the “Social network search paper” [16].

The results are presented in

Table 3. Constrained mathematical functions: results.

Problem	Optimal Value	Min	Mean	SD
F1: Regular	4.0	4.9366707	7.7976277	1.2864387
K1		6.1580119	8.3346652	1.4332139
K10		4.8932043	8.3361816	2.1063807
F4: Regular	0	0.5792942	1.2246958	0.4669716
K1		0.4926308	1.1226469	0.3590086
K10		0.3677987	0.7755602	0.2605655
F15: Regular	1.0	1.0011398	1.0037043	0.0016193
K1		1.0015624	1.0036164	0.0014540
K10		1.0005868	1.0020976	0.0009990
F19: Regular	1.8	1.8992905	2.8110932	0.6328919
K1		1.9383421	2.8363200	0.7562850
K10		1.8464927	2.6629982	0.6749263

and

Table 4. Constrained real-world engineering problems: results.

Problem	Min	Mean	SD
Himmelblau: Regular	−30,657.9674780	−30,614.6671935	46.7047285
K1	−30,661.6926538	−30,617.3556259	41.0359641
K10	−30,664.7063796	−30,632.6070263	37.8750887
Speed Reducer: Regular	3001.0626050	3017.7170651	15.8469398
K1	3007.3394980	3,861,193,802	11,350,246,612
K10	2995.8899061	3022.6586688	17.3112739
Helical Spring (Continuous): Regular	0.0128217	0.0137514	0.0024751
K1	0.0128348	241,432,136	445,942,789
K10	0.0127191	0.0130895	0.0004680
Three-Bar Truss: Regular	263.8964592	263.9010257	0.0045371
K1	263.8993757	171,170,628	596,231,272
K10	263.8958559	263.9016936	0.0142651
Welded Beam: Regular	1.7877069	1.9412587	0.1396002
K1	1.8468012	2.4978728	0.6138675
K10	1.7267594	1.8433164	0.1386996
Car Side Impact: Regular	23.1909253	23.7333314	0.3408969
K1	23.0061082	24.0275379	0.9740723
K10	23.0063521	23.5621429	0.3670337
Piston Lever: Regular	11.6042840	158.7909856	140.7437185
K1	10.7263761	288.6032338	372.5353723
K10	8.8328989	147.0207044	148.6049030
Pressure Vessel: Regular	6192.8564472	6961.1577255	586.5114785
K1	6116.4369864	7778.6237699	1403.6690382
K10	6064.1829642	6245.8480922	219.3360143
I-Beam Deflection: Regular	0.0130743	0.0131431	0.0001068
K1	0.0130810	6,633,817,936	12,917,884,142
K10	0.0130741	0.0130890	0.0000172
RCC Beam: Regular	359.2080576	359.7354269	1.0471172
K1	359.2081085	360.7671037	1.8385462
K10	359.2080000	359.2455595	0.1067344
Cantilever Stepped Beam (Continuous): Regular	66,137.0891994	68,785.1731999	1724.2742879
K1	71,300.4482711	2,325,340,868,702	4,974,943,631,868
K10	65,003.3343667	68,694.2597915	2743.8691191
Tubular Column: Regular	26.4863894	26.4927486	0.0158673
K1	26.4864146	26.5707110	0.1419728
K10	26.4863615	26.4868792	0.0014304
Corrugated Bulkhead: Regular	6.8510643	6.9412326	0.2375057
K1	6.8477011	7.0067943	0.2877396
K10	6.8431122	6.8539581	0.0137809
Spring (Discrete): Regular	2.6588350	3.0571963	0.4861107
K1	3.0174411	1,968,852,696,717,080	4,567,348,336,406,490
K10	2.6587265	3.0993425	0.4300373

, and the best results are shown in bold italics.

Since the penalty parameter (10⁹) is used, in some cases, the results are shown as struckthrough numerical values if the result violates the constraints, resulting in very high values. In such cases, the penalty parameter is multiplied with the violation. This only occurred for the “K1” strategy.

For example, the RCC Beam is briefed here. The total cost is to be minimized in this problem. The variables are the reinforcement area, A (x1), beam width, b (x2) and beam depth, h (x3). The first two are discrete variables and the third one is continuous. The problem has two constraints. The range of different variables are: x1: [6, 6.16, 6.32, 6.6, 7, 7.11, 7.2, 7.8, 7.9, 8, 8.4]; x2: [28, 29, 30, …, 40]; and 5 ≤ x3 ≤ 10.

The cost function is to minimize f(x) = 2.9 × A + 0.6 × b × h.

The MATLAB code for the objective function is

function z = RCC_Beam (x) x1 = [6, 6.16, 6.32, 6.6, 7, 7.11, 7.2, 7.8, 7.9, 8, 8.4]; nx1 = numel(x1); x2 = 28:40; nx2 = numel(x2); % Beam Design Parameters As = x1(min(floor(x(1) × nx1 + 1),nx1)); b = x2(min(floor(x(2) × nx2 + 1),nx2)); h = x(3); % Objective Function z = 29.4 × As + 0.6 × b × h; end

One simulation of the RCC Beam function is carried out separately to observe the converging pattern. The codes are run for 100 iterations with a population size of 5. This is a three-dimensional problem with a Lower Bound = [0, 0, 5] and Upper Bound = [1, 1, 10]. When simulated by feeding the same set as in the initial population to all three algorithms, we obtain

359.7777068588173392527096439153 for the regular penalty approach.

360.62954282304576736351009458303 for “K1” strategy.

359.35295887363128031211090274155 for the “K10” strategy.

It is clear from the graph (Figure 3) that “K10” also performs better here.

A summary of the simulations is presented in

Table 5. Summary of the results: four constrained mathematical functions.

Algorithm	Minimum	Mean	Standard Deviation	Total
Regular	0	1	2	3
K1	0	0	0	0
K10	4	3	2	9

and

Table 6. Summary of the results: 14 constrained real-world engineering problems.

Algorithm	Minimum	Mean	Standard Deviation	Total
Regular	0	3	5	8
K1	1	0	0	1
K10	13	11	9	33

for the “minimum”, “mean”, and “standard deviation” metrics. For the mathematical functions (Table 5), the “K10” algorithm performs well in 9 of the 12 cases, followed by the “Regular” algorithm. “K1” algorithm draws a blank for this set. Table 2 shows that the “K10” algorithm performs well in 33 cases out of 42 cases, with a significant performance rate of approximately 79%. The important “mean” metric yields better results in 11/14 cases. In the case of “standard deviation”, the Regular strategy performs slightly better, although not as well as “K10”.

5. Conclusions and Future Work

The same set of initialized candidate solutions was passed into all three strategies simultaneously. Then, these approximate solutions were independently and iteratively moved towards the required accuracy level. The “Regular” algorithm, which updates the draft solution even if one or more constraints are violated, performs better than the “K1” algorithm, which regenerates the solution only once if the constraints are violated. In this “K1” algorithm, the solution and cost are not updated if there is any constraint violation. This shows that the candidate solution generated from an unfeasible solution may yield a feasible solution during the next iteration. When regenerated a maximum of 10 times (K10), the results improved significantly without any increase in NFEs. The main inference is that if the number of regenerations is increased, then the results could improve. However, this results in a linear increase in the execution time to some extent, which may be compensated through the use of fewer NFEs. The NFEs used in the second algorithm (K1) will be much fewer than those used in the regular algorithm, and the NFEs in the third algorithm (K10) will be less than or equal to those of the regular algorithm. If the evaluation of the objective function is complex and expensive, and the constraints are simple, then this strategy of initially checking the constraints will help to improve the results. For complex objective functions involving a greater number of constraints, the performance of the algorithm depends on the relative computation time between the objective function and checking the constraints. Future work includes analyzing this strategy using more benchmarks with a higher number of constraints.