Next Article in Journal
Topology on Soft Continuous Function Spaces

Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

# A New Smoothing Nonlinear Penalty Function for Constrained Optimization

by
Touna Yang
1,
Nguyen Thanh Binh
2,3,*,
Tran Manh Thang
4 and
Duong Thi Hoa
3
1
School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China
2
Department of Mathematics, Shanghai University, Shanghai 200444, China
3
Yen Bai Teacher’s Training College, Yen Bai City 320000, Vietnam
4
Department of Education and Training Yen Bai, Yen Bai City 320000, Vietnam
*
Author to whom correspondence should be addressed.
Math. Comput. Appl. 2017, 22(2), 31; https://doi.org/10.3390/mca22020031
Submission received: 5 November 2016 / Revised: 29 March 2017 / Accepted: 4 April 2017 / Published: 12 April 2017

## Abstract

:
In this study, a new smoothing nonlinear penalty function for constrained optimization problems is presented. It is proved that the optimal solution of the smoothed penalty problem is an approximate optimal solution of the original problem. Based on the smoothed penalty function, we develop an algorithm for finding an optimal solution of the optimization problems with inequality constraints. We further discuss the convergence of this algorithm and test this algorithm with three numerical examples. The numerical examples show that the proposed algorithm is feasible and effective for solving some nonlinear constrained optimization problems.

## 1. Introduction

Consider the following constrained optimization problem:
$( P ) min f ( x ) s . t . g i ( x ) ≤ 0 , i = 1 , 2 , … , m , x ∈ R n ,$
where the functions $f , g i : R n → R , i ∈ I = { 1 , 2 , … , m }$, are continuously differentiable functions.
Let $X 0 = { x ∈ R n | g i ( x ) ≤ 0 , i ∈ I }$ be the feasible solution set and we assume that $X 0$ is not empty.
For a general constrained optimization problem, the penalty function method has attracted many researchers in both theoretical and practical aspects. However, to obtain an optimal solution for the original problem, the conventional quadratic penalty function method usually requires that the penalty parameter tends to infinity, which is undesirable in practical computation. In order to overcome the drawbacks of the quadratic penalty function method, exact penalty functions were proposed to solve problem $( P )$. Zangwill [1] first proposed the $l 1$ exact penalty function
$F 1 ( x , ρ ) = f ( x ) + ρ ∑ i = 1 m g i + ( x ) ,$
where $ρ > 0$ is a penalty parameter, and $g i + ( x ) = max { 0 , g i ( x ) } , i = 1 , 2 , … , m$. It was proved that there exists a fixed constant $ρ 0 > 0$, for any $ρ > ρ 0$, and any global solution of the exact penalty problem is also a global solution of the original problem. Therefore, the exact penalty function methods have been widely used for solving constrained optimization problems (see, e.g., [2,3,4,5,6,7,8,9]).
Recently, the nonlinear penalty function of the following form has been investigated in [10,11,12,13]:
$F k ( x , ρ ) = f ( x ) k + ρ ∑ i = 1 m ( g i + ( x ) ) k 1 k ,$
where $f ( x )$ is assumed to be positive and $k ∈ ( 0 , + ∞ )$. It is called the k-th power penalty function in [14,15]. Obviously, if $k = 1$, the nonlinear penalty function $F k ( x , ρ )$ is reduced to the $l 1$ exact penalty function. In [12], it was shown that the exact penalty parameter corresponding to $k ∈ ( 0 , 1 ]$ is substantially smaller than that of the $l 1$ exact penalty function. Rubinov and Yang [13] also studied a penalty function as follows:
$ψ ρ k ( x , c ) = [ f ( x ) − c ] k + ρ ∑ i = 1 m g i + ( x ) ,$
where $c ∈ R$ such that $f ( x ) − c > 0$ for any $x ∈ R n$, and $k ∈ ( 0 , + ∞ )$. The corresponding penalty problem of $( P )$ is defined as
$( P ρ ) min ψ ρ k ( x , c ) s . t . x ∈ R n .$
In fact, the original problem $( P )$ is equivalent to the problem as follows:
$( P ′ ) min [ f ( x ) − c ] k s . t . g i ( x ) ≤ 0 , i = 1 , 2 , … , m , x ∈ R n .$
Obviously, the penalty problem $( P ρ )$ is the $l 1$ exact penalty problem of problem $( P ′ )$ defined as (1).
It is noted that these penalty functions $F 1 ( x , ρ ) , F k ( x , ρ ) ( 0 < k ≤ 1 )$ and $ψ ρ k ( x , c )$ are not differentiable at x such that $g i ( x ) = 0$ for some $i ∈ I$, which prevents the use of gradient-based methods and causes some numerical instability problems in its implementation, when the value of the penalty parameter becomes large [3,5,6,8]. In order to use existing gradient-based algorithms, such as a Newton method, it is necessary to smooth the exact penalty function. Thus, the smoothing of the exact penalty function attracts much attention [16,17,18,19,20,21,22,23,24]. Pinar and Zenios [21] and Wu et al. [22] discussed a quadratic smoothing approximation to nondifferentiable exact penalty functions for constrained optimization. Binh [17] and Xu et al. [23] proposed a second-order differentiability technique to the $l 1$ exact penalty function. It is shown that the optimal solution of the smoothed penalty problem is an approximate optimal solution of the original optimization problem. Zenios et al. [24] discussed an algorithm for the solution of large-scale optimization problems.
In this study, we aim to develop the smoothing technique for the nonlinear penalty function (3). First, we define the following smoothing function $q ϵ , ρ k ( t )$ by
$q ϵ , ρ k ( t ) = 0 if t ≤ 0 , 2 m 2 ρ 2 9 ϵ 2 t 3 k if 0 ≤ t ≤ ϵ m ρ 1 k , t k + ϵ 3 m ρ e − m ρ ϵ t k + 1 − 10 ϵ 9 m ρ if t ≥ ϵ m ρ 1 k ,$
where $0 < k < + ∞ , ρ > 0$ and $ϵ > 0$. By considering this smoothing function, a new smoothing nonlinear penalty function is obtained. We use this smoothing nonlinear penalty function that is able to convert a constrained optimization problem into minimizations of a sequence of continuously differentiable functions and propose a corresponding algorithm for solving constrained optimization problems.
The rest of this paper is organized as follows. In Section 2, we propose a new smoothing penalty function for inequality constrained optimization problems, and some fundamental properties of its are proved. In Section 3, an algorithm based on the smoothed penalty function is presented and its global convergence is proved. In Section 4, we report results on application of this algorithm to three test problems and compare the results obtained with other similar algorithms. Finally, conclusions are discussed in Section 5.

## 2. Smoothing Nonlinear Penalty Functions

In this section, we first construct a new smoothing function. Then, we introduce our smoothing nonlinear penalty function and discuss its properties.
Let $q k ( t ) : R → R$ be as follows:
$q k ( t ) = 0 if t ≤ 0 , t k if t ≥ 0 ,$
where $0 < k < + ∞$. Obviously, the function $q k ( t )$ is $C 1$ on $R$ for $k > 1$, but it is not $C 1$ for $0 < k ≤ 1$. It is useful in defining exact penalty functions for constrained optimization problems (see, e.g., [14,15,21]). Consider the nonlinear penalty function
$F ρ k ( x , c ) = [ f ( x ) − c ] k + ρ ∑ i = 1 m q k ( g i ( x ) ) ,$
and the corresponding penalty problem
$( N P ρ ) min F ρ k ( x , c ) s . t . x ∈ R n .$
As previously mentioned, for any $ϵ > 0$ and $ρ > 0$, the function $q ϵ , ρ k ( t )$ is defined as:
$q ϵ , ρ k ( t ) = 0 if t ≤ 0 , 2 m 2 ρ 2 9 ϵ 2 t 3 k if 0 ≤ t ≤ ϵ m ρ 1 k , t k + ϵ 3 m ρ e − m ρ ϵ t k + 1 − 10 ϵ 9 m ρ if t ≥ ϵ m ρ 1 k ,$
where $0 < k < + ∞$.
Figure 1 shows the behavior of $q k ( t )$ and $q ϵ , ρ k ( t )$.
In the following, we discuss the properties of $q ϵ , ρ k ( t )$.
Lemma 1.
For $0 < k < + ∞$ and any $ϵ > 0$, we have
(i)
$q ϵ , ρ k ( t )$ is continuously differentiable for $k > 1 3$ on $R$, where
$[ q ϵ , ρ k ( t ) ] ′ = 0 i f t ≤ 0 , 2 k m 2 ρ 2 3 ϵ 2 t 3 k − 1 i f 0 ≤ t ≤ ϵ m ρ 1 k , k t k − 1 − 1 3 k t k − 1 e − m ρ ϵ t k + 1 i f t ≥ ϵ m ρ 1 k .$
(ii)
$lim ϵ → 0 q ϵ , ρ k ( t ) = q k ( t )$.
(iii)
$q k ( t ) ≥ q ϵ , ρ k ( t ) , ∀ t ∈ R$.
Proof.
(i) First, we prove that $q ϵ , ρ k ( t )$ is continuous. Obviously, the function $q ϵ , ρ k ( t )$ is continuous at any $t ∈ R \ 0 , ( ϵ m ρ ) 1 k$. We only need to prove that $q ϵ , ρ k ( t )$ continuous at the separating points: 0 and $( ϵ m ρ ) 1 k$.
(1) For $t = 0$, we have
$lim t → 0 − q ϵ , ρ k ( t ) = lim t → 0 − 0 = 0 , lim t → 0 + q ϵ , ρ k ( t ) = lim t → 0 + 2 m 2 ρ 2 9 ϵ 2 t 3 k = 0 ,$
which implies
$lim t → 0 − q ϵ , ρ k ( t ) = lim t → 0 + q ϵ , ρ k ( t ) = 0 = q ϵ , ρ k ( 0 ) .$
Thus, $q ϵ , ρ k ( t )$ is continuous at $t = 0$.
(2) For $t = ( ϵ m ρ ) 1 k$, we have
$lim t → ( ϵ m ρ ) 1 k − q ϵ , ρ k ( t ) = lim t → ( ϵ m ρ ) 1 k − 2 m 2 ρ 2 9 ϵ 2 t 3 k = 2 ϵ 9 m ρ , lim t → ( ϵ m ρ ) 1 k + q ϵ , ρ k ( t ) = lim t → ( ϵ m ρ ) 1 k + t k + ϵ 3 m ρ e − m ρ ϵ t k + 1 − 10 ϵ 9 m ρ = 2 ϵ 9 m ρ ,$
which implies
$lim t → ( ϵ m ρ ) 1 k − q ϵ , ρ k ( t ) = lim t → ( ϵ m ρ ) 1 k + q ϵ , ρ k ( t ) = 2 ϵ 9 m ρ = q ϵ , ρ k ( ϵ m ρ ) 1 k .$
Thus, $q ϵ , ρ k ( t )$ is continuous at $t = ( ϵ m ρ ) 1 k$.
Next, we will show that $q ϵ , ρ k ( t )$ is continuously differentiable, i.e., $[ q ϵ , ρ k ( t ) ] ′$ is continuous. Actually, we only need to prove that $[ q ϵ , ρ k ( t ) ] ′$ is continuous at the separating points: 0 and $( ϵ m ρ ) 1 k$.
(1) For $t = 0$, we have
$lim t → 0 − [ q ϵ , ρ k ( t ) ] ′ = lim t → 0 − 0 = 0 , lim t → 0 + [ q ϵ , ρ k ( t ) ] ′ = lim t → 0 + 2 k m 2 ρ 2 3 ϵ 2 t 3 k − 1 = 0 ,$
which implies
$lim t → 0 − [ q ϵ , ρ k ( t ) ] ′ = lim t → 0 + [ q ϵ , ρ k ( t ) ] ′ = 0 = [ q ϵ , ρ k ( 0 ) ] ′ .$
Thus, $[ q ϵ , ρ k ( t ) ] ′$ is continuous at $t = 0$.
(2) For $t = ( ϵ m ρ ) 1 k$, we have
$lim t → ( ϵ m ρ ) 1 k − [ q ϵ , ρ k ( t ) ] ′ = lim t → ( ϵ m ρ ) 1 k − 2 k m 2 ρ 2 3 ϵ 2 t 3 k − 1 = 2 k 3 ϵ m ρ k − 1 k , lim t → ( ϵ m ρ ) 1 k + [ q ϵ , ρ k ( t ) ] ′ = lim t → ( ϵ m ρ ) 1 k + k t k − 1 − 1 3 k t k − 1 e − m ρ ϵ t k + 1 = 2 k 3 ϵ m ρ k − 1 k ,$
which implies
$lim t → ( ϵ m ρ ) 1 k − [ q ϵ , ρ k ( t ) ] ′ = lim t → ( ϵ m ρ ) 1 k + [ q ϵ , ρ k ( t ) ] ′ = 2 k 3 ϵ m ρ k − 1 k = q ϵ , ρ k ( ϵ m ρ ) 1 k ′ .$
Thus, $[ q ϵ , ρ k ( t ) ] ′$ is continuous at $t = ( ϵ m ρ ) 1 k$.
(ii) For $∀ t ∈ R$, by the definition of $q k ( t )$ and $q ϵ , ρ k ( t ) ,$ we have
$q k ( t ) − q ϵ , ρ k ( t ) = 0 if t ≤ 0 , t k − 2 m 2 ρ 2 9 ϵ 2 t 3 k if 0 ≤ t ≤ ϵ m ρ 1 k , 10 ϵ 9 m ρ − ϵ 3 m ρ e − m ρ ϵ t k + 1 if t ≥ ϵ m ρ 1 k .$
When $0 ≤ t ≤ ϵ m ρ 1 k$, let $u = t k$. Then, we have $0 ≤ u ≤ ϵ m ρ$. Consider the function:
$G ( u ) = u − 2 m 2 ρ 2 9 ϵ 2 u 3 , 0 ≤ u ≤ ϵ m ρ ,$
and we have
$G ′ ( u ) = 1 − 2 m 2 ρ 2 3 ϵ 2 u 2 , 0 ≤ u ≤ ϵ m ρ .$
Obviously, $G ′ ( u ) > 0$ for $0 ≤ u ≤ ϵ m ρ$. Moreover, $G ( 0 ) = 0$ and $G ( ϵ m ρ ) = 7 ϵ 9 m ρ$. Hence, we have
$0 ≤ q k ( t ) − q ϵ , ρ k ( t ) ≤ 7 ϵ 9 m ρ .$
When $t ≥ ϵ m ρ 1 k$, we have
$0 < q k ( t ) − q ϵ , ρ k ( t ) = 10 ϵ 9 m ρ − ϵ 3 m ρ e − m ρ ϵ t k + 1 ≤ 10 ϵ 9 m ρ .$
Thus, we have
$0 ≤ q k ( t ) − q ϵ , ρ k ( t ) ≤ 10 ϵ 9 m ρ .$
That is,
$lim ϵ → 0 q ϵ , ρ k ( t ) = q k ( t ) .$
(iii) For $∀ t ∈ R$, from (ii), we have
$q k ( t ) − q ϵ , ρ k ( t ) ≥ 0 ,$
which is $q k ( t ) ≥ q ϵ , ρ k ( t )$.
This completes the proof. ☐
In this study, we always assume that $c < 0$ and $| c |$ is large enough, such that $f ( x ) − c > 0$ for all $x ∈ R n$. Let
$F ϵ , ρ k ( x , c ) = [ f ( x ) − c ] k + ρ ∑ i = 1 m q ϵ , ρ k g i ( x ) , 0 < k < + ∞ .$
Then, $F ϵ , ρ k ( x , c )$ is continuously differentiable at any $x ∈ R n$ and is a smooth approximation of $F ρ k ( x , c )$. We have the following smoothed penalty problem:
$( N P ϵ , ρ ) min F ϵ , ρ k ( x , c ) s . t . x ∈ R n .$
Lemma 2.
We have that
$0 ≤ F ρ k ( x , c ) − F ϵ , ρ k ( x , c ) ≤ 10 ϵ 9 , 0 < k < + ∞$
for any $x ∈ R n , ϵ > 0$ and $ρ > 0$.
Proof.
For any $x ∈ R n , ϵ > 0 , ρ > 0$, we have
$F ρ k ( x , c ) − F ϵ , ρ k ( x , c ) = ρ ∑ i = 1 m q k ( g i ( x ) ) − q ϵ , ρ k ( g i ( x ) ) .$
Note that
$q k ( g i ( x ) ) − q ϵ , ρ k ( g i ( x ) ) = 0 if g i ( x ) ≤ 0 , [ g i ( x ) ] k − 2 m 2 ρ 2 9 ϵ 2 [ g i ( x ) ] 3 k if 0 ≤ g i ( x ) ≤ ϵ m ρ 1 k , 10 ϵ 9 m ρ − ϵ 3 m ρ e − m ρ ϵ [ g i ( x ) ] k + 1 if g i ( x ) ≥ ϵ m ρ 1 k ,$
for any $i ∈ I$.
By Lemma 1, we have
$0 ≤ ∑ i = 1 m ( q k ( g i ( x ) ) − q ϵ , ρ k ( g i ( x ) ) ) ≤ 10 ϵ 9 ρ ,$
which implies
$0 ≤ ρ ∑ i = 1 m ( q k ( g i ( x ) ) − q ϵ , ρ k ( g i ( x ) ) ) ≤ 10 ϵ 9 .$
Hence,
$0 ≤ F ρ k ( x , c ) − F ϵ , ρ k ( x , c ) ≤ 10 ϵ 9 .$
This completes the proof. ☐
Lemma 3.
Let $x *$ and $x ρ *$ be optimal solutions of problem $( P )$ and problem $( N P ρ ) ,$ respectively. If $x ρ *$ is a feasible solution to problem $( P )$, then $x ρ *$ is an optimal solution for problem $( P )$.
Proof.
Under the given conditions, we have that
$0 < [ f ( x ρ * ) − c ] k = F ρ k ( x ρ * , c ) ≤ F ρ k ( x * , c ) = [ f ( x * ) − c ] k , 0 < k < + ∞ .$
Therefore, $f ( x ρ * ) − c ≤ f ( x * ) − c$, which is $f ( x ρ * ) ≤ f ( x * )$.
Since $x *$ is an optimal solution and $x ρ *$ is feasible to problem $( P )$, which is
$f ( x ρ * ) ≥ f ( x * ) .$
Therefore, $x ρ *$ is an optimal solution for problem $( P )$.
This completes the proof. ☐
Theorem 1.
Let $x ρ *$ and $x ϵ , ρ *$ be the optimal solutions of problem $( N P ρ )$ and problem $( N P ϵ , ρ ) ,$ respectively, for some $ρ > 0$ and $ϵ > 0$. Then, we have that
$0 ≤ F ρ k ( x ρ * , c ) − F ϵ , ρ k ( x ϵ , ρ * , c ) ≤ 10 ϵ 9 , 0 < k < + ∞ .$
Furthermore, if $x ρ *$ satisfies the conditions of Lemma 3 and $x ϵ , ρ *$ is feasible to problem $( P )$, then $x ϵ , ρ *$ is an optimal solution for problem $( P )$.
Proof.
By Lemma 2, for $ρ > 0$ and $ϵ > 0$, we obtain
$0 ≤ F ρ k ( x ρ * , c ) − F ϵ , ρ k ( x ρ * , c ) ≤ F ρ k ( x ρ * , c ) − F ϵ , ρ k ( x ϵ , ρ * , c ) ≤ F ρ k ( x ϵ , ρ * , c ) − F ϵ , ρ k ( x ϵ , ρ * , c ) ≤ 10 ϵ 9 .$
That is,
$0 ≤ F ρ k ( x ρ * , c ) − F ϵ , ρ k ( x ϵ , ρ * , c ) ≤ 10 ϵ 9 ,$
and
$0 ≤ [ f ( x ρ * ) − c ] k + ρ ∑ i = 1 m q k ( g i ( x ρ * ) ) − [ f ( x ϵ , ρ * ) − c ] k + ρ ∑ i = 1 m q ϵ , ρ k ( g i ( x ϵ , ρ * ) ) ≤ 10 ϵ 9 .$
From the definition of $q k ( t ) , q ϵ , ρ k ( t )$ and the fact that $x ρ * , x ϵ , ρ *$ are feasible for problem $( P )$, we have
$∑ i = 1 m q k ( g i ( x ρ * ) ) = ∑ i = 1 m q ϵ , ρ k ( g i ( x ϵ , ρ * ) ) = 0 .$
Note that $f ( x ϵ , ρ * ) − c > 0$, and from (8), we have
$0 < [ f ( x ϵ , ρ * ) − c ] k ≤ [ f ( x ρ * ) − c ] k .$
Therefore, $f ( x ϵ , ρ * ) − c ≤ f ( x ρ * ) − c$, which is $f ( x ϵ , ρ * ) ≤ f ( x ρ * )$.
As $x ϵ , ρ *$ is feasible to $( P )$ and by Lemma 3, $x ρ *$ is an optimal solution to $( P )$, we have
$f ( x ϵ , ρ * ) ≥ f ( x ρ * ) .$
Thus, $x ϵ , ρ *$ is an optimal solution for problem $( P )$.
This completes the proof. ☐
Definition 1.
A feasible solution $x *$ of problem $( P )$ is called a KKT point, if there exists a $μ * ∈ R m$ such that the solution pair $( x * , μ * )$ satisfies the following conditions:
$∇ f ( x * ) + ∑ i = 1 m μ i * ∇ g i ( x * ) = 0 ,$
$μ i * g i ( x * ) = 0 , g i ( x * ) ≤ 0 , μ i * ≥ 0 , i ∈ I .$
Theorem 2.
Suppose the functions $f , g i ( i ∈ I )$ in problem (P) are convex. Let $x *$ and $x ϵ , ρ *$ be the optimal solutions of problem $( P )$ and problem $( N P ϵ , ρ ) ,$ respectively. If $x ϵ , ρ *$ is feasible to problem $( P )$, and there exists a $μ * ∈ R m$ such that the pair $( x ϵ , ρ * , μ * )$ satisfies the conditions in Equations (9) and (10), then we have that
$0 ≤ [ f ( x ϵ , ρ * ) − c ] k − [ f ( x * ) − c ] k ≤ 10 ϵ 9 , 0 < k < + ∞ .$
Proof.
Since the functions $f , g i ( i ∈ I )$ are continuously differentiable and convex, we see that
$f ( x * ) ≥ f ( x ϵ , ρ * ) + ∇ f ( x ϵ , ρ * ) T ( x * − x ϵ , ρ * ) ,$
$g i ( x * ) ≥ g i ( x ϵ , ρ * ) + ∇ g i ( x ϵ , ρ * ) T ( x * − x ϵ , ρ * ) , i = 1 , 2 , … , m .$
After applying the conditions given in Equations (9), (10), (12) and (13), we see that
$f ( x * ) ≥ f ( x ϵ , ρ * ) + ∇ f ( x ϵ , ρ * ) T ( x * − x ϵ , ρ * ) = f ( x ϵ , ρ * ) − ∑ i = 1 m μ i * ∇ g i ( x ϵ , ρ * ) T ( x * − x ϵ , ρ * ) ≥ f ( x ϵ , ρ * ) − ∑ i = 1 m μ i * g i ( x * ) − g i ( x ϵ , ρ * ) = f ( x ϵ , ρ * ) − ∑ i = 1 m μ i * g i ( x * ) ≥ f ( x ϵ , ρ * ) .$
Therefore, $f ( x * ) − c ≥ f ( x ϵ , ρ * ) − c > 0$. Thus,
$[ f ( x ϵ , ρ * ) − c ] k ≤ [ f ( x * ) − c ] k ≤ [ f ( x * ) − c ] k + ρ ∑ i = 1 m q k ( g i ( x * ) ) = F ρ k ( x * , c ) .$
By Lemma 2, we have
$F ρ k ( x * , c ) ≤ F ϵ , ρ k ( x * , c ) + 10 ϵ 9 .$
It follows that
$[ f ( x ϵ , ρ * ) − c ] k ≤ F ϵ , ρ k ( x * , c ) + 10 ϵ 9 = [ f ( x * ) − c ] k + ρ ∑ i = 1 m q ϵ , ρ k ( g i ( x * ) ) + 10 ϵ 9 = [ f ( x * ) − c ] k + 10 ϵ 9 .$
Since $x ϵ , ρ *$ is feasible to $( P )$, which is
$f ( x * ) ≤ f ( x ϵ , ρ * ) ,$
then
$f ( x * ) − c ≤ f ( x ϵ , ρ * ) − c ,$
and, by $f ( x * ) − c > 0$, we have
$0 < [ f ( x * ) − c ] k ≤ [ f ( x ϵ , ρ * ) − c ] k .$
Combining Equations (14) and (15), we have that
$[ f ( x * ) − c ] k ≤ [ f ( x ϵ , ρ * ) − c ] k ≤ [ f ( x * ) − c ] k + 10 ϵ 9 ,$
which is
$0 ≤ [ f ( x ϵ , ρ * ) − c ] k − [ f ( x * ) − c ] k ≤ 10 ϵ 9 , 0 < k < + ∞ .$
This completes the proof. ☐

## 3. Algorithm

In this section, by considering the above smoothed penalty function, we propose an algorithm to find an optimal solution of problem $( P )$, defined as Algorithm 1.
Definition 2.
For $ϵ > 0$, a point $x ϵ * ∈ X 0$ is called an ϵ-feasible solution to $( P )$, if it satisfies $g i ( x ϵ * ) ≤ ϵ , ∀ i ∈ I$.
 Algorithm 1: Algorithm for solving problem $( P )$ Step 1: Let the initial point $x 1 0$. Let $ϵ 1 > 0 , ρ 1 > 0 , 0 < γ < 1 , β > 1$ and choose a constant $c < 0$ such that $f ( x ) − c > 0 , ∀ x ∈ X 0$, let $j = 1$ and go to Step 2. Step 2: Use $x j 0$ as the starting point to solve the following problem: $( N P ϵ j , ρ j ) min x ∈ R n F ϵ j , ρ j k ( x , c ) = [ f ( x ) − c ] k + ρ j ∑ i = 1 m q ϵ j , ρ j k g i ( x ) .$ Let $x ϵ j , ρ j *$ be an optimal solution of $( N P ϵ j , ρ j )$ (the solution of $( N P ϵ j , ρ j )$ we obtained by the BFGS method given in [25]). Step 3: If $x ϵ j , ρ j *$ is $ϵ$-feasible for problem $( P )$, then the algorithm stops and $x ϵ j , ρ j *$ is an approximate optimal solution of problem $( P )$. Otherwise, let $ρ j + 1 = β ρ j , ϵ j + 1 = γ ϵ j , x j + 1 0 = x ϵ j , ρ j *$ and $j = j + 1$. Then, go to Step 2.
Remark 1.
From $0 < γ < 1 , β > 1 ,$ we can easily see that as $j → + ∞$, the sequence ${ ϵ j } → 0$ and the sequence ${ ρ j } → + ∞$.
Theorem 3.
For $k > 1 3$, suppose that for $ϵ ∈ ( 0 , ϵ 1 ]$ and $ρ ∈ [ ρ 1 , + ∞ ) ,$ the set
$arg min x ∈ R n F ϵ , ρ k ( x , c ) ≠ ∅ .$
Let ${ x ϵ j , ρ j * }$ be the sequence generated by Algorithm 1. If $lim ∥ x ∥ → + ∞ f ( x ) = + ∞$ and the sequence ${ F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) }$ is bounded, then ${ x ϵ j , ρ j * }$ is bounded and the limit point of ${ x ϵ j , ρ j * }$ is the solution of $( P )$.
Proof.
First, we prove that ${ x ϵ j , ρ j * }$ is bounded. Note that
$F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) = [ f ( x ϵ j , ρ j * ) − c ] k + ρ j ∑ i = 1 m q ϵ j , ρ j k ( g i ( x ϵ j , ρ j * ) ) , j = 0 , 1 , … .$
From the definition of $q ϵ , ρ k ( t )$, we have
$ρ j ∑ i = 1 m q ϵ j , ρ j k ( g i ( x ϵ j , ρ j * ) ) ≥ 0 .$
Suppose, on the contrary, that the sequence ${ x ϵ j , ρ j * }$ is unbounded and without loss of generality $∥ x ϵ j , ρ j * ∥ → + ∞$ as $j → + ∞$, and $f ( x ϵ j , ρ j * ) − c > 0$. Then, $lim j → + ∞ [ f ( x ϵ j , ρ j * ) − c ] k = + ∞$, and from Equations (17) and (18), we have
$F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) ≥ [ f ( x ϵ j , ρ j * ) − c ] k → + ∞ , j = 0 , 1 , … ,$
which contradicts with the sequence ${ F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) }$ being bounded. Thus, ${ x ϵ j , ρ j * }$ is bounded.
Next, we prove that the limit point of ${ x ϵ j , ρ j * }$ is the solution of problem $( P )$. Let $x *$ be a limit point of ${ x ϵ j , ρ j * }$. Then, there exists the subset $J ⊂ N$ such that $x ϵ j , ρ j * → x *$ for $j ∈ J$, where $N$ is the set of natural numbers. We have to show that $x *$ is an optimal solution of problem $( P )$. Thus, it is sufficient to show $( i ) x * ∈ X 0$ and $( ii ) f ( x * ) ≤ inf x ∈ X 0 f ( x )$.
(i) Suppose $x * ∉ X 0$. Then, there exists $θ 0 > 0$ and the subset $J ′ ⊂ J$, such that $g i ′ ( x ϵ j , ρ j * ) ≥ θ 0 > 0$ for any $j ∈ J ′$ and some $i ′ ∈ I$.
If $θ 0 ≤ g i ′ ( x ϵ j , ρ j * ) < ϵ j m ρ j 1 k$, from the definition of $q ϵ , ρ k ( t )$ and $x ϵ j , ρ j *$ is the optimal solution according j-th values of the parameters $ϵ j , ρ j$ for any $x ∈ X 0$, we have
$[ f ( x ϵ j , ρ j * ) − c ] k + 2 m 2 ρ j 3 θ 0 3 k 9 ϵ j 2 ≤ F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) ≤ F ϵ j , ρ j k ( x , c ) = [ f ( x ) − c ] k ,$
which contradicts with $ρ j → + ∞$ and $ϵ j → 0$.
If $g i ′ ( x ϵ j , ρ j * ) ≥ θ 0 ≥ ϵ j m ρ j 1 k$ or $g i ′ ( x ϵ j , ρ j * ) ≥ ϵ j m ρ j 1 k ≥ θ 0$, from the definition of $q ϵ , ρ k ( t )$ and $x ϵ j , ρ j *$ is the optimal solution according j-th values of the parameters $ϵ j , ρ j$ for any $x ∈ X 0$, we have
$[ f ( x ϵ j , ρ j * ) − c ] k + ρ j θ 0 k + ϵ j 3 m e − m ρ j ϵ j θ 0 k + 1 − 10 ϵ j 9 m ≤ F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) ≤ F ϵ j , ρ j k ( x , c ) = [ f ( x ) − c ] k ,$
which contradicts with $ρ j → + ∞$ and $ϵ j → 0$.
Thus, $x * ∈ X 0$.
(ii) For any $x ∈ X 0$, we have
$[ f ( x ϵ j , ρ j * ) − c ] k ≤ F ϵ j , ρ j k ( x ϵ j , ρ j * , c ) ≤ F ϵ j , ρ j k ( x , c ) = [ f ( x ) − c ] k .$
We know that $f ( x ϵ j , ρ j * ) − c > 0$, so $f ( x ϵ j , ρ j * ) − c ≤ f ( x ) − c$. Therefore, $f ( x * ) ≤ inf x ∈ X 0 f ( x )$ holds.
This completes the proof. ☐

## 4. Numerical Examples

In this section, we apply the Algorithm 1 to three test problems. The proposed algorithm is implemented in Matlab (R2011A, The MathWorks Inc., Natick, MA, USA).
In each example, we take $ϵ = 10 − 6$. Then, it is expected to get an $ϵ$-solution to problem $( P )$ with Algorithm 1, and the numerical results are presented in the following tables.
Example 1.
Consider the following problem ([20], Example 4.1)
$min f ( x ) = x 1 2 + x 2 2 + 2 x 3 2 + x 4 2 − 5 x 1 − 5 x 2 − 21 x 3 + 7 x 4 s . t . g 1 ( x ) = 2 x 1 2 + x 2 2 + x 3 2 + 2 x 1 + x 2 + x 4 − 5 ≤ 0 , g 2 ( x ) = x 1 2 + x 2 2 + x 3 2 + x 4 2 + x 1 − x 2 + x 3 − x 4 − 8 ≤ 0 , g 3 ( x ) = x 1 2 + 2 x 2 2 + x 3 2 + 2 x 4 2 − x 1 − x 4 − 10 ≤ 0 .$
For $k = 2 3$, let $x 1 0 = ( 0 , 0 , 0 , 0 ) , ρ 1 = 6 , β = 10 , ϵ 1 = 0 . 01 , γ = 0 . 01$ and choose $c = − 100$. The results are shown in Table 1.
For $k = 1$, let $x 1 0 = ( 5 , 5 , 5 , 5 ) , ρ 1 = 10 , β = 4 , ϵ 1 = 0 . 01 , γ = 0 . 1$ and choose $c = − 100$. The results are shown in Table 2.
The results in Table 1 and Table 2 show that the convergence of Algorithm 1 and the objective function values are almost the same. By Table 1, we obtain that an approximate optimal solution $x * = ( 0 . 166332 , 0 . 828748 , 2 . 013798 , − 0 . 959021 )$ after two iterations with function value $f ( x * ) = − 44 . 233325$. In [20], the obtained approximate optimal solution is $x * = ( 0 . 170056 , 0 . 841066 , 2 . 004907 , − 0 . 968785 )$ with function value $f ( x * ) = − 44 . 225989$. Numerical results obtained by our algorithm are slightly better than the results in [20].
Example 2.
Consider the following problem ([22], Example 3.2)
$min f ( x ) = − x 1 − x 2 s . t . g 1 ( x ) = − 2 x 1 4 + 8 x 1 3 − 8 x 1 2 + x 1 − 2 ≤ 0 , g 2 ( x ) = − 4 x 1 4 + 32 x 1 3 − 88 x 1 2 + 96 x 1 + x 2 − 36 ≤ 0 , 0 ≤ x 1 ≤ 3 , 0 ≤ x 2 ≤ 4 .$
For $k = 3 4$, let $x 1 0 = ( 3 , 1 ) , ρ 1 = 5 , β = 10 , ϵ 1 = 0 . 1 , γ = 0 . 1$, and choose $c = − 10$. The results are shown in Table 3.
For $k = 1$, let $x 1 0 = ( 0 , 1 ) , ρ 1 = 6 , β = 10 , ϵ 1 = 0 . 02 , γ = 0 . 01$ and choose $c = − 10$. The results are shown in Table 4.
The results in Table 3 and Table 4 show that the convergence of Algorithm 1 and the objective function values are almost the same. By Table 3, we obtain an approximate optimal solution is $x * = ( 2 . 112103 , 3 . 900086 )$ after 2 iterations with function value $f ( x * ) = − 6 . 012190$. In [22], the obtained global solution is $x * = ( 2 . 3295 , 3 . 1784 )$ with function value $f ( x * ) = − 5 . 5080$. Numerical results obtained by our algorithm are much better than the results in [22].
Example 3.
Consider the following problem ([26], Example 4.1)
$min f ( x ) = x 1 2 + x 2 2 − cos ( 17 x 1 ) − cos ( 17 x 2 ) + 3 s . t . g 1 ( x ) = ( x 1 − 2 ) 2 + x 2 2 − 1 . 6 2 ≤ 0 , g 2 ( x ) = x 1 2 + ( x 2 − 3 ) 2 − 2 . 7 2 ≤ 0 , 0 ≤ x 1 ≤ 2 , 0 ≤ x 2 ≤ 2 .$
For $k = 2 3$, let $x 1 0 = ( 0 , 1 ) , ρ 1 = 1 , β = 3 , ϵ 1 = 0 . 01 , γ = 0 . 01$ and choose $c = − 2$. The results are shown in Table 5.
For $k = 3 4$, let $x 1 0 = ( 0 , 0 ) , ρ 1 = 1 , β = 9 , ϵ 1 = 0 . 01 , γ = 0 . 01$, and choose $c = − 2$. The results are shown in Table 6.
The results in Table 5 and Table 6 show that the convergence of Algorithm 1 and the objective function values are almost the same. By Table 5, we obtain that an approximate optimal solution is $x * = ( 0 . 725355 , 0 . 399258 )$ after two iterations with function value $f ( x * ) = 1 . 837548$. In [26], the obtained approximate optimal solution is $x * = ( 0 . 7255 , 0 . 3993 )$ with function value $f ( x * ) = 1 . 8376$. Numerical results obtained by our algorithm are slightly better than the results in [26].

## 5. Conclusions

In this study, we have proposed a new smoothing approach to the nonsmooth penalty function and developed a corresponding algorithm to solve constrained optimization with inequality constraints. It is shown that any optimal solution of the smoothed penalty problem is shown to be an approximate optimal solution or a global solution of the original optimization problem. Furthermore, the numerical results given in Section 4 show that the Algorithm 1 has a good convergence for an approximate optimal solution.

## Acknowledgments

The authors would like to express their gratitude to anonymous referees’ detailed comments and remarks that help us improve our presentation of this paper considerably. This work is supported by grants from the National Natural Science Foundation of China (No. 11371242 and 61572099).

## Author Contributions

All authors contributed equally to this work.

## Conflicts of Interest

The authors declare no competing interests.

## References

1. Zangwill, W.I. Nonlinear programming via penalty function. Manag. Sci. 1967, 13, 334–358. [Google Scholar] [CrossRef]
2. Bazaraa, M.S.; Goode, J.J. Sufficient conditions for a globally exact penalty function without convexity. Math. Program. Study 1982, 19, 1–15. [Google Scholar]
3. Di Pillo, G.; Grippo, L. An exact penalty function method with global conergence properties for nonlinear programming problems. Math. Program. 1986, 36, 1–18. [Google Scholar] [CrossRef]
4. Di Pillo, G.; Grippo, L. Exact penalty functions in constrained optimization. SIAM J. Control. Optim. 1989, 27, 1333–1360. [Google Scholar] [CrossRef]
5. Han, S.P.; Mangasrian, O.L. Exact penalty function in nonlinear programming. Math. Program. 1979, 17, 251–269. [Google Scholar] [CrossRef]
6. Lasserre, J.B. A globally convergent algorithm for exact penalty functions. Eur. J. Oper. Res. 1981, 7, 389–395. [Google Scholar] [CrossRef]
7. Mangasarian, O.L. Sufficiency of exact penalty minimization. SIAM J. Control. Optim. 1985, 23, 30–37. [Google Scholar] [CrossRef]
8. Rosenberg, E. Exact penalty functions and stability in locally Lipschitz programming. Math. Program. 1984, 30, 340–356. [Google Scholar] [CrossRef]
9. Yu, C.J.; Teo, K.L.; Zhang, L.S.; Bai, Y.Q. A new exact penalty function method for continuous inequality constrained optimization problems. J. Indus. Mgmt. Optimiz. 2010, 6, 895–910. [Google Scholar] [CrossRef]
10. Huang, X.X.; Yang, X.Q. Convergence analysis of a class of nonlinear penalization methods for constrained optimization via first-order necessary optimality conditions. J. Optim. Theory Appl. 2003, 116, 311–332. [Google Scholar] [CrossRef]
11. Rubinov, A.M.; Glover, B.M.; Yang, X.Q. Extended Lagrange and penalty functions in continuous optimization. Optimization 1999, 46, 327–351. [Google Scholar] [CrossRef]
12. Rubinov, A.M.; Yang, X.Q.; Bagirov, A.M. Penalty functions with a small penalty parameter. Optim. Methods Softw. 2002, 17, 931–964. [Google Scholar] [CrossRef]
13. Rubinov, A.M.; Yang, X.Q. Lagrange-Type Functions in Constrained Non-Convex Optimization; Kluwer Academic Publishers: Dordrecht, Netherlands, 2003. [Google Scholar]
14. Binh, N.T.; Yan, W.L. Smoothing approximation to the k-th power nonlinear penalty function for constrained optimization problems. J. Appl. Math. Bioinform. 2015, 5, 1–19. [Google Scholar]
15. Yang, X.Q.; Meng, Z.Q.; Huang, X.X.; Pong, G.T.Y. Smoothing nonlinear penalty functions for constrained optimization. Numer. Funct. Anal. Optimiz. 2003, 24, 351–364. [Google Scholar] [CrossRef]
16. Binh, N.T. Smoothing approximation to l1 exact penalty function for constrained optimization problems. J. Appl. Math. Inform. 2015, 33, 387–399. [Google Scholar] [CrossRef]
17. Binh, N.T. Second-order smoothing approximation to l1 exact penalty function for nonlinear constrained optimization problems. Theor. Math. Appl. 2015, 5, 1–17. [Google Scholar]
18. Binh, N.T. Smoothed lower order penalty function for constrained optimization problems. IAENG Int. J. Appl. Math. 2016, 46, 76–81. [Google Scholar]
19. Chen, C.H.; Mangasarian, O.L. Smoothing methods for convex inequalities and linear complementarity problems. Math. Program. 1995, 71, 51–69. [Google Scholar] [CrossRef]
20. Meng, Z.Q.; Dang, C.Y.; Jiang, M.; Shen, R. A smoothing objective penalty function algorithm for inequality constrained optimization problems. Numer. Funct. Anal. Optimiz. 2011, 32, 806–820. [Google Scholar] [CrossRef]
21. Pinar, M.C.; Zenios, S.A. On smoothing exact penalty function for convex constrained optimization. SIAM J. Optim. 1994, 4, 486–511. [Google Scholar] [CrossRef]
22. Wu, Z.Y.; Lee, H.W.J.; Bai, F.S.; Zhang, L.S. Quadratic smoothing approximation to l1 exact penalty function in global optimization. J. Ind. Manag. Optim. 2005, 1, 533–547. [Google Scholar]
23. Xu, X.S.; Meng, Z.Q.; Sun, J.W.; Huang, L.G.; Shen, R. A second-order smooth penalty function algorithm for constrained optimization problems. Comput. Optim. Appl. 2013, 55, 155–172. [Google Scholar] [CrossRef]
24. Zenios, S.A.; Pinar, M.C.; Dembo, R.S. A smooth penalty function algorithm for network-structured problems. Eur. J. Oper. Res. 1995, 83, 220–236. [Google Scholar] [CrossRef]
25. Nocedal, J.; Wright, S.T. Numerical Optimization; Springer: New York, NY, USA, 1999. [Google Scholar]
26. Sun, X.L.; Li, D. Value-estimation function method for constrained global optimization. J. Optim. Theory Appl. 1999, 102, 385–409. [Google Scholar] [CrossRef]
Figure 1. The behavior of $q k ( t )$ and $q ϵ , ρ k ( t ) .$
Figure 1. The behavior of $q k ( t )$ and $q ϵ , ρ k ( t ) .$
Table 1. Results of Algorithm 1 with $k = 2 3 , x 1 0 = ( 0 , 0 , 0 , 0 )$ for Example 1.
Table 1. Results of Algorithm 1 with $k = 2 3 , x 1 0 = ( 0 , 0 , 0 , 0 )$ for Example 1.
j$ρ j$$ϵ j$$x ϵ j , ρ j *$$f ( x ϵ j , ρ j * )$$g 1 ( x ϵ j , ρ j * )$$g 2 ( x ϵ j , ρ j * )$$g 3 ( x ϵ j , ρ j * )$
160.01(0.214257, 0.952510, 1.934008, −1.051001)−44.2664550.0695000.044928−1.353205
2600.0001(0.166332, 0.828748, 2.013798, −0.959021)−44.233325−0.000071−0.000006−1.911178
Table 2. Results of Algorithm 1 with $k = 1 , x 1 0 = ( 5 , 5 , 5 , 5 )$ for Example 1.
Table 2. Results of Algorithm 1 with $k = 1 , x 1 0 = ( 5 , 5 , 5 , 5 )$ for Example 1.
j$ρ j$$ϵ j$$x ϵ j , ρ j *$$f ( x ϵ j , ρ j * )$$g 1 ( x ϵ j , ρ j * )$$g 2 ( x ϵ j , ρ j * )$$g 3 ( x ϵ j , ρ j * )$
1100.01(0.049915, 1.064341, 1.936277, −1.083612)−44.010514−0.0324690.064160−0.600569
2400.001(0.161485, 0.837434, 2.012060, −0.962236)−44.2334620.0000030.000003−1.870398
31600.0001(0.169837, 0.833494, 2.009512, −0.963710)−44.2338130.0000000.000000−1.892242
Table 3. Results of Algorithm 1 with $k = 3 4 , x 1 0 = ( 3 , 1 )$ for Example 2.
Table 3. Results of Algorithm 1 with $k = 3 4 , x 1 0 = ( 3 , 1 )$ for Example 2.
j$ρ j$$ϵ j$$x ϵ j , ρ j *$$f ( x ϵ j , ρ j * )$$g 1 ( x ϵ j , ρ j * )$$g 2 ( x ϵ j , ρ j * )$
150.1(2.112050, 3.900242)−6.0122920.0000390.000052
2500.01(2.112103, 3.900086)−6.012190−0.000020−0.000008
Table 4. Results of Algorithm 1 with $k = 1 , x 1 0 = ( 0 , 1 )$ for Example 2.
Table 4. Results of Algorithm 1 with $k = 1 , x 1 0 = ( 0 , 1 )$ for Example 2.
j$ρ j$$ϵ j$$x ϵ j , ρ j *$$f ( x ϵ j , ρ j * )$$g 1 ( x ϵ j , ρ j * )$$g 2 ( x ϵ j , ρ j * )$
160.02(2.111875, 3.900776)−6.0126510.0002320.000278
2600.0002(2.112763, 3.898814)−6.011577−0.000755−0.000109
Table 5. Results of Algorithm 1 with $k = 2 3 , x 1 0 = ( 0 , 1 )$ for Example 3.
Table 5. Results of Algorithm 1 with $k = 2 3 , x 1 0 = ( 0 , 1 )$ for Example 3.
j$ρ j$$ϵ j$$x ϵ j , ρ j *$$f ( x ϵ j , ρ j * )$$g 1 ( x ϵ j , ρ j * )$$g 2 ( x ϵ j , ρ j * )$
110.01(0.016363, 1.092060)2.2715142.567409−3.649496
230.0001(0.725355, 0.399258)1.837548−0.775873−0.000000
Table 6. Results of Algorithm 1 with $k = 3 4 , x 1 0 = ( 0 , 0 )$ for Example 3.
Table 6. Results of Algorithm 1 with $k = 3 4 , x 1 0 = ( 0 , 0 )$ for Example 3.
j$ρ j$$ϵ j$$x ϵ j , ρ j *$$f ( x ϵ j , ρ j * )$$g 1 ( x ϵ j , ρ j * )$$g 2 ( x ϵ j , ρ j * )$
110.01(0.387905, 1.086812)2.4486841.220019−3.479255
290.0001(0.725356, 0.399258)1.837548−0.775876−0.000000

## Share and Cite

MDPI and ACS Style

Yang, T.; Binh, N.T.; Thang, T.M.; Hoa, D.T. A New Smoothing Nonlinear Penalty Function for Constrained Optimization. Math. Comput. Appl. 2017, 22, 31. https://doi.org/10.3390/mca22020031

AMA Style

Yang T, Binh NT, Thang TM, Hoa DT. A New Smoothing Nonlinear Penalty Function for Constrained Optimization. Mathematical and Computational Applications. 2017; 22(2):31. https://doi.org/10.3390/mca22020031

Chicago/Turabian Style

Yang, Touna, Nguyen Thanh Binh, Tran Manh Thang, and Duong Thi Hoa. 2017. "A New Smoothing Nonlinear Penalty Function for Constrained Optimization" Mathematical and Computational Applications 22, no. 2: 31. https://doi.org/10.3390/mca22020031