A Lifting-Penalty Method for Quadratic Programming with a Quadratic Matrix Inequality Constraint

Abstract

In this paper, a lifting-penalty method for solving the quadratic programming with a quadratic matrix inequality constraint is proposed. Additional variables are introduced to represent the quadratic terms. The quadratic programming is reformulated as a minimization problem having a linear objective function, linear conic constraints and a quadratic equality constraint. A majorization–minimization method is used to solve instead a $l_1$ penalty reformulation of the minimization problem. The subproblems arising in the method can be solved by using the current semidefinite programming software packages. Global convergence of the method is proven under some suitable assumptions. Some examples and numerical results are given to show that the proposed method is feasible and efficient.

1. Introduction

In this paper, we consider the quadratic programming with a quadratic matrix inequality constraint of the following form:

$$\begin{array}{cc}\hfill \underset{x}{min}& x^{T}Qx+q^{T}x,\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\sum }_{i,j=1}^d{x}_i{x}_j{A}_{ij}+{\sum }_{i=1}^n{x}_i{A}_{i0}+A_{00}⪰0,\hfill \\ \hfill & \mathcal{A}x-b\in K,\hfill \end{array}$$

(1)

where q, $x\in {\mathbb{R}}^n$, Q, $A_{ij}\in {\mathbb{S}}^m$, $\mathcal{A}$ is a linear operator, the cone K is a product of semidefinite cones, second-order cones, nonnegative orthants and Euclidean spaces, ${\mathbb{S}}^m$ is the space of $m\times m$ symmetric matrices, and $A⪰0$ indicates that A is positive semidefinite.

Results of [1,2,3,4,5,6] have shown that problem (1) has important applications in many areas, including control theory and robust optimization. Moreover, it contains a wide range of optimization problems, the well-known linear and quadratic programming problems [7,8,9,10,11,12,13,14] and bilinear matrix inequality (BMI) feasibility problem [15] are its special case. The quadratic programming [16,17] and the problem of checking the solvability of a general BMI [18] are NP-hard. The problem (1) is not so easy to be solved computationally. It can be categorized as a special case of more general nonlinear semidefinite programming (NSDP). The first- and second-order optimality conditions and numerical methods for NSDP problems have been developed; see, for example, [19,20,21,22,23,24,25].

Nonsmooth optimization methods suited for eigenvalue optimization were used to compute locally optimal solutions for the BMI problem in [26]. A linearized convex-concave semidefinite programming algorithm for solving the following problem:

$$\begin{array}{cc}\hfill \underset{x}{min}& f\left(x\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& A\left(x\right)-B\left(x\right)⪯0,\hfill \\ \hfill & x\in \mathcal{X},\hfill \end{array}$$

(2)

was proposed in [1], where $f\left(x\right)$ is convex, $\mathcal{X}$ is a nonempty and closed convex set, and $A\left(x\right)$ and $B\left(x\right)$ are positive semidefinite convex. Based on the fact that the bilinear form $X^{T}Y+Y^{T}X$ can be decomposed as a difference between two positive semidefinite convex mappings, the method was applied to BMI optimization formulations of the static state/output feedback controller design problems. Following the same line of the work in [1], an iterative procedure to search a local optimum of more general nonconvex problem was developed in [27], based on a positive semidefinite convex overestimate of a positive semidefinite nonconvex matrix mapping. These local methods may not be able to obtain a global optimum.

It is easy to see that the problem (1) is equivalent to rank one constrained optimization problem, by introducing additional variable $y_{ij}$ that represents the quadratic term $x_i{x}_j$. Rank-constrained optimization problems are considered in [28,29]. In [30], polynomial-time checkable sufficient conditions, which guarantees that the semidefinite relaxations of quadratically constrained quadratic programs are exact, are given.

To attain global optimal solution, optimization approaches have been proposed. Based on the generalized Benders decomposition, a global approach is proposed for problems with BMI constraints in [31]. A slight general formulation of the BMI feasibility problem known as the following BMI eigenvalue problem (BMIEP):

$$\begin{array}{cc}\hfill \underset{x,y,\lambda }{min}& \lambda \hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \lambda I-A(x,y)⪰0,\hfill \\ \hfill & l\le x\le u,\hfill \end{array}$$

was dealt in [32], where $A(x,y)={\sum }_{ij}{x}_i{x}_j{A}_{ij}+{\sum }_i{x}_i{A}_{i0}+{\sum }_j{y}_j{A}_{j0}+A_{00}$. They proposed robust Branch-and-Cut algorithms, which improves the first implemented branch-and-bound algorithm [33] for the BMIEP. Other branch-and-bound algorithms can refer to [34,35]. Though the approaches based on the generalized Benders decomposition and branch-and-bound algorithms are global methods, it is in general impractical to solve large-scale problems. A solution method of reduction of variables is proposed for BMI problems in system and control designs in [36]. The proposed method consists of a principle of variable classification, a procedure for problem transformation and a hybrid algorithm. The proposed method can address feasibility, single-objective, and multiobjective problems with BMI constraints. However, it can fail in two circumstances.

The aim of this paper is to present a lifting-penalty method for the solution of problem (1). We reformulate a quadratic matrix inequality constraint as linear matrix inequality constraints and a single quadratic equality constraint, but instead of a rank one constraint or a quadratic matrix equality constraint. Then, we use a majorization–minimization method to solve instead a $l_1$ penalty reformulation of the minimization problem. For the fixed penalty problem, global convergence to a Karush–Kuhn–Tucker (KKT) point of the majorization–minimization method is proven. The organization of this paper is as follows. In Section 2, a lifting-penalty method, which can obtain an $ϵ$-optimal solution of the problem (1), and its global convergence are given. In Section 3, numerical results are reported to show that the proposed method is efficient.

Throughout this paper, we use the notations in Table 1.

Table 1. Mathematical symbols and their meaning.

Symbol	Meaning
${\varphi }^{\prime }\left(x\right)$	The derivative of $\varphi$ with respect to x
$▽\varphi \left(x\right)$	The gradient of $\varphi$ at x
${\mathbb{S}}_{+}^m$	The set of symmetric positive semidefinite matrices of dimension $m\times m$
${\mathbb{S}}_{++}^m$	The set of symmetric positive definite matrices of dimension $m\times m$
I	The identity matrix of appropriate dimension
$A⪰B$	$A-B$ belongs to ${\mathbb{S}}_{+}^m$
$A\succ B$	$A-B$ belongs to ${\mathbb{S}}_{++}^m$
$A•B$	The inner product of A and B
$\mathrm{tr}\left(A\right)$	The trace (sum of diagonal elements) of the matrix A
${\lambda }_{min}\left(A\right)$	The minimum eigenvalue of the matrix A
$N_{\Omega }\left(x\right)$	The normal cone to the set $\Omega$ at the point $x\in \Omega$

2. The Lifting-Penalty Method

This section concerns a lifting-penalty method for solving the problem (1). Its motivation is simple to describe: Rather than solving the original problem (1) directly, we reformulate it as a minimization problem having a linear objective function, linear conic constraints and a quadratic equality constraint by lifting x to a symmetric matrix Y, and then we use $l_1$ penalty method to solve the reformulation of (1). For a given value of the penalty parameter, the majorization–minimization (MM) method is used to solve the penalty problem. Global convergence of the MM method is proven under strict feasibility and boundedness of the original problem (1). An $ϵ$-optimal solution to problem (1) is obtained by solving the penalized problem as long as the penalty parameter is chosen appropriately.

Letting $Y=x{x}^T$, problem (1) can be stated equivalently as the following minimization problem:

$$\begin{array}{cc}\hfill \underset{x,Y}{min}& f(x,Y)=\mathrm{tr}\left(QY\right)+q^{T}x\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\sum }_{i,j=1}^d{y}_{ij}{A}_{ij}+{\sum }_{i=1}^n{x}_i{A}_{i0}+A_{00}⪰0,\hfill \\ \hfill & Y=x{x}^T,\hfill \\ \hfill & \mathcal{A}x-b\in K.\hfill \end{array}$$

(3)

By Schur complement Theorem and the equivalence between the equality $Y=x{x}^T$ and the following system

$$\begin{array}{c}Y-x{x}^T⪰0,\hfill \\ \mathrm{tr}(Y-x{x}^T)=\mathrm{tr}\left(Y\right)-x^{T}x=0,\hfill \end{array}$$

(4)

we can state problem (3) equivalently as follows:

$$\begin{array}{cc}\hfill \underset{x,Y}{min}& f(x,Y)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\sum }_{i,j=1}^d{y}_{ij}{A}_{ij}+{\sum }_{i=1}^n{x}_i{A}_{i0}+A_{00}⪰0,\hfill \\ \hfill & \left(\begin{array}{cc}1& x^T\\ x& Y\end{array}\right)⪰0,\hfill \\ \hfill & \mathrm{tr}\left(Y\right)-x^{T}x=0,\hfill \\ \hfill & \mathcal{A}x-b\in K.\hfill \end{array}$$

(5)

We assume throughout this paper that the feasible set of (1) is bounded. Hence, assume without loss of generality that $x^{T}x\le \kappa /2$. By using a penalty function similar to the $l_1$ penalty function defined in [37], we consider the following penalized problem:

$$\begin{array}{cc}\hfill \underset{x,Y}{min}& f_{\mu }(x,Y)=f(x,Y)+\mu |\mathrm{tr}\left(Y\right)-x^{T}x|=f(x,Y)+\mu (\mathrm{tr}\left(Y\right)-x^{T}x)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\sum }_{i,j=1}^d{y}_{ij}{A}_{ij}+{\sum }_{i=1}^n{x}_i{A}_{i0}+A_{00}⪰0,\hfill \\ \hfill & \left(\begin{array}{cc}1& x^T\\ x& Y\end{array}\right)⪰0,\mathrm{tr}\left(Y\right)\le \kappa ,\hfill \\ \hfill & \mathcal{A}x-b\in K,\hfill \end{array}$$

(6)

where $\mu >0$ is the penalty parameter. To simplify the notation, we denote $(b,-\kappa )$ and $K\times {\mathbb{R}}_{+}$ by $\tilde{b}$ and $\tilde{K}$, respectively, and define the linear operator $\tilde{\mathcal{A}}$: $(x,Y)\to (\mathcal{A}x,-\mathrm{tr}(Y\left)\right)$. Hence, we can rewrite constraints $\mathrm{tr}\left(Y\right)\le \kappa$ and $\mathcal{A}x-b\in K$ more compactly as $\tilde{\mathcal{A}}(x,Y)-\tilde{b}\in \tilde{K}$.

Let $\Omega$ denote the feasible set for problem (5). Unfortunately, strict feasibility of $\Omega$ (i.e., Slater’s constraint qualification) fails. Therefore, we cannot establish similar results to the exactness (Theorem 4.4 in [37]) of the $l_1$ penalty function. However, when the penalty parameter $\mu$ is large enough, it can be proven that an $ϵ$-optimal solution to problem (1) can be obtained by solving the penalized problem as long as the penalty parameter is sufficiently large.

Let ${\Omega }_c$ denote the feasible set for (6), $(x_f,Y_f)$ be a feasible solution of (5), $(x_{\mu }^{\ast },Y_{\mu }^{\ast })$ and $(x^{\ast },Y^{\ast })$ be optimal solutions of (6) and (5), respectively, and $(x_c^{\ast },Y_c^{\ast })$ be an optimal solution of the following problem:

$$\underset{x,Y}{min}\{f(x,Y):(x,Y)\in {\Omega }_c\}.$$

(7)

Theorem 1.

Let $ϵ>0$ be a given constant, and assume that $\mu \ge (f(x_f,Y_f)-f(x_c^{\ast },Y_c^{\ast }))/ϵ$, then

$$\begin{array}{c}\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast }\le ϵ,\\ f(x_{\mu }^{\ast },Y_{\mu }^{\ast })\le f(x^{\ast },Y^{\ast })-\mu (\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast })\le f(x^{\ast },Y^{\ast }).\end{array}$$

Proof. 

By noting that $(x_f,Y_f)$ is a feasible solution of (5), we have that $\mathrm{tr}\left(Y_f\right)-x_f^T{x}_f=0.$ Therefore,

$$\begin{array}{cc}\hfill f(x_f,Y_f)=& f(x_f,Y_f)+\mu (\mathrm{tr}\left(Y_f\right)-x_f^T{x}_f)=f_{\mu }(x_f,Y_f)\hfill \\ \hfill \ge & f_{\mu }(x_{\mu }^{\ast },Y_{\mu }^{\ast })=f(x_{\mu }^{\ast },Y_{\mu }^{\ast })+\mu (\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast })\hfill \\ \hfill \ge & f(x_c^{\ast },Y_c^{\ast })+\mu (\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast }).\hfill \end{array}$$

(8)

It follows from (8) and the assumption on $\mu$ that

$$\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast }\le (f(x_f,Y_f)-f(x_c^{\ast },Y_c^{\ast }))/\mu \le ϵ.$$

Because $(x^{\ast },Y^{\ast })$ is an optimal solution of (5), by replacing $(x_f,Y_f)$ in (8) by $(x^{\ast },Y^{\ast })$, we have that

$$f(x^{\ast },Y^{\ast })\ge f_{\mu }(x_{\mu }^{\ast },Y_{\mu }^{\ast })=f(x_{\mu }^{\ast },Y_{\mu }^{\ast })+\mu (\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast }).$$

Hence,

$$f(x_{\mu }^{\ast },Y_{\mu }^{\ast })\le f(x^{\ast },Y^{\ast })-\mu (\mathrm{tr}\left(Y_{\mu }^{\ast }\right)-x_{\mu }^{\ast T}{x}_{\mu }^{\ast })\le f(x^{\ast },Y^{\ast }).$$

 □

Based on Theorem 1, we give the following Algorithm 1 for finding an $ϵ$-optimal solution for problem (1).

Algorithm 1: The lifting-penalty method for solving (1)

Step 1.  Choose an initial point $(x_f,Y_f)\in \Omega$, ${\mu }_1>0$, $s_1>s_2>0$, ${ϵ}_1>0$. Set $(x^1,Y^1)=(x_f,Y_f)$ and $j=1$. Step 2.  Taking $(x^j,Y^j)$ to be the initial point, compute an optimal solution $(x^{j+1},Y^{j+1})$ of (6) with $\mu ={\mu }_j$. Step 3.  Set ${\mu }^{j+1}=\left\{\begin{array}{cc}{s}_1{\mu }_j,\hfill & \mathrm{tr}\left(Y^j\right)-x^{jT}{x}^j>0.1;\hfill \\ s_2{\mu }_j,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$ Step 4.  If $\mathrm{tr}\left(Y^{j+1}\right)-x^{(j+1)T}{x}^{j+1}\le {ϵ}_1$, stop; else, set $j=j+1$, and go to Step 2.

In Step 1, to obtain a feasible point of the problem (5), we use the MM method to solve the following problem:

$$\begin{array}{cc}\hfill \underset{x,Y}{min}& \mathrm{tr}\left(Y\right)-x^{T}x\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& (x,Y)\in {\Omega }_c.\hfill \end{array}$$

(9)

More detailed discussion on the MM method can be found in [38,39,40,41]. In Step 2, we use the MM method to solve the problem (6) with $\mu ={\mu }_j$. Let

$${\widehat{f}}_{\mu }(x,Y,z)=f(x,Y)+\mu \mathrm{tr}\left(Y\right)-\mu z^{T}z-2\mu z^T(x-z),$$

where $z\in {\mathbb{R}}^n$. From the definition of ${\widehat{f}}_{\mu }(x,Y,z)$, we have that

$$\begin{array}{c}{\widehat{f}}_{\mu }(x,Y,z)\ge f_{\mu }(x,Y),\mathrm{for}\mathrm{any}z\in {\mathbb{R}}^n,\hfill \\ {\widehat{f}}_{\mu }(x,Y,x)=f_{\mu }(x,Y).\hfill \end{array}$$

That is, ${\widehat{f}}_{\mu }(x,Y,z)$ is the majorization function of $f_{\mu }(x,Y)$ at z. Therefore, we can apply the following Algorithm 2 to the problem (6) for $\mu ={\mu }_j$.

Algorithm 2: The MM method for (6) with a fixed value of the parameter $\mu$

Step 1. Choose an initial point $(x^1,Y^1)\in {\Omega }_c$, ${ϵ}_2>0$; Set $k=1$.

Step 2. Taking $(x^k,Y^k)$ to be the initial point, compute an optimal solution $(x^{k+1},Y^{k+1})$ of thefollowing problem: $$\begin{array}{cc}\hfill \underset{x,Y}{min}& {\widehat{f}}_{\mu }(x,Y,x_k)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& (x,Y)\in {\Omega }_c.\hfill \end{array}$$ (10)

Step 3. If $\parallel (x^{k+1},Y^{k+1})-(x^k,Y^k)\parallel \le {ϵ}_2$, stop; else, set $k=k+1$, and go to Step 2.

In Algorithm 2, linear conic programming problem (10) can be solved by software packages SeDuMi [42], SDPT3 [43] or SDPNAL [44]. The MM method has global convergence properties, which states that the cluster points of $\left\{(x^k,Y^k)\right\}$ are all KKT points of (6) for any given $\mu$. To prove it, we need a constraint qualification. We first give the following Lemma.

Lemma 1.

If strict feasibility of the problem (1) holds, then strict feasibility of the problem (6) is also ensured.

Proof. 

If the problem (1) is strictly feasible, then there exists a point $\tilde{x}\in {\mathbb{R}}^n$ such that

$$\sum _{i,j=1}^d{\tilde{x}}_i{\tilde{x}}_j{A}_{ij}+\sum _{i=1}^n{\tilde{x}}_i{A}_{i0}+A_{00}\succ 0\mathrm{and}\mathcal{A}\tilde{x}-b\in \mathrm{ri}K,$$

(11)

where $\mathrm{ri}K$ is the relative interior of the cone K.

Taking $\overline{Y}=\tilde{x}{\tilde{x}}^T$, we know that $(\tilde{x},\overline{Y})$ satisfies

$$\begin{array}{c}G(\tilde{x},\overline{Y})={\sum }_{i,j=1}^d{\overline{y}}_{ij}{A}_{ij}+{\sum }_{i=1}^n{\tilde{x}}_i{A}_{i0}+A_{00}\succ 0,\hfill \\ \tilde{\mathcal{A}}(\tilde{x},\overline{Y})-\tilde{b}\in \mathrm{ri}\tilde{K},\left(\begin{array}{cc}1& {\tilde{x}}^T\\ \tilde{x}& \overline{Y}\end{array}\right)⪰0,\left(\begin{array}{cc}1& {\tilde{x}}^T\\ \tilde{x}& \overline{Y}\end{array}\right)\nsucc 0.\hfill \end{array}$$

(12)

Denote the vector of all eigenvalues of $Z\in {\mathbb{S}}^m$ arranged in non-increasing order by $\lambda \left(Z\right)$. By the definition of $G(\tilde{x},\overline{Y})$ and Mirsky’s Theorem ([45], Cor 4.12), we have that for any

$$\epsilon \in (0,{\lambda }_{min}\left(G(\tilde{x},\overline{Y})\right)/2\parallel \lambda (\sum _{i=1}^d{A}_{ii}){\parallel }_{\infty }),$$

$$\begin{array}{ll}& \parallel \lambda \left(G(\tilde{x},\overline{Y}+\epsilon I)\right)-\lambda \left(G(\tilde{x},\overline{Y})\right){\parallel }_{\infty }={\parallel \lambda (G(\tilde{x},\overline{Y})+\epsilon {\sum }_{i=1}^d{A}_{ii})-\lambda \left(G(\tilde{x},\overline{Y})\right)\parallel }_{\infty }\\ \le & \parallel \lambda (\epsilon {\sum }_{i=1}^d{A}_{ii}){\parallel }_{\infty }=\epsilon {\parallel \lambda ({\sum }_{i=1}^d{A}_{ii})\parallel }_{\infty }\le {\lambda }_{min}\left(G(\tilde{x},\overline{Y})\right)/2.\end{array}$$

Therefore, $\lambda \left(G(\tilde{x},\overline{Y}+\epsilon I)\right)>0$. Take $\tilde{Y}=\overline{Y}+\epsilon I$, we have that

$$G(\tilde{x},\tilde{Y})\succ 0,\tilde{Y}-\tilde{x}{\tilde{x}}^T=\epsilon I\succ 0.$$

(13)

From Schur complement Theorem and (13), we know that $(\tilde{x},\tilde{Y})$ is a strictly feasible point for (6). □

Lemma 2.

If the feasible set of the problem (1) is nonempty and bounded, then, for any given $\mu >0$, ${\Omega }_c$ and the solution sets of the problems (6) and (10) are nonempty and compact.

Proof. 

Because the feasible set of the problem (1) is a nonempty subset of ${\Omega }_c$, ${\Omega }_c$ is nonempty. It is trivial to show that ${\Omega }_c$ is closed. We will prove ${\Omega }_c$ is bounded. For any $(x,Y)\in {\Omega }_c$, $Y⪰0$ together with $\mathrm{tr}\left(Y\right)\le \kappa$ implies that Y is bounded, and $x^{T}x\le \mathrm{tr}\left(Y\right)\le \kappa$ implies that x is bounded. As a result, ${\Omega }_c$ is nonempty and compact.

From the continuity of $f_{\mu }(·)$ and ${\widehat{f}}_{\mu }(·)$, we know that the solution sets of the problems (6) and (10) are also nonempty and compact. □

By introducing the product topology on ${\mathbb{R}}^n\times {\mathbb{S}}^m$ with the induced inner product

$$⟨(x_1,Y_1),(x_2,Y_2)⟩=⟨x_1,x_2⟩+⟨Y_1,Y_2⟩=x_1^T{x}_2+\mathrm{tr}\left(Y_1{Y}_2\right),$$

(14)

for any fixed z and any direction $(d,D)\in {\mathbb{R}}^n\times {\mathbb{S}}^m$, we calculate the derivative of ${\widehat{f}}_{\mu }(x,Y,z)$ with respect to $(x,Y)$ as follows:

$${\widehat{f}}_{\mu }^{\prime }(x,Y,z)(d,D)=⟨q-2\mu z,d⟩+⟨Q+\mu I,D⟩.$$

(15)

The gradient $▽{\widehat{f}}_{\mu }(x,Y,z)$ of ${\widehat{f}}_{\mu }(x,Y,z)$ with respect to $(x,Y)$, therefore, may be interpreted as the pair of matrices:

$$▽{\widehat{f}}_{\mu }(x,Y,z)=(q-2\mu z,Q+\mu I).$$

Theorem 2.

Suppose that Slater’s constraint qualification for problem (1) holds, and the feasible set of (1) is bounded. For any given $\mu >0$, let $\left\{(x^k,Y^k)\right\}$ be the sequence generated by Algorithm 2, then ${lim}_{k\to \infty }{f}_{\mu }(x^k,Y^k)={inf}_k{f}_{\mu }(x^k,Y^k)$, and any cluster point of $\left\{(x^k,Y^k)\right\}$ is a KKT point of (6).

Proof. 

From the boundedness of the feasible set of problem (1) and Lemma 2, we know that problem (10) has a nonempty and compact solution set and there exists a point $(x^{k+1},Y^{k+1})\in arg{min}_{(x,Y)\in {\Omega }_c}\{{\widehat{f}}_{\mu }(x,Y,x^k)\}.$

By noting that $f_{\mu }(·)$ is continuous and $\left\{(x^k,Y^k)\right\}$ is bounded, we know that ${inf}_k\left\{f_{\mu }(x^k,Y^k)\right\}$ is finite. It follows from definitions of $f_{\mu }(x,Y)$ and ${\widehat{f}}_{\mu }(x,Y,z)$ that

$$f_{\mu }(x^{k+1},Y^{k+1})\le {\widehat{f}}_{\mu }(x^{k+1},Y^{k+1},x^k)\le {\widehat{f}}_{\mu }(x^k,Y^k,x^k)=f_{\mu }(x^k,Y^k).$$

That is, $\left\{f_{\mu }(x^k,Y^k)\right\}$ is a monotonically decreasing sequence. Hence,

$$\underset{k\to \infty }{lim}{f}_{\mu }(x^k,Y^k)=\underset{k}{inf}{f}_{\mu }(x^k,Y^k).$$

Because $(x^{k+1},Y^{k+1})$ is an optimal solution of (10), we have that

$$0\in \left(q-2\mu x^k,Q+\mu I\right)+N_{{\Omega }_c}(x^{k+1},Y^{k+1}).$$

(16)

We first consider the case $(x^{k+1},Y^{k+1})=(x^k,Y^k)$ for some integer $k\ge 0$. When $(x^{k+1},Y^{k+1})=(x^k,Y^k)$, it follows from (16) that

$$0\in \left(q-2\mu x^{k+1},Q+\mu I\right)+N_{{\Omega }_c}(x^{k+1},Y^{k+1}).$$

(17)

Denote $G_1(x,Y)={\sum }_{i,j=1}^d{y}_{ij}{A}_{ij}+{\sum }_{i=1}^n{x}_i{A}_{i0}+A_{00},$ $G_2(x,Y)=\left(\begin{array}{cc}1& x^T\\ x& Y\end{array}\right)$, $G_3(x,Y)=\tilde{\mathcal{A}}(x,Y)-\tilde{b}$ and $G_i^k=G_i(x^k,Y^k)$. From Lemma 1, we know that Slater’s constraint qualification for (6) holds. This, together with convexity, implies that

$$N_{{\Omega }_c}(x^{k+1},Y^{k+1})={{\left(G_1^{k+1}\right)}^{\prime }}^{\ast }{N}_{{\mathbb{S}}_{+}^m}\left(G_1^{k+1}\right)+{{\left(G_2^{k+1}\right)}^{\prime }}^{\ast }{N}_{{\mathbb{S}}_{+}^{n+1}}\left(G_2^{k+1}\right)+{\tilde{\mathcal{A}}}^{\ast }{N}_{\tilde{K}}\left(G_3^{k+1}\right),$$

(18)

where ${{\left(G_i^{k+1}\right)}^{\prime }}^{\ast }$ and ${\tilde{\mathcal{A}}}^{\ast }$ are the adjoint operators of ${\left(G_i^{k+1}\right)}^{\prime }$ and $\tilde{\mathcal{A}}$, respectively. It follows from (17) and (18) that $(x^{k+1},Y^{k+1})$ is a KKT point of problem (6).

Next, we assume that $(x^{k+1},Y^{k+1})\ne (x^k,Y^k)$ for all $k\ge 0$. Because $\left\{(x^k,Y^k)\right\}$ is bounded, we know that $\left\{(x^k,Y^k)\right\}$ has cluster points and any cluster point $(\overline{x},\overline{Y})\in {\Omega }_c$. By the equality ${lim}_{k\to \infty }{f}_{\mu }(x^k,Y^k)={inf}_k{f}_{\mu }(x^k,Y^k)$, we have that $f_{\mu }(\overline{x},\overline{Y})={inf}_k{f}_{\mu }(x^k,Y^k)$. Now, we prove that $(\overline{x},\overline{Y})$ is a KKT point of (6). For any $k\ge 0$, we have that

$$\begin{array}{c}{f}_{\mu }(x^{k+1},Y^{k+1})-f_{\mu }(x^k,Y^k)\hfill \\ =\mathrm{tr}\left((Q+\mu I)(Y^{k+1}-Y^k)\right)+q^T(x^{k+1}-x^k)-\mu {\left(x^{k+1}\right)}^T{x}^{k+1}+\mu x^{kT}{x}^k\hfill \\ =\mathrm{tr}\left((Q+\mu I)(Y^{k+1}-Y^k)\right)+{(q-2\mu x^k)}^T(x^{k+1}-x^k)+2\mu x^{kT}(x^{k+1}-x^k)-\mu {\left(x^{k+1}\right)}^T{x}^{k+1}+\mu x^{kT}{x}^k\hfill \\ =⟨(q-2\mu x^k,Q+\mu I),(x^{k+1}-x^k,Y^{k+1}-Y^k)⟩-\mu {(x^{k+1}-x^k)}^T(x^{k+1}-x^k).\hfill \end{array}$$

(19)

It follows from (16) that $-(q-2\mu x^k,Q+\mu I)\in N_{{\Omega }_c}(x^{k+1},Y^{k+1})$, which implies that

$$⟨(q-2\mu x^k,Q+\mu I),(x^{k+1}-x^k,Y^{k+1}-Y^k)⟩=⟨-(q-2\mu x^k,Q+\mu I),(x^k-x^{k+1},Y^k-Y^{k+1})⟩\le 0.$$

(20)

From (19) and (20), we have that

$$f_{\mu }(x^{k+1},Y^{k+1})-f_{\mu }(x^k,Y^k)\le -\mu {\parallel x^{k+1}-x^k\parallel }^2,$$

(21)

which implies that

$$\mu \underset{l\to \infty }{lim}\sum _{k=1}^l{\parallel x^{k+1}-x^k\parallel }^2\le f_{\mu }(x^1,Y^1)-\underset{l\to \infty }{lim}{f}_{\mu }(x^{l+1},Y^{l+1})<+\infty .$$

Hence, ${lim}_{k\to \infty }(x^{k+1}-x^k)=0.$ Let $k_j$ satisfy that $x^{k_j}\to \overline{x}$ and $Y^{k_j}\to \overline{Y}$, then

$$\underset{j\to \infty }{lim}{x}^{k_j}=\underset{j\to \infty }{lim}{x}^{k_j-1}=\overline{x}.$$

(22)

Because $(x^{k_j},Y^{k_j})$ is an optimal solution, there exists $D^{k_j}\in N_{{\Omega }_c}(x^{k_j},Y^{k_j})$ such that

$$D^{k_j}=-(q-2\mu x^{k_j-1},Q+\mu I)$$

for any $k_j\ge 1$. Hence, for any $(\tilde{x},\tilde{Y})\in {\Omega }_c$,

$$⟨-(q-2\mu x^{k_j-1},Q+\mu I),(\tilde{x}-x^{k_j},\tilde{Y}-Y^{k_j})⟩=⟨D^{k_j},(\tilde{x}-x^{k_j},\tilde{Y}-Y^{k_j})⟩\le 0,$$

which, from the convergence of the subsequence $\left\{(x^{k_j},Y^{k_j})\right\}$ and (22), gives rise to

$$⟨-(q-2\mu \overline{x},Q+\mu I),(\tilde{x}-\overline{x},\tilde{Y}-\overline{Y})⟩\le 0,$$

which implies that $-(q-2\mu \overline{x},Q+\mu I)\in N_{{\Omega }_c}(\overline{x},\overline{Y})$. That is, $0\in ▽f_{\mu }(\overline{x},\overline{Y})+N_{{\Omega }_c}(\overline{x},\overline{Y})$, which, together with Slater’s constraint qualification, implies that $(\overline{x},\overline{Y})$ is a KKT point of problem (6). □

3. Numerical Experiments

In this section, the examples and some preliminary numerical results taken by the lifting-penalty method (LPM) and a modified augmented Lagrangian method (MALM) [46] are given below.

All numerical experiments are done by running MATLAB${}^{\circledR }$ 2016 on a notebook PC Intel${}^{\circledR }$ Core${}^{\mathrm{TM}}$ i7-4810MQ CPU with 2.8 GHz and 16 GB RAM. The linear conic optimization problems in our method are solved by a SDPT3 solver. The optimization subproblems in MALM are solved by the subroutine fmincon. The parameters in the algorithms are set as follows:

$$\begin{array}{cc}\hfill \mathrm{LPM}:& {\mu }_1=0.25,s_1=4,s_2=1.4,{ϵ}_1={ϵ}_2={10}^{-6};\hfill \\ \hfill \mathrm{MALM}:& \tau =0.25,\gamma =10,c_1=1.\hfill \end{array}$$

Example 1

(HE1 in D in Section V, [3]). Find matrix K such that $A+BKC$ is Hurwitz, i.e., eigenvalues of matrix $A+BKC$ all belong to the left half plane $\mathcal{D}=\{t\in \mathbb{C}:t+t^{\ast }<0\}$ of the complex plane, where $t^{\ast }$ is the conjugate of t,

$$A=\left(\begin{array}{cccc}-0.0366& 0.0271& 0.0188& -0.4555\\ 0.0482& -1.01& 0.0024& -4.0208\\ 0.1002& 0.3681& -0.7070& 1.42\\ 0& 0& 1& 0\end{array}\right),B=\left(\begin{array}{cc}0.4422& 0.1761\\ 3.5446& -7.5922\\ -5.52& 4.49\\ 0& 0\end{array}\right),$$

$$C=\left(\begin{array}{cccc}0& 1& 0& 0\end{array}\right).$$

The problem amounts to solving a nonconvex feasibility problem $S\left(K\right)\succ 0$ (see [3,47,48]). We solved it by solving the following non-strict optimization problem:

$$\underset{K,\lambda }{min}\lambda \mathrm{s}.\mathrm{t}.\lambda I+S\left(K\right)⪰0,-50\le K_{ij}\le 50,-1\le \lambda \le 0.$$

From Figure 2 in [3], we know that the problem is nonconvex. For the starting point $K^{\left(0\right)}={(0,0)}^T$, our method ended with the final solution $K={(-0.4293,0.5514)}^T$.

Example 2.

Let $x\in {\mathbb{R}}^n$, $\lambda \in \mathbb{R}$,

$$\begin{array}{cc}\hfill \underset{x,\lambda }{max}& \lambda ,\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& -\lambda I+{\sum }_{i,j=1}^d{x}_i{x}_j{A}_{ij}+{\sum }_{i=1}^n{x}_i{A}_{i0}+A_{00}⪰0,\hfill \\ & x_i\in [-1,1],i=1,\dots ,n,\hfill \end{array}$$

where $A_{ij}\in {\mathbb{S}}^m$ is generated by the Matlab functionsprandn(m,m,r). We symmetrized the matrices by copying the upper triangular part to the lower one after creation.

In our experiment, 20 problem instances are randomly generated for each value of $(n,m,d)$, and the additional variable Y is a $d\times d$ symmetric matrix, $r=0.2$. Table 2 lists n, m, d and the average CPU time in seconds, respectively.

Table 2. Numerical results of Example 2.

			Time in Seconds			Time in Seconds			Time in Seconds
			LPM	MALM		LPM	MALM		LPM	MALM
30	10	6	1.71	4.07	10	4.62	6.86	15	5.97	11.91
40			2.19	6.22		2.99	8.47		11.28	13.23
50			2.10	6.46		9.12	9.42		12.29	13.91
30	15	6	1.91	4.42	10	3.46	6.46	15	13.90	10.04
40			1.92	6.75		8.31	9.36		12.32	14.89
50			2.25	7.75		6.78	10.32		13.63	17.07
30	20	6	1.85	12.93	10	2.42	17.79	15	27.19	34.87
40			2.12	18.71		3.32	28.41		24.83	43.37
50			2.51	23.90		9.02	35.78		28.70	51.18
30	30	6	2.21	14.65	10	2.89	23.25	15	26.64	39.39
40			2.47	20.68		3.14	30.03		42.43	46.73
50			2.57	28.45		5.00	37.70		46.62	55.09

From the results listed in Table 2, we find that our method requires less time than the LPM for most instances. Moreover, consuming time seems to be more sensitive to d, compared with n and m. Preliminary computational experiences show that our method is competitive with the LPM.

4. Conclusions

In this paper, a lifting-penalty method for solving a quadratic optimization problem involving a quadratic matrix inequality constraint is introduced. By introducing additional variables, we reformulate a quadratic matrix inequality constraint as linear matrix inequality constraints and a single quadratic equality constraint but instead of a rank one constraint or a quadratic matrix equality constraint. Its global convergence result has been given under mild assumptions. Then, the method was applied to a feasibility problem and a problem of maximizing the smallest eigenvalue of a symmetric matrix. The numerical results show that the proposed method is competitive with the LPM. Note, however, that linear conic optimization subproblems arising in our method have the same feasible set, so the development of an efficient method for solving a family of linear conic optimization problems with the same feasible set is our future work.