The Generalized Trust-Region Sub-Problem with Additional Linear Inequality Constraints—Two Convex Quadratic Relaxations and Strong Duality

: In this paper, we study the problem of minimizing a general quadratic function subject to a quadratic inequality constraint with a ﬁxed number of additional linear inequality constraints. Under a regularity condition, we ﬁrst introduce two convex quadratic relaxations (CQRs), under two different conditions, that are minimizing a linear objective function over two convex quadratic constraints with additional linear inequality constraints. Then, we discuss cases where the CQRs return the optimal solution of the problem, revealing new conditions under which the underlying problem admits strong Lagrangian duality and enjoys exact semideﬁnite optimization relaxation. Finally, under the given sufﬁcient conditions, we present necessary and sufﬁcient conditions for global optimality of the problem and obtain a form of S-lemma for a system of two quadratic and a ﬁxed number of linear inequalities.


Introduction
Consider the following generalized trust-region subproblem with additional linear inequality constraints: where A, B ∈ R n×n are symmetric matrices but not necessarily positive semidefinite, a, b, c i ∈ R n and β, d i ∈ R, i = 1, ..., m. Model problem (1) arises for instance in nonlinear optimization problems with linear inequality constraints when the trust-region methods are applied to solve [1] or in general nonlinear programming for computing search directions when sequential quadratic programming methods are employed [2] and polyhedral data uncertainty [3,4] or in robust optimization problems under matrix norm [5].
Although some special cases of problem (1) are polynomially solvable, it is a difficult problem in general. When B = I, b = 0, β < 0 and (c i , d i ) = (0, 0), i = 1, ..., m, problem (1) is known as the trust-region subproblem (TRS) which is fundamental in the trust-region methods for unconstrained optimization [1]. Several efficient algorithms have been introduced for TRS in the literature [6][7][8][9][10]. Specifically, TRS has many nice properties such as exact semidefinite optimization (SDO) relaxation and strong duality [7,11]. In general, these important features do not hold for the following extended trust-region subproblem (eTRS) with a fixed number of linear inequality constraints, even for eTRS with one linear inequality constraint [12][13][14]: Jeyakumar and Li [15] proved that the SDO-relaxation of eTRS is exact whenever the dimension condition dim Null(A − λ min (A)I) ≥ s + 1, is satisfied where s = dim span{c 1 , ..., c m } and λ min (A) denotes the smallest eigenvalue of A. They also derived necessary and sufficient optimality conditions for eTRS under the Slater condition and the dimension condition (2). Later, in Reference [16], the authors obtained the exactness of SDO-relaxation of eTRS under the following condition which is more general than the dimension condition of Jeyakumar and Li. Most recently, in Reference [17], the authors have studied variants of TRS having additional conic constraints, where A ∈ R n×n is a symmetric matrix, H ∈ R m×n , h ∈ R m and K ⊆ R m is a closed convex cone. Assuming λ min (A) < 0, they introduced the following convex relaxation for (4): and showed that this convex relaxation is exact if there exists nonzero z ∈ Null(A − λ min (A)I) such that Hz ∈ K and a T z ≤ 0. When (c i , d i ) = (0, 0), i = 1, ..., m, problem (1) reduces to the generalized trust-region subproblem (GTRS) [18]. The GTRS has been well studied in the literature and several methods have been proposed to solve it under various assumptions [11,[18][19][20][21][22][23][24][25]. It has strong duality and exact SDO-relaxation under the Slater condition [11,22]. Under the assumption that the Hessian of quadratic functions are simultaneously diagonalizable, Ben-Tal and den Hertog [26] proved that GTRS admits a second order cone programming (SOCP) reformulation. Under the same assumption, they generalized this result to GTRS with two quadratic inequality constraints. They showed that under certain additional conditions, the optimal solution of the original problem can be recovered from the optimal solution of the SOCP relaxation [26]. However, in Reference [26], it has been illustrated that even in the simplest case where B 0 and the second constraint is linear (eTRS with m = 1), the SOCP relaxation may not be exact. After that, Locatelli [27] extended the SOCP relaxation to eTRS and gave conditions under which the SOCP relaxation is tight. Moreover, in Reference [28], it has been shown that the SOCP relaxation of eTRS and its SDO-relaxation are equivalent. Also, through this equivalence, new conditions are introduced that ensure the exactness of the SDO-relaxation of eTRS and are more general than the condition (3) [28]. Most recently, in Reference [29], the authors have proposed a new convex quadratic reformulation for GTRS that minimizes a linear objective function subject to two convex quadratic constraints. They also have shown that the optimal solution of GTRS can be recovered from the optimal solution of the new reformulation.
The main contributions of this paper are as follows: (i) Under a regularity condition, we present two convex quadratic relaxations (CQRs) under two different conditions for problem (1) that minimize a linear objective function subject to two convex quadratic constraints with a fixed number of additional linear inequality constraints. Our CQRs are inspired by the one proposed for GTRS in Reference [29]. Then we derive sufficient conditions under which problem (1) is equivalent to exactly one of the CQRs and the optimal solution of (1) can be recovered from an optimal solution of the CQRs. These sufficient conditions are easy to verify and involve only one (any) optimal solution of CQRs. We also show that under these conditions the attractive features of GTRS such as strong Lagrangian duality and exact SDO-relaxation hold for (1). It should be noted that in the case of GTRS, the CQRs reduce to the ones proposed in Reference [29]. The CQRs are always exact for GTRS but in the presence of linear constraints, they are not exact in general. (ii) Exploiting the results in (i), we also derive sufficient conditions that are expressed in terms of the data of the model problem (1) for exactness of the CQRs, strong Lagrangian duality and consequently for tightness of the SDO-relaxation. In the case of eTRS, these sufficient conditions reduce to the one presented in Reference [17] that is the existing best results in the literature. As a consequence, we present necessary and sufficient conditions for global optimality of problem (P) under the new condition together with the Slater condition. We also obtain a form of S-lemma for the system of two quadratic and a fixed number of linear inequalities. (iii) The sufficient conditions in (i) and (ii) ensure the exactness of the CQRs and the SDO-relaxation of problem (1). It is worth noting that solving large-scale semidefinite problems is still an intractable task. In contrast, the CQRs are significantly more tractable than SDOs, and advanced commercial software is available to solve them [30].
The rest of the paper is organized as follows-in Section 2, we introduce the CQRs and discuss when and how one can obtain an optimal solution of problem (1) from an optimal solution of the CQRs, revealing new sufficient conditions for strong duality of problem (1). In Section 3, we use the results in Section 2 to derive sufficient conditions based on the data of the original problem for exactness of the CQRs, strong Lagrangian duality and exact SDO-relaxation. We also present necessary and sufficient conditions for global optimality of problem (1) and an application of strong duality to S-lemma. Notation 1. Throughout this paper, for a symmetric matrix A, A 0(A 0) denotes that A is positive definite (positive semidefinite). Moreover, Null(A) and Rank(A) denote its Null space and Rank. Finally, A • B := trac(AB) is the usual matrix inner product of two symmetric matrices A and B.

Convex Quadratic Relaxation, Global Minimization and Strong Duality
In this section, following the idea of Reference [29], first we present two new convex quadratic relaxations for problem (1) under two different conditions. Then we discuss cases where problem (1) is equivalent to one of the CQRs and its global optimal solution can be obtained from an optimal solution of the CQRs. This equivalence reveals new conditions under which problem (1) enjoys strong Lagrangian duality. We start by considering the following assumptions Assumption 1. There existsλ ≥ 0 such that A +λB 0. Assumption 2. The Slater condition holds for problem (1), that is, there existsx with q 2 (x) < 0, c T ix ≤ d i for i = 1, ..., m.
Assumptions 1 and 2 ensure that problem (1) is solvable as proved in the following lemma. Lemma 1. Suppose that Assumptions 1 and 2 hold. Then problem (1) has an optimal solution, that is, the infimum in (1) is always attainable.

Proof. See Appendix A.1.
Assumption 1 implies that matrices A and B are simultaneously diagonalizable by congruence [31], that is, there exists an invertible matrix S and diagonal matrices D = diag(α 1 , · · · , α n ) and E = diag(e 1 , · · · , e n ) such that S T AS = D and S T BS = E. Define I PSD := {λ ≥ 0|A + λB 0}. By Assumption 1, I PSD = ∅. It is easy to see that We have two cases for the set I PSD as follows, whereλ 1 = max{0, λ 1 }: In the sequel, we introduce two new CQRs (6) and (7) corresponding to Condition 1 and Condition 2, respectively, by defining In the case where both A and B are positive semidefinite, problem (7) is equal to problem (1) that is already a convex quadratic problem. Hence, from now on, we suppose that at least one of q 1 (x) and q 2 (x) is nonconvex. As it is shown in the proof of Lemma 1, problem (1) is equivalent to its epigraph as follows: Problems (6) and (7) are both convex. Under Condition 1, problem (6) is a convex relaxation of problem (8) and hence p * 1 ≤ p * . To see this, let x be a feasible solution of problem (8). Since q 2 (x) ≤ 0 andλ 1 , λ 2 ≥ 0, then Therefore, the feasible region of problem (6) contains that of problem (8) and since the two problems have the same objective function, then p * 1 ≤ p * . Next, suppose that Condition 2 holds. Problems (7) and (1) have the same feasible region and since h 1 (x) ≤ q 1 (x), we have p * 2 ≤ p * . The following lemma states that problems (6) and (7) are bounded from below and their optimal values are attained.

Lemma 2.
Under Assumptions 1 and 2, problems (6) and (7) are bounded from below and their optimal values are attained.
In the case of GTRS, the CQRs (6) and (7) reduce to the ones introduced in Reference [29]. Under Assumptions 1 and 2, the CQRs (6) and (7) are always exact for GTRS [29] while in the presence of linear constraints, they are not exact in general as illustrated in the following example. Example 1. Consider the following GTRS: The CQR relaxation of (9) is It is easy to see that the CQR (10) is exact, that is, p * = p * 2 = −1. If we add the linear constraint x ≤ 1 2 to problem (9), then the resulting CQR is not exact.
In the following theorems, we state cases where the nonconvex problem (1) is equivalent to one of the CQR (6) or (7), that is, p * = p * 1 or p * = p * 2 and the optimal solution of (1) can be obtained from an optimal solution of the CQRs. In particular, this equivalence results in sufficient conditions for the strong Lagrangian duality for problem (1). Theorem 1. Suppose that Assumptions 1, 2 and Condition 1 hold, (x * , t * ) is an optimal solution of problem (6) and one of the following holds: Then problem (1) is equivalent to (6), that is, p * = p * 1 , strong duality holds for (1) and its Lagrangian dual problem is solvable. Also, in case (1) or (3), x * is an optimal solution to (1) and in case (2) or (4),x * := x * + α * z is an optimal solution for (1) where α * is the positive root of the quadratic equation q 2 (x * + αz) = 0.

Theorem 2.
Suppose that Assumptions 1, 2 and Condition 2 hold, x * is an optimal solution of problem (7) and one of the following holds: q 2 (x * ) < 0 and there exists nonzero z ∈ Null(A +λ 1 B) such that (a +λ 1 b) T z = 0 and c T i z ≤ 0 for i = 1, ..., m.

New Conditions for Strong Duality and Exact SDO-Relaxation
In the previous section, we derived sufficient conditions for exactness of the CQRs and strong Lagrangian duality of problem (1) based on an optimal solution of CQRs (6) and (7) (see Theorems 1 and 2). Here we exploit the results in Theorems 1 and 2 to derive new sufficient conditions that are expressed in terms of the data of the original problem for the exactness of the CQRs, strong Lagrangian duality and tightness of the SDO-relaxation of problem (P). Recall that we have assumed at least one of A and B is not positive semidefinite. Otherwise, by Assumption 2, problem (1) is a convex optimization problem that satisfies the Slater condition and hence, it has strong duality and exact SDO-relaxation.
The so-called SDO-relaxation of (1) is The dual of (11) is which is also the Lagrangian dual problem of (1). Note that by Assumption 1, problem (12) is strictly feasible and hence, d * = p * r . This together with the fact that d * ≤ p * r ≤ p * imply that the strong duality holds for (1) if and only if the SDO-relaxation for (1) is exact.

Remark 1. Condition 3 can be verified easily by solving a linear programming problem. To verify condition
it is sufficient to solve the following linear programming problem: c T i z ≤ 0, i = 1, ..., m. (14) is either unbounded or has multiple optimal solutions. If problem (14) is unbounded, then obviously condition (13) holds. If problem (14) is bounded, thenp = 0. In this case, since z = 0 is an optimal solution of (14), condition (13) holds if and only if (14) has multiple optimal solutions. The same discussion holds when λ 2 is replaced by λ 1 orλ 1 .

Remark 2.
It is worth noting that the sufficient conditions for strong duality of problem (1) established in Theorems 1 and 2 are more general than Condition 3. The following is an example where Condition 3 does not hold, while the condition in Theorem 2 does.

Example 2.
Consider the following one-dimensional problem: The CQR relaxation of (15) is The optimal solution of (16) is x * = 1 and q 2 (x * ) = 0. Hence, the sufficient condition in Theorem 2 is fulfilled. However, Condition 3 is not fulfilled. Moreover, it is easy to verify that strong duality holds for problem (15) and the SDO-relaxation and the CQR (16) are exact.

Remark 3.
For the case of eTRS which is a special case of (4), the convex relaxation (5) is equal to problem (7). Also, it is easy to see that in the case of eTRS, Condition 3 reduces to the condition in Reference [17].
We now present necessary and sufficient conditions for global optimality of (1) whenever Condition 3 and Assumptions 1 and 2 are satisfied. (1), suppose that Condition 3, Assumptions 1 and 2 hold. Let x * be a feasible solution of (1). Then x * is a global minimizer of (1) if and only if there exist nonnegative multipliers λ * i , i = 0, ..., m such that the following conditions hold

Corollary 1. For problem
Proof. Let x * be a global minimizer of (1). Recall that by Lemma 3, strong duality holds for problem (1). Suppose that (λ * 0 , λ * 1 , ..., λ * m ) is an optimal solution of Lagrangian dual of (1) and d * denotes the dual optimal value. We have where the last inequality follows from λ * i ≥ 0, i = 0, ..., m and feasibility of x * . We conclude that the two inequalities in this chain hold with equality. Since the inequality in the second line is an equality, we conclude that x * is a minimizer of the minimization problem in the first line. This gives relations (17) and (20). Moreover, it follows from the last line that λ * which with the fact that each term in this sum is nonpositive, we obtain (18) and (19). Conversely, suppose that x * satisfies (17) to (20). We have the following chain of inequalities: where the first inequality comes from weak duality property, the second equality follows from (17) and (20), the third equality follows from (18) and (19) and the last inequality comes from the fact that x * is a feasible solution of (1). Therefore, q 1 (x * ) = p * and so x * solves (1).
As a consequence of our strong duality result, we obtain the following form of the celebrated S-lemma [32] for a system of two quadratic and a fixed number of linear inequalities. Lemma 4. Let A, B ∈ R n×n , a, b, c i ∈ R n , β, γ, d i ∈ R, i = 1, ..., m. Suppose that Assumptions 1, 2 and Condition 3 are satisfied. Then the following statements are equivalent: Proof. It is easy to see that (2) ⇒ (1). So, in the sequel, we show that (1) ⇒ (2). To see this, let (1) holds. Then, By Lemma 3, we have strong duality for the above problem. Suppose that (λ 0 , λ 1 , ..., λ m ) is the dual optimal solution. This means that which implies (2).

Conclusions
Under a regularity condition, we introduced two convex quadratic relaxations (CQRs) corresponding to two different conditions for model problem (1) that are the problems of minimizing a linear objective function over two convex quadratic constraints with additional linear inequality constraints. We presented sufficient conditions based on an optimal solution of the CQRs under which the problem (1) is equivalent to exactly one of the CQRs. We also showed that this equivalence reveals the strong duality holds for (1) and consequently problem (1) enjoys exact SDO-relaxation. We also derived new sufficient conditions based on the data of the model problem (1) for strong Lagrangian duality, exact SDO-relaxation and exact CQRs.
Possible subjects for future research direction would be finding more general conditions for exactness of CQRs (maybe even necessary and sufficient conditions), and other relaxations that are as simple as problems (6) and (7), but tight under more general conditions. Consider epigraph reformulation of (1) as following: First we show that the infimum in (A1) is attainable. To this end, note that under Assumptions 1 and 2, problem (1) without the linear constraints is bounded from below (see Theorem 5 of Reference [33]). This implies that (A1) is equivalent to the following problem: wheret is the optimal (infimum) value of q 1 (x) over the constraint q 2 (x) ≤ 0 and M is a sufficiently large constant. Let S denote the feasible region of problem (A2). The set S is closed and the objective function in problem (A2) is continuous. Therefore, to prove that the infimum in (A1) is attainable, it is sufficient to establish that S is bounded. Sincet ≤ t ≤ M, we only need to show that there existŝ M > 0 such that ||x|| ≤M for all (x, t) ∈ S. To do so, let h(x) := q 1 (x) +λq 2 (x) whereλ is the same as in Assumption 1. The function h is strictly convex and for any (x, t) ∈ S, we have h(x) ≤ M, implying that S is bounded. Next, we show that problem (A1) is equivalent to problem (1) and hence the infimum in (1) is always attainable. Let (x * , t * ) be an optimal solution of (A1). Since q 2 (x * ) ≤ 0 and c T i x * ≤ d i , i = 1, ..., m, x * is feasible for (1) and since q 1 (x * ) ≤ t * , we have p * ≤ t * = p * 0 . We show that p * = p * 0 . Suppose by contradiction that p * < p * 0 . Then by definition of infimum, there exists a feasible solution of (1), x, such that q 1 (x) < p * 0 . Set t = q 1 (x). Then (x, t) is a feasible solution of (A1) with a smaller objective value that contradicts the fact that p * 0 is the optimal value of (A1). Therefore, p * = p * 0 and since q 1 (x * ) = t * = p * , x * is the optimal solution of (1).
The fact that z T Bz > 0 (see Lemma 3.4 of Reference [22]) with (A19) imply that the above equation has a positive root α * . Then following the same discussion as in part (2) where λ 2 is replaced by λ 1