Nonsmooth Levenberg-Marquardt Type Method for Solving a Class of Stochastic Linear Complementarity Problems with Finitely Many Elements

Abstract: Our purpose of this paper is to solve a class of stochastic linear complementarity problems (SLCP) with finitely many elements. Based on a new stochastic linear complementarity problem function, a new semi-smooth least squares reformulation of the stochastic linear complementarity problem is introduced. For solving the semi-smooth least squares reformulation, we propose a feasible nonsmooth Levenberg–Marquardt-type method. The global convergence properties of the nonsmooth Levenberg–Marquardt-type method are also presented. Finally, the related numerical results illustrate that the proposed method is efficient for the related refinery production problem and the large-scale stochastic linear complementarity problems.

In addition to [1,2], Luo and Lin first considered a quasi-Monte Carlo method in [8,9]; they proved that the ERM method is convergent under mild conditions and studied the properties of the ERM problems.In [10], Chen, Zhang and Fukushima considered SLCP including the expectation of matrix is positive semi-definite.Under the condition of a new error bound, they use the ERM formulation to solve the SLCP.In [11], they also studied the ERM formulation, where the involved function is a stochastic R 0 function.In [12], Zhou and Caccetta put forward a new model of the stochastic linear complementarity problem with only finitely many elements.A feasible semi-smooth Newton method was also given.In [14], Ma also considered a feasible semi-smooth Gauss-Newton method for solving this new stochastic linear complementarity problem.The stochastic linear complementarity problem with only finitely many elements is to find a vector x ∈ n , such that: where where is equivalent to: x ≥ 0, Mx + q ≥ 0, x T ( Mx + q) = 0, where ( 4) is called the linear complementarity problem.
The main motivation of this paper is to use the advantages of [12,15,16,18,19] to solve the stochastic linear complementary problem denoted as (3).Firstly, we put forward a new robust reformulation of the complementarity Problem (3).Then, based on the new robust reformulation, we propose a feasible nonsmooth Levenberg-Marquardt-type method to solve (3).The numerical results in Section 4 showed that the given Method 1 is efficient for the related refinery production problem and the large-scale stochastic linear complementarity problems.We also make a comparison with Method 1 and the scaled trust region method (STRM) in [20]; preliminary numerical experiments showed that the numerical results of Method 1 are as good as the numerical results of the STRM method.Additionally, it generates less iterations in contrast to the STRM method.Additionally, we also make a comparison with Method 1 and the method in [13] for solving the related refinery production problem.The preliminary numerical experiments also indicate that Method 1 is quite robust.In other words, Method 1 has the following advantages.

•
Faster reduction of the complementarity gap x T ( Mx + q).
The remainder of this paper is organized as follows.In Section 2, we review some background definitions and some useful properties.In Section 3, we define a merit function θ(z) = 1  2 F(z) 2 and give a feasible nonsmooth Levenberg-Marquardt-type method.Some discussions about this method are also given.In Section 4, the numerical results indicate that the given method is efficient for solving stochastic linear complementarity problems, such as the related refinery production problem and the large-scale stochastic linear complementarity problems.

Preliminaries
In this section, we give some related definitions and some properties; some of them can be found in [6,14,15,[19][20][21][22][23]; some of them are given for the first time.
Let G : m −→ n be a locally-Lipschitzian function.The B-subdifferential of G at x is: where D G is the differentiable points set and G (x) is the Jacobian of G at a point x ∈ n .The Clarke generalized Jacobian of G is defined as: for all x, y ∈ n , x = y and x i = y i .

Proposition 2. ([21]
) Suppose G is a locally-Lipschitzian function and strongly semi-smooth at x. Additionally, it is also directionally differentiable in a neighborhood of x; we get:
For any (x, y) ∈ (m+1)n , we know that: , where I is the n × n identity matrix.Hence, by Proposition 5, we know that the following proposition is set up.Proposition 6. Suppose ( 4) is R-regular at x * and (x * , y * ) is a solution of (9).Then, all V ∈ ∂ C F(x * , y * ) are nonsingular.
Proof of Proposition 7. The proof is similar to the ([15], Lemma 2.5) and therefore omitted here.
In the following part of this paper, we rewrite Ψ as: where ψ : 2 → is defined as: Proposition 8.The function Ψ : n → defined in (8) satisfies: (b) If ∇Ψ(x * ) = 0 and M is a P 0 matrix, we know that x * is a solution of (4). (c) If ( 4) is strictly feasible and x −→ Mx + q is monotone, then L(c) = {x ∈ n |Ψ(x) ≤ c} are compact for all c ∈ .
Proof of Proposition 8.The proof is similar to the one of ([15], Theorem 2.7), so we skip the details here.

The Feasible Nonsmooth Levenberg-Marquardt-Type Method and Its Convergence Analysis
In this section, we define a merit function θ(z) = 1 2 F(z) 2 and give a feasible nonsmooth Levenberg-Marquardt-type method.At the same time, we also give some discussions about this method.
(a) θ(z) is continuously differentiable on (m+1)n with ∇θ(z) = H T F(z) for any H ∈ ∂ C F(z).(b) Assume x −→ Mx + q is monotone, if LCP( M, q) has a strictly feasible solution, then for all c > 0, we know that the level set: For some monotone stochastic linear complementarity problems, the stationary points of (10) may not be a solution.Such as [12] ).
By simple computation, we know that the above of problem is a monotone SLCP, obviously; all points x ≥ 1 are feasible, but this example has no solution.By: , and (0, 1, 0) is a stationary point of the constraint optimization problem: However, x = 0 is not a solution of this example.Therefore, in the following proposition, we give some conditions for (3).
Proof of Proposition 10.Assuming that z * = (x * , y * ) is a stationary point of ( 10), if M(ω i )x * + q(ω i ) − y * i = 0, i = 1, 2, ...m, by (12), we know that x * is the stationary point of the following problem: Similar to the proof of Theorem 3 in [24], it can be shown that x * is a solution of Ψ(x) = 0. Thus, x * is a solution of (3).Now, we present the feasible nonsmooth Levenberg-Marquardt-type method for solving (3).
and find the solution d k of the equations: Step 3. If  We now investigate the convergence properties of Method 1.In the following sections, we assume that Method 1 generates an infinite sequence.
Theorem 1. Method 1 is well defined for a monotone SLCP (3).If Method 1 does not stop at a stationary point in finite steps, an infinite sequence {z k } is generated with {z k } ⊂ Z, and any accumulation point of the sequence {z k } is a stationary point of θ.
Proof of Theorem 1. Method 1 is well defined for the reason of ν k > 0, and d k is always a descent direction for θ.Now, we consider the following two situations respectively.
(I) If the direction d k is accepted by an infinite number of times in Step 3 of Method 1, we get: Since ∇θ(z k ) = 0 implies d k = 0, we have: From [17], we know that {θ(z k )} is monotonically decreasing.Obviously, this implies that the sequence { F(z k ) } is also monotonically decreasing.Since ) is accepted by an infinite number of times in view of our assumptions, therefore we get F(z k ) → 0 for k → ∞ by γ ∈ (0, 1).This means that any accumulation point of {z k } is the solution of (10); therefore, it is also a stationary point of θ.
(II) This case is the negation of Case (I); without loss of generality, we assume that the Levenberg-Marquardt direction is never accepted.If the direction P Z [z k − t k ∇θ(z k )] − z k is accepted by an infinite number of times in Step 4 of Method 1, we have: By (b) in Lemma 1, taking x := z k − t k ∇θ(z k ), y := z k , we get: where . By the Armijo line search properties, we know that any accumulation point of {z k } is a stationary point of θ, and this completes the proof.
Theorem 2. Let x * ∈ n be a R−regular solution; then the whole sequence generated by Method 1 converges to z * Q-quadratically.
Proof of Theorem 2. By Proposition 6, there is a constant c 1 > 0, such that, for all z k ∈ (z * , δ 1 ), where δ 1 is a sufficiently small positive constant, the matrices H T k H k + ν k I are nonsingular, and . Furthermore, by Proposition 2, there exists a constant c 2 > 0, such that: for all z k ∈ (z * , δ 2 ), where δ 2 is a sufficiently small positive constant.Moreover, in view of the upper semicontinuity of the C-subdifferential, we have: where , and δ 3 is a sufficiently small positive constant.Denote δ = min(δ 1 , δ 2 , δ 3 ), for z k ∈ (z * , δ).Note that, from (13) and Lemma 1, we have: Since F is a locally-Lipschitzian function and ν k = F(z k ) , by premultiplying this equation by (H T k H k + ν k I) −1 and taking norms both sides, we get: where τ = 2c 1 (ζc 2 + L).Therefore, similar to the proof of ([20], Theorem 2.3), we know that the rate of convergence is Q-quadratic.This completes the proof.

Numerical Results
In this section, firstly, we make a numerical comparison between Method 1 and the scaled trust region method (STRM) in [20].We apply Method 1 and the scaled trust region method to solve Examples 1 and 2. Secondly, we use Method 1 to solve the related refinery production problem, which also has been studied in [4,13].Finally, numerical results about large-scale stochastic linear complementarity problems are also presented.We implement Method 1 in MATLAB and test the method on the given test problems using the reformulation from the previous section.Additionally, all of these problems were done on a PC (Acer) with i5-3210M and RAM of 2 GB.Throughout the computational experiments, the parameters in Method 1 are taken as: The stopping criteria for Method 1 are θ(z k ) ≤ 10 −15 or k max = 5000.
The parameters in the STRM method (see [20]) are taken as: The stopping criteria for the STRM method are D k g k ≤ 10 −15 or k max = 5000.
In the tables of the numerical results, DIM denotes the dimension of the problem (the dimension of the variable x); x * denotes the solution of θ(x, y) = 0; In the following part of this section, we give the detailed description of the given test problems.
Numerical results of Example 1 are given in Table 1, Figures 1 and 2, respectively.x 0 are chosen randomly in 2 ; y 0 are chosen randomly in 4 and λ = 0.1.From Table 1, we can see that the merit functions associated with p ∈ (1, 2), for example p = 1.5, are more effective than the Fischer-Burmeister merit function, for which exactly p = 2.
In Table 2, we give the numerical comparison of Method 1 with fmincon, which is a MATLAB tool box for constrained optimization.We use the sequential quadratic programming (SQP) method in the fmincon tool box to solve Example 1 by p = 1.1 and the same initial points.From Table 2, we can see that Method 1 is more effective than fmincon.
From Table 3, Figures 3 and 4, we can see that the iterations of Method 1 are less than the STRM method.In Method 1, when p = 5, the function value falls faster.When p is larger, a greater number of iterations is needed in the STRM method.In Table 4, we also give the comparison of Method 1 with fmincon.For the propose of comparison, we fixed p = 10 and the same initial points.From Table 4, we can see that Method 1 is also more effective than fmincon.Example 3.This example is a refinery production problem, which is also considered in [2,13].

•
For each (j, i), calculate the average υ j,i of ω k j ; it belongs to I j,i .

•
For each (j, i), the estimated probability of υ j,i is p j,i = k j,i /K, where k j,i is the number of ω k j ∈ I j,i .

Remark 1.
In this paper, we use: The computation cost of our method is greatly reduced.In fact, when we think about the general case of ω 1 , ω 2 , ω 3 and ω 4 varying the random distribution of discrete approximation by a 5-, 9-, 7-and 11-point distribution, respectively.This yields a joint discrete distribution of 5 × 9 × 7 × 11 = 3465 realizations.Then, F(z) is a function of 17,335 (3465 × 5 + 10 = 17,335) dimensions.This is a more complex optimization problem.
In Table 7, we give the comparison of Method 1 with the SQP method in the fmincon tool box, when the dimensions of Example 4 are 10, 100, 200, 300 and 400; where θ(x, y) = 1 2 F(x, y) 2 .x 0 are chosen randomly in n .y 0 are chosen randomly in 2n , λ = 0.0001.

Conclusions
In this paper, we introduced a feasible nonsmooth Levenberg-Marquardt-type method to solve the stochastic linear complementarity problems with finitely many elements.This method used a linear least squares reformulation of the stochastic linear complementarity problem and applied a feasible nonsmooth Levenberg-Marquardt-type method to solve the reformulated problem.The finally given numerical results showed that the given method is efficient to solve the large-scale stochastic linear complementarity problem and related refinery production problem.Additionally, the method can choose the initial points in a large scope with less computations and high precision.
and go to Step 1.

Figure 4 .
Figure 4. Numerical results for Example 2 by the STRM method.The x-axis represents the iteration step; the y-axis represents θ(x, y) = 1 2 F(x, y) 2 .

Table 2 .
Numerical results for Example 1 by Method 1 and fmincon.

Table 3 .
Numerical results for Example 2. Numerical results for Example 2 by Method 1.The x-axis represents the iteration step; the y-axis represents θ(x, y) = 1 2 F(x, y) 2 .

Table 4 .
Numerical results for Example 2 by Method 1 and fmincon.

Table 5 .
Numerical results for Example 3 based on Condition 1.

Table 6 .
Numerical results for Example 3 based on Condition 2.

Table 7 .
Numerical results for Example 4. By the numerical results of Example 4, we can see that Method 1 is very suitable to solve large-scale SLCP.Moreover, Method 1 can be used flexible by adjusting the value of p.