Experiments with Active-Set LP Algorithms Allowing Basis Deficiency

Abstract

An interesting question for linear programming (LP) algorithms is how to deal with solutions in which the number of nonzero variables is less than the number of rows of the matrix in standard form. An approach is that of basis deficiency-allowing (BDA) simplex variations, which work with a subset of independent columns of the coefficient matrix in standard form, wherein the basis is not necessarily represented by a square matrix. We describe one such algorithm with several variants. The research question deals with studying the computational behaviour by using small, extreme cases. For these instances, we must wonder which parameter setting or variants are more appropriate. We compare the setting of two nonsimplex active-set methods with Holmström’s TomLab LpSimplex v3.0 commercial sparse primal simplex commercial implementation. All of them update a sparse QR factorization in Matlab. The first two implementations require fewer iterations and provide better solution quality and running time.

1. Introduction

Since the introduction of the simplex method for linear programming (LP) in 1947, there has been interest in viewing algorithms from a nonlinear optimisation point of view. Most well known and developed are the ideas of interior point methods, established at the time of the first barrier methods of Dikin in 1967. Although the questions on the simplex method originate from its beginning in 1947, there is still a lot of ongoing research; in the last three years one can find publications such as [1,2,3,4,5,6,7,8], just to cite a few. Various aspects of the method are investigated in these works, where Im and Wolkowicz [6] specifically also focus on degeneracy and related issues. However, these recent works neither work with a nonsquare basis, nor do they include a sparse linear algebra procedure to efficiently solve the sparse least squares subproblems that we need in our algorithmic approach.

A generalization of the well-known dual simplex method can be found in [9,10], which discuss relaxing operations toward nonsimplex steps. One can find more recent considerations in this line in [11,12] with the incorporation of steepest-edge pivoting rules (see also [13,14]). Gill and Murray [9] presented a nonsimplex active-set algorithm for solving a problem $\left(P\right)$ starting with a feasible point $x_0$. Descriptions and analyses can also be found in [14,15,16]. We investigated and developed a nonsimplex active-set (NSA) method for a dual linear program in standard form $\left(D\right)$, [17]. A challenge for the development is to have to deal with high degeneracy, which for a simplex-like method provides basis-deficiency, i.e., the dual variable set has less than n variables. Therefore, we aimed for the development of a method, which is also called a basis deficiency-allowing (BDA) simplex variation (see [11,12,18]).

This paper presents a study of the computational behavior of a Matlab implementation on a set of extreme cases and sparse inequality-constrained linear programs customized from the literature. We compare two nonsimplex active-set methods with the commercial solver of Holmström called TomLab LpSimplex v3.0 sparse primal simplex implementation. Various numerical linear algebra procedures are used in Matlab to keep a sparse QR factorization of the tall-and-skinny basis matrix updated. To elaborate upon the experiments, we introduce the notation and the main algorithm and its implementation in Section 2. We report on computational findings in Section 3. Section 4 summarises our findings.

2. Materials and Methods

Consider the nonsymmetric primal dual relation in linear programming, wherein we deviate from the usual notation here in exchanging b and c, x and y, n and m, and $\left(P\right)$ and $\left(D\right)$ in the notation of e.g., [14]. Now consider

$$\begin{array}{cccc}\left(P\right)\hfill & \hfill min& \ell \left(x\right):=c^{T}x\hfill & ,x\in {\mathbb{R}}^n\hfill \\ & \hfill \mathrm{s}.\mathrm{t}.& A^{T}x\ge b\hfill \end{array}\begin{array}{cccc}\left(D\right)\hfill & \hfill max& \mathcal{L}\left(y\right):=b^{T}y\hfill & ,y\in {\mathbb{R}}^m\hfill \\ & \hfill \mathrm{s}.\mathrm{t}.& Ay=c\hfill & ,y\ge 0\hfill \end{array},$$

(1)

where $A\in {\mathbb{R}}^{n\times m}$ with $m\ge n$ and $\mathrm{rank}\left(A\right)=n$. Let $\mathcal{F}$ and $\mathcal{G}$ denote the feasible region of $\left(P\right)$ and $\left(D\right)$, respectively. Here, c and b are, respectively, the primal and dual gradient of the cost functions to be optimized, whereas $a_j$ and $b_j$ are, respectively, the coefficient vector and right-hand side value of the jth primal constraint $a_j^{T}x-b_j\ge 0$.

The idea is the following. We start with a dual feasible point $y_0$. From there, we separate the index set $[1:m]:=\{1,2,\dots ,m\}$ into an ordered basic set ${\mathcal{B}}_k$ with $m_k:=\left|{\mathcal{B}}_k\right|$ elements, with $m_k\le n$ and its complement representing the columns in ${\mathcal{N}}_k:=[1:m]\setminus {\mathcal{B}}_k$, which can have more than $m-n$ zero elements. Let $A_k\in {\mathbb{R}}^{n\times m_k}$ and $N_k\in {\mathbb{R}}^{n\times (m-m_k)}$ be submatrices of A formed by the columns corresponding to ${\mathcal{B}}_k$ and ${\mathcal{N}}_k$, respectively. Notice that $\mathrm{rank}\left(A_k\right)=m_k$. We will use the notation $\mathcal{R}\left(A\right)$ to represent the range of the columns in A, while $null\left(A\right)$ denotes its null space, and $\left|A\right|$ denotes the number of its nonzeros. Column j of A is denoted by $a_j$. Similarly, $a_i^T$ is row i of matrix $A^T$. Moreover, $b_{{\mathcal{B}}_k}$ represents an $m_k$ vector with the elements of ${\mathcal{B}}_k$ of b and $b_{{\mathcal{N}}_k}$ has the nonbasic elements. The index k will be left out if it is clear from the context, ${\parallel ·\parallel }_2^2$ denotes the square of the Euclidean 2-norm, and symbol § will be used in citations as an abbreviation of chapter/section for brevity purposes.

2.1. Algorithm

The algorithm works like the simplex method in determining in each iteration a nonbasic variable that enters the basis and a basic variable(s) leaving the basis. The big difference is that it can handle basis deficiency. This means that $m_k\le n$ and $m_k$ may reduce and grow. A pseudocode is given in Algorithm 1.

• Notice that in line 4, if we have a complementary pair of primal dual feasible points, then $x_k$ is optimal for (P).
• Line 15 determines step size and basis-leaving variable(s), unless in line 12 we found that we have an unbounded dual objective function.

The algorithm does not require a primal feasible starting point. In that sense, it is an exterior method. In earlier studies, we have shown [19] that the algorithm is equivalent to the primal BDA simplex variation given by Pan in [11]. However, we put emphasis on a description, which is easy to understand due to a geometrical interpretation and its independence of implementation details. The algorithm also fits in the primal-feasibility search loop of the Sagitta method described by Santos–Palomo [20], which is related to a loop used in dual active-set methods for quadratic programming. Moreover, Li [21] extended the algorithm to deal with upper and lower bounds on y.

Practically, one obtains the primal-feasibility phase I of Dax’s 1978 [22] idea for Rosen’s 1960 paper [10] when one replaces the min-ratio test by a most-obtuse-angle row rule to determine the leaving variable and removing an objective function consideration. In this context, we focused earlier [23] on the classical example of Powell [24], which illustrates the cycling behaviour of a most-obtuse-angle simplex pivoting rule described in [12,25].

Algorithm 1 Pseudocode active-set algorithm.

1: Set $k\leftarrow 0$, $y_0$ feasible point of (D) with corresponding ${\mathcal{B}}_0$ and $A_0$ 2: $x_k\leftarrow$ solves $A_k^{T}x=b_{{\mathcal{B}}_k}$ with residuals $r_k$ following from $r_{{\mathcal{N}}_k}=N_k^T{x}_k-b_{{\mathcal{N}}_k}$ 3: if$r_k\ge 0$then 4:     return $x_k$ is optimal 5: select $p\in {\mathcal{N}}_k$ with $r_{kp}<0$ 6: if$a_p\notin \mathcal{R}\left(A_k\right)$then 7:     ${\mathcal{B}}_{k+1}\leftarrow {\mathcal{B}}_k\cup \left\{p\right\}$, ${\mathcal{N}}_{k+1}\leftarrow [1:m]\setminus {\mathcal{B}}_{k+1}$, set $A_{k+1}$ 8:     $y_{k+1}\leftarrow$ follows from $A_{k+1}{y}_{{\mathcal{B}}_{k+1}}=c$, $k\leftarrow k+1$, goto step 2 9: else 10:     solve $A_k\delta =a_p$ 11:     if $\delta \le 0$ then 12:         return Unbounded $\mathcal{L}$ 13:     else 14:         Direction $d_k$ is initially unit vector $e_p$ and elements of ${\mathcal{B}}_k$ follow from $-\delta$ 15:         Step-size $\tau \leftarrow {min}_{q\in {\mathcal{B}}_k}\left\{\frac{y_{kq}}{-d_{kq}}\right|d_{kq}<0\}$, $Q\leftarrow {argmin}_{q\in {\mathcal{B}}_k}\left\{\frac{y_{kq}}{-d_{kq}}\right|d_{kq}<0\}$ 16:         $y_{k+1}\leftarrow y_k+\tau d_k,{\mathcal{B}}_{k+1}\leftarrow {\mathcal{B}}_k\setminus Q\cup \left\{p\right\},{\mathcal{N}}_{k+1}\leftarrow {\mathcal{N}}_k\cup Q$ 17:         Set $A_{k+1}$, $k\leftarrow k+1$, goto step 2

The algorithm ends with an optimal solution for $\left(P\right)$ and $\left(D\right)$. However, the solution does not necessarily correspond to a square basis. Notice that the algorithm maintains dual feasibility and complementary slackness. In fact, two algorithmic strategies now emerge. One of them, Reliable Sagitta, relies on starting Algorithm 1 from a dual feasible point. The other one, Sparse Sagitta—abbreviated as Sagitta—starts Algorithm 1 as soon as a solution of $Ay=c$ is found with a suitable, linearly independent subset of columns of A where y is possibly violating nonnegativity. These two alternatives, along with the chosen touchstone for a fair comparison, are outlined in Section 2.2.

2.2. Implementation

To run the algorithm, we first find a feasible dual vertex, i.e., a vector $y\ge 0$ such that $Ay=c$ with columns of A corresponding to positive components of y being linearly independent. This is called a phase-I procedure. One option is to use the procedure of the nonnegative least squares (NNLS) approach described in [19,26]. One can apply the NNLS algorithm [27] or Dax algorithm [28] to the positive semidefinite quadratic problem

$$\underset{y\ge 0}{min}\frac{1}{2}{\parallel Ay-c\parallel }_2^2,$$

(2)

which implies solving a sequence of unconstrained least squares problems of the form $min\parallel A_k{z-c\parallel }_2$ (NNLS) or of the form $min\parallel A_{k}w-s_k{\parallel }_2$ (Dax). Such an approach does not require artificial variables and obtains a feasible direction of the feasible region $\mathcal{F}$ (i.e., a so-called direction of $\mathcal{F}$) or a dual basic feasible solution with $m_k\le n$. It appears that the computation can be described in terms of a primal null-space descent direction [29]. Instead of (2), one can focus on its Wolfe dual, which is a least-distance, strictly convex quadratic problem

$$min\frac{1}{2}{\parallel u\parallel }_2^2 \mathrm{s}.\mathrm{t}.A^{T}u\ge v,$$

(3)

where $v:=A^{T}c$. This also implies solving a sequence of least squares problems. To solve (3), one can use any of the primal, dual, and primal dual active-set methods introduced in [30]. Note that a primal direction $d:=u-c$ is easy to generate.

Once a dual feasible vertex $y^{*}$ is obtained, a phase-II procedure is performed to reach optimality. A traditional way to continue is to artificially enlarge $\mathcal{B}$ from $m_k$ to n columns and then apply the primal simplex method to $\left(D\right)$ starting from a degenerate dual vertex of $\mathcal{G}$. However, the interesting approach we follow is not to artificially enlarge $\mathcal{B}$, but to use $y^{*}$ as the starting point for a nonsimplex active-set method for linear programs in standard form as sketched in Algorithm 1. We shall refer to this first algorithm for solving linear programs, such as Reliable Sagitta (RelSag v1.1), for which termination is guaranteed under a dual nondegeneracy assumption (see [17]).

The second variant of phase I to solve (2) adapts the inner loop, called Loop B in [27] and step S2 in [29]. This loop sequentially removes indices from $\mathcal{B}$ of those columns of $A_k$ corresponding to negative multiplier least-squares estimates ${\overline{y}}_{B_i}$ of actual Lagrange multipliers $y_{B_i}$ for $\left(P\right)$. We substitute this loop by the following steps to determine the search direction: Compute temporal variables ${\overline{y}}_B$ solving

$$min\parallel A_k{y}_B-c\parallel .$$

(4)

Then we set $y\leftarrow [{\overline{y}}_B;0]$, and take $d\leftarrow A_k{\overline{y}}_B-c$ and go on to prepare the next iteration. Notice that ${\overline{y}}_B$ is not constrained to be nonnegative.

This NNLS modification finishes with a dual solution $y^{*}$ verifying $Ay=c$, but nonnegativity does not hold in general. Hence the primal-feasibility search loop is not ensured to start from a feasible solution $y\in \mathcal{G}$, and termination is not guaranteed. In the event that this loop ends, we can check whether $y\ge 0$ stops with the optimal solution, or deletes a negative element $y_{B_i}$ to restart the whole process. Note that we have delayed the elimination of negative multipliers until they have become actual multipliers. We shall refer to this second algorithm for solving linear programs such as Sparse Sagitta (SpaSag v1.1).

The comparison of the two nonsimplex active-set methods described above was done with the commercial code of Holmström TomLab LpSimplex v3.0 [31] sparse primal simplex implementation, applied to solve $\left(D\right)$. Details of this Matlab solver can be found in http://tomlab.biz/products/base/solvers/lpSimplex.php (accessed on 15 November 2022). This code was previously known as LpSolve, but should not be confused with the freely available compiled code lp_solve from http://lpsolve.sourceforge.net/5.5/ (accessed on 15 November 2022). To make results comparable, the input of the problems in all the solvers are taken the same way. However, LpSimplex deals with the columns of A, whereas RelSag and SpaSag deal with the rows of $A^T$; hence all of them are under the same conditions when solving $\left(P\right)$ and $\left(D\right)$.

We also considered comparison with other available software, which appeared less appropriate.

1.

LinProg v2.1 [32] included in Matlab Optimization Toolbox v2.1. Its active-set method does not allow sparse arithmetic as described above. Other available Matlab codes like those of Morgan [33] have not been considered because he ultimately rejected the sparse dynamic LU updating that he implemented in favour of a systematic LU restart made from scratch on the basis matrix after each column exchange.

2.

We did not compare our code to commercial or freely available compiled codes like Mosek [34] or Minos [35], because the running time measurements are not comparable with those of our interpreted Matlab code. Although solution time is not the only metric that we have used here (number of iterations, quality of solutions), we consider that our solvers must be at least comparable in running time with implementations developed under the same conditions.

3.

It is not our aim to use the Matlab environment to exclude alternative implementations like Cplex, XPress-MP, Coin-or, or lp_solve, to name just a few. We plan to compare against them with larger problems when we develop our own implementation. Moreover, although it is part of the folklore that sparse QR-based methods can suffer more fill-in than similar LU based ones, we expect that easier updating and downdating techniques as those of Cholmod described recently by Davis and Hager in [36] can be used to show the scalability of the developed implementations to larger models.

3. Results

This section first describes details of the used settings in Section 3.1 and then focuses on the question of how to make use of specific sparse matrix computations. We discuss results for small, extreme cases from the literature in Section 3.2 to find out which settings are more appropriate for which type of instances. As we will show, this might provide some advantage compared to a general commercial solver like Tomlab. In Section 3.3, we deal with features to handle sparse instances. In earlier work, we also reported on computational tests with 23 out of the smallest 31 Netlib LP problems, which were originally put forward in standard form $\left(D\right)$ (i.e., those with no simple bounds on our dual variables), which are currently being analyzed by Im and Wolkowicz [6]. The details of these tests can be found in [37], and a summary of these tests has been included in [17].

3.1. Settings

The used implementation (called Sagitta) is intended to find a minimum feasible point of $\left(P\right)$, without using an initial feasible point. Default values for the parameters are

• maximum number of iterations (default is $5n$);
• set of zero tolerances [38] tol (default is $[5·{10}^{-5},{10}^{-8},{10}^{-6}]$); and
• character string named o with options. The length of this character string is 5, with o(1), …, o(5) denoting each individual option. Their interpretation has been summarized in

  Table 1. Constraint selection criteria if descent direction d exists.

  o(1)	Meaning
  ‘s’	Most opposite to d no normalization
  ‘t’	Most opposite to d with normalization
  ‘a’	Among opposite to d, most opposite to $(-c)$ no normalization
  ‘b’	Among opposite to d, most opposite to $(-c)$ with normalization
  ‘o’	Among opposite to d, most opposite to $d+(-c)$ no normalization
  ‘p’	Among opposite to d, most opposite to $d+(-c)$ with normalization

  ,

  Table 2. Activation criteria in absence of descent direction.

  o(5)	Meaning
  ‘s’	Most violated $a_j^{T}x\ge b_j$ without normalization (Dantzig)
  ‘t’	Most violated with normalization

  and

  Table 3. Deactivation criteria.

  o(5)	Meaning
  ‘s’	Most negative multiplier (actual/estimate) without normalization (Dantzig)
  ‘t’	Most negative multiplier (actual/estimate) with normalization

  . The setting o(3) controls the choice between RelSag (o(3)=‘s’) and SpaSag (o(3)=‘n’, by default. Reporting of intermediate results and use of updating formulae for residues and search direction are asserted, respectively, with o(2)=‘s’ and o(4)=‘s’. The default is ‘n’ due to lack of a detailed rounding error analysis for such formulae.

Our implementation does not make use of simple bounds. Furthermore, it uses a null-space steepest-descent direction d as the search direction. A constraint $a_j^{T}x\ge b_j$ is said to be opposite to direction d if $a_j^{T}d<0$. In the presence of descent direction d (see Table 1), one of the available addition criteria is to add the constraint $a_j^{T}x\ge b_j$ most opposite with Euclidean normalization to the descent direction if o(1)=‘t’ by default. The implementation chooses the opposite $-c$ of the cost vector if o(1)=‘b’, or chooses the difference between the descent direction and the cost vector if o(1)=‘p’. The selection is made among those constraints opposite to d in the last two cases. The implementation can also select not to normalize using the options o(1)=‘s’, o(1)=‘a’, and o(1)=‘o’, respectively. It is worth noting that all criteria given in Table 1 have a strong geometrical interpretation, trying to take advantage of the fact that both d and $-c$ are descent directions.

Setting o(5) controls the choice of the addition criterion in the absence of a descent direction (see Table 2). It chooses the most violated constraint with ${\ell }_2$ norm if o(5)=‘t’ by default or without normalization if o(5)=‘s’. A constraint deletion (Table 3) can be performed both in the presence or in the absence of a descent direction. The former (only available for RelSag) chooses the constraint with the most negative multiplier estimate with ${\ell }_2$ norm if o(5)=‘t’ by default or without normalization if o(5)=‘s’. The latter (only available for SpaSag) works in a similar way by using actual multipliers instead of multiplier estimates when there is no violated constraint. Moreover, in a constraint exchange this eventuality may be needed in the case of primal feasibility; the constraint to be deleted is chosen in both algorithms by using a min-ratio test similar to that used in the primal simplex method [20,37].

The criteria shown in Table 2 can be thought of as being different ways by which to weigh the residues of the violated constraints. When all of them have unit weight, the classical Dantzig rule applies o(5)=‘s’ and when the weight for constraint i is $\parallel a_i{\parallel }^{-1}$, rule o(5)=‘t’ is selected. In this way, steepest-edge rules work with weight $\parallel {\delta }^i{\parallel }^{-1}=\frac{1}{\sqrt{1+\parallel {\delta }_B^i{\parallel }^2}}$, with ${\delta }_B^i:=A_k^{†}{a}_i$. Note that, in the simplex case, $a_i\in \mathcal{R}\left(A_k\right)$ for all nonbasic i because of the regularity of $A_k$, but the compatibility of $A_k{\delta }_B^i$=$a_i$ is not guaranteed in the nonsimplex case, and hence we have to rely on the pseudoinverse $A_k^{†}:={\left(A_k^T{A}_k\right)}^{-1}$$A_k^T$ (see e.g., [39] (p. 17, Equation (1.2.27))). Thus,

$$\parallel {\delta }_B^i\parallel =\parallel A_k^{†}{a}_i\parallel \le \parallel A_k^{†}\parallel \parallel a_i\parallel \mathrm{implies}\frac{1}{\parallel {\delta }_B^i\parallel }\ge \frac{1}{\parallel A_k^{†}\parallel \parallel a_i\parallel }.$$

One can approximate the weights of the steepest edges by

$$\frac{1}{\parallel {\delta }^i\parallel }=\frac{1}{\sqrt{1+\parallel {\delta }_B^i{\parallel }^2}}\approx \frac{1}{\parallel {\delta }_B^i\parallel }\approx \frac{1}{\parallel A_k^{†}\parallel \parallel a_i\parallel }.$$

Notice that the factor $\parallel A_k^{†}{\parallel }^{-1}$ is shared for all i. This reasoning implies that rule o(5)=‘t’ can be thought of as a rough approximate steepest-edge rule. An easier rule to approximate $\frac{1}{\parallel {\delta }^i\parallel }$ in a simplex context has been described by Świȩtanowski [40].

Both nonsimplex active-set methods have been implemented by using range-space techniques by maintaining the sparse QR factorization of $A_k$. Specifically, the purpose of the modified sparse least squares (MSLS) toolbox v1.3 (that we have developed and described in [29] (§3)) is the sparse updating and downdating of the Cholesky factor $R_k$ of $A_k^T{A}_k$. The orthonormal factor of the QR factorization of the full column rank matrix $A_k\in {\mathbb{R}}^{n\times m_k}$ (with $m_k\le n$) is implicitly maintained as a product $A_k{R}_k^{-1}$. The triangular factor $R_k$ recurs in sparse form, and is recomputed from scratch by using the sparse Matlab QR factorization primitive qr when a refactorization is triggered due to an accumulation of rounding errors. Our toolbox consists of the routines SqrIni (initialization of data structures), SqrIns (addition of a constraint to the working set), SqrDel (deletion of a constraint from the working set), SqrRei (refactorization of $A_k$), Givens (computation of a Givens rotation) and EjemMSLS (application example).

By using Matlab notation, we store the working set $\mathcal{B}$ in a vector ACT. A call to SqrIni(mode,scal) is used to initialize the use of NaNs as a lower triangular static data structure L, such that L(ACT,ACT) is an upper bound for the transposed Cholesky factor of A(:,ACT)’*A(:,ACT) independent of what ACT is. When mode=‘dmperm’, both the rows and columns of A (and consequently b and c) are reordered for A to be in lower block triangular form (LBTF). Hence, the static structure is tightly set up in L(ACT,ACT). With mode=‘colmmd’, only the columns of A (and consequently b) are reordered for the static structure set up in L(ACT,ACT) to be an upper bound for that Cholesky factor. Scaling techniques can be a priori applied (scal=‘noscale’, by default), like that in which all columns of A and c have a Euclidean norm unit (scal=‘txtbook’), or like that used in Cplex, where the columns of A are normalized in ${\ell }_{\infty }$ norm, and then the same is done with the rows of A, when scal=‘cplex92’ (see Bixby [41] (p. 271)).

Because we are not using the orthogonal factor due to sparsity considerations, the least squares subproblems are solved by using corrected seminormal equations (CSNE) (see e.g., [39] (p. 126)). Thus, the descent direction d is computed by solving (4) with CSNE. This requires us to perform an iterative refinement step after having found the solution ${\overline{y}}_B$ of the seminormal equations

$$R_k^T{R}_k{y}_B=A_k^T{A}_k{y}_B=A_k^{T}c.$$

Then, $d=A_k{\overline{y}}_B-c$ is determined as the orthogonal projection of $-c$ onto null $\left(A_k^T\right)$. This means that d is a null-space steepest-descent direction. The same technique is used to solve the compatible system $A_k{y}_B=c$ when $c\in \mathcal{R}\left(A_k\right)$. To test whether $v\in {\mathbb{R}}^n$ is in $\mathcal{R}\left(A_k\right)$, we check whether the residual of $min\parallel A_k{\delta }_B-v\parallel$ is zero. As a solution $\overline{x}$ of the undetermined system $A_k^{T}x=b_B$ we take that of the minimum norm by using corrected seminormal equations of the second kind (e.g., [39] (pp. 7, 70, 126)). This requires an iterative refinement step after having found a solution $\overline{z}$ of the seminormal equations of the second kind,

$$R_k^T{R}_{k}z=A_k^T\left(A_{k}z\right)=b_B\mathrm{and}\overline{x}=A_k\overline{z}.$$

The commercial program TomLab LpSimplex v3.0 [31] enhances a refined Matlab implementation of the primal simplex method, in which sparse matrices can be handled in the solution process. A sparse QR factorization of the basis matrix is maintained with an explicit orthogonal factor. The updating and downdating is done with customized slight modifications of the dense Matlab primitives qrinsert and qrdelete, to facilitate dealing with sparse operands. Notice that the authors of the software claim it handles sparsity exploitation better. Default settings were chosen for LpSimplex.

3.2. Extreme Cases

The question we have is what options of Sagitta are necessary to solve extreme cases from the literature. This relates to the general question in the investigation of optimization techniques of which type of algorithms are more appropriate to solve which type (defined by their characteristics) of optimization problems. The investigation does not require big instances, but extremely designed cases like the well-known Klee–Minty box. We report on our finding here.

Gass mentioned in [42] that for any new LP code, it is good to find out which instances may work badly. He specifically designed textbook examples. We gathered 27 from a wealth of different sources in a Matlab routine called ExampSag, which can be found in the Appendix A.

We solved a first set of problems involving primal degenerate (problems 9 and 10) and dual degenerate (problems 14 to 16) problems, corresponding, respectively, to [14] (p. 61), [38] (p. 31), and [43] (p. 136). Sagitta solved all of them without any problem and avoided cycling in the last three cases by using minimum-norm solutions instead of basic ones when solving the underdetermined compatible systems. It is interesting to note that in the latter two cases, cycling appears if we use basic solutions. This illustrates perfectly that the possibility of a cycle cannot be ruled out for some pivoting rules. We looked for cases unfavourable for the Expand technique of solving the degeneration of [44]. We found them in instances of [45] (§2, §4), where the Sagitta code appeared effective in obtaining correct results (problems 19 to 22) for those with a nonzero cost vector.

The literature provides instances that are problematic for interior point methods (Mészáros [46] (pp. 8, 9) and Stojković and Stanimirović [47] (pp. 436, 437)). These are problems 23 to 26 in the Appendix A. Basically, they illustrate that approximate arithmetic may lead to challenges for accepting the optimal solution.

The interesting two cases from Mészáros [46] (pp. 8, 9) could be solved by Sagitta without adjusting the used tolerances. The case where the simplex method was predicted to encounter challenges has a unique solution of ${(1,1,1,0)}^T$. Tomlab provides as solution ${(1,2,0,0)}^T$ which implies a row residue ${\parallel c-Ay\parallel }_{\infty }$ of ${10}^{-8}$ and Sagitta gives ${(1-{10}^{-8}/3,0,2-{10}^{-8}/3,0)}^T$ corresponding to a row residue of $\frac{{10}^{-8}}{3}$.

Consider the two problems presented in Stojković and Stanimirović [47]. The authors claim that by using their method in exact arithmetic they obtain the optimal solutions 0 and −24,196,384 corresponding to active sets $\{1,3,12,16,17\}$ and $\{1,3,6,12,14,16,17,20,21\}$, respectively. Their message is that using interior point methods and approximate arithmetic, the problems cannot be solved. Tomlab, using approximate arithmetic, solves the first of 13 iterations (Iterats) up to $-0.000021164$ with a minimum residual (ResMin) value of 0 and minimum multiplier (MulMin) value $-6.12\times {10}^{-13}$ and is not able to solve the second one.

The results of Sagitta are given in

Table 4. Settings and results for Stojković and Stanimirović [47] problems.

No	Setting	Optimal Value	Iterats	ResMin	MulMin
0	‘tnnnt’	−1.725308995636442$\times {10}^{-5}$	6	−3.73$\times {10}^{-9}$	+5.19$\times {10}^{-10}$
0	‘snnns’	+4.457393847569538$\times {10}^{-5}$	8	−7.45$\times {10}^{-9}$	−1.01$\times {10}^{-9}$
0	‘bnnnt’	+7.356378257536402$\times {10}^{-5}$	9	−1.49$\times {10}^{-8}$	−1.27$\times {10}^{-12}$
0	‘annns’	+1.560049165136415$\times {10}^{-5}$	8	−7.45$\times {10}^{-9}$	−1.01$\times {10}^{-9}$
0	‘tnsnt’	−1.725308995636442$\times {10}^{-5}$	6	−3.73$\times {10}^{-9}$	+5.19$\times {10}^{-10}$
0	‘snsns’	−2.117272130221947$\times {10}^{-5}$	10	−2.27$\times {10}^{-13}$	+1.62$\times {10}^{-9}$
0	‘bnsnt’	+7.356378257536402$\times {10}^{-5}$	11	−1.49$\times {10}^{-8}$	−1.27$\times {10}^{-12}$
0	‘ansns’	−2.115917629476000$\times {10}^{-5}$	17	+0.00$\times {10}^0$	−1.27$\times {10}^{-12}$
1	‘tnnnt’	−2.419638386181237$\times {10}^7$	9	−1.53$\times {10}^{-5}$	+1.00$\times {10}^0$
1	‘snnns’	−2.419638389627836$\times {10}^7$	12	−1.53$\times {10}^{-5}$	+4.01$\times {10}^{-4}$
1	‘bnnnt’	−2.419638415649983$\times {10}^7$	10	−3.82$\times {10}^{-6}$	+4.44$\times {10}^{-4}$
1	‘tnsnt’	−2.419638386181237$\times {10}^7$	9	−1.53$\times {10}^{-5}$	+1.00$\times {10}^0$
1	‘snsns’	−2.419638389627836$\times {10}^7$	12	−1.53$\times {10}^{-5}$	+4.01$\times {10}^{-4}$
1	‘ansns’	−2.419638396563628$\times {10}^7$	12	−7.63$\times {10}^{-6}$	+1.00$\times {10}^0$

. For the first problem with label 0, the setting ‘annns’ provides active set $\{1,3,10,12,14,16,17,20\}$ with small values for $y_{10}$, $y_{14}$, and $y_{20}$. The setting ‘ansns’ results in active set $\{1,2,3,10,12,16,17,20\}$ with small values for $y_2$, $y_{10}$, and $y_{20}$. For the second problem with label 1, Sagitta cannot find the optimal solution with settings ‘annns’ and ‘bnsnt’. Setting ‘ansns’ gives active set $\{1,3,6,12,14,16,17,20,21\}$.

We focus now on problems designed by Powell [48] to provide challenges for interior point methods

$$\begin{array}{ccc}\hfill min& x_2\hfill & ,x\in {\mathbb{R}}^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& cos(2k\pi /m)x_1+sin(2k\pi /m)x_2\ge -1\hfill & ,k\in \{1,\dots ,m\},\hfill \end{array}$$

with $m\ge 3$. We focus on the problem for $m\in \{3,4,\dots ,500\}$. We run Sagitta with tolerance $\sqrt{\epsilon }$ and settings o(1)=‘b’ and o(1)=‘t’ by using o(5)=‘t’ in both cases. For the cases where m is a multiple of 4, the algorithm requires only one iteration. For the cases where m is not a multiple of 4, the setting o(1)=‘b’ requires two iterations. This good behaviour was also predicted by Sherali et al. [49] and Künzi and Tzschach [50], Pan [51], as it fits the feasible set. However, the setting o(1)=‘t’, requires approximately ${log}_2(mA+B)$ iterations with $B\approx 10.2$ and $A\approx 0.9$. Notice that this increases to nine iterations for $m=499$.

Klee and Minty [52] started a worst-case example for the simplex method, where depending on the pivot rule chosen, the number of iterations is exponential in the dimension n of the problem. We focus on the description given by Paparrizos et al. [53] with three variants. For $0<\epsilon \le \frac{1}{3}$, the first formulation of the problem is

$$\begin{array}{ccc}\hfill max& \sum _{j=1}^n{\epsilon }^{n-j}{y}_j\hfill & ,y\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& y_1\le 1\hfill \\ & 2\sum _{j=1}^{i-1}{\epsilon }^{i-j}{y}_j+y_i\le 1\hfill & ,i\in \{2,\dots ,n\}\hfill \\ & y\ge 0.\hfill \end{array}$$

The second formulation follows from reordering the columns of matrix A:

$$\begin{array}{ccc}\hfill max& \sum _{j=1}^n{\epsilon }^{n-j}{y}_{2j-1}\hfill & ,y\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& y_1\le 1\hfill \\ & 2\sum _{j=1}^{i-1}{\epsilon }^{i-j}{y}_{2j-1}+y_{2i-1}\le 1\hfill & ,i\in \{2,\dots ,n\}\hfill \\ & y\ge 0.\hfill \end{array}$$

The third formulation adds a parameter $\mu :=\frac{1}{\epsilon }\ge 3$:

$$\begin{array}{ccc}\hfill max& \sum _{j=1}^n{\mu }^{n-j}{y}_j\hfill & ,y\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& 2\sum _{j=1}^{i-1}{\mu }^{i-j}{y}_j+y_i\le {\mu }^{2(i-1)}\hfill & ,i\in \{1,\dots ,n\}\hfill \\ & y\ge 0.\hfill \end{array}$$

We introduce a fourth variant (also with parameter $\mu$) in order to obtain the second formulation counterpart with the modified right-hand side. This seems to be a natural generalization, but was not explicitly given by Paparrizos et al. [53]:

$$\begin{array}{ccc}\hfill max& \sum _{j=1}^n{\mu }^{n-j}{y}_{2j-1}\hfill & ,y\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& 2\sum _{j=1}^{i-1}{\mu }^{i-j}{y}_{2j-1}+y_{2i-1}\le {\mu }^{2(i-1)}\hfill & ,i\in \{1,\dots ,n\}\hfill \\ & y\ge 0.\hfill \end{array}$$

In the procedure KleMinSp in the Appendix A, we extend to the possibility to convert the problem into standard form by adding dual variables in a negative or positive sense; e.g., KleMinSp(3,10,-3) is the dual of the problem appearing in [43] (p. 270) and in [14] (p. 61). We varied parameter $\mu \in \mathbb{N}$ and activation criteria. The results of the experiments are given in

Table 5. Settings for Klee and Minty [52] problems.

o(5)	o(1)	Mode	Number of Iterations (Iterats)
s	t	$\{-3,-4\}$	$n+\left\{\begin{array}{cc}{2}^n-1\hfill & ,\mathrm{for} \mu \ge 5\hfill \\ 2^{n-1}\hfill & ,\mathrm{for} (\mu =3)\vee (\mu =4)\hfill \end{array}\right\}$
s	t	$\{3,4\}$	$n+\left\{\begin{array}{cc}0\hfill & ,\mathrm{for} \mu \ge 5\hfill \\ 2^{n-1}-1\hfill & ,\mathrm{for} (\mu =3)\vee (\mu =4)\hfill \end{array}\right\}$
s	b	$\{-3,-4\}$	$n+\frac{1}{3}(2^{n+1}-\left\{\begin{array}{c}1,\mathrm{for} n\mathrm{is} \mathrm{odd}\hfill \\ 2,\mathrm{for} n \mathrm{is} \mathrm{even}\hfill \end{array}\right\})$
s	b	$\{3,4\}$	$n+\frac{1}{3}(2^n-\left\{\begin{array}{c}1,\mathrm{for} n\mathrm{is} \mathrm{odd}\hfill \\ 2,\mathrm{for} n \mathrm{is} \mathrm{even}\hfill \end{array}\right\})$
t	t	$\{-3,-4\}$	$n+\left\{\begin{array}{c}1,\mathrm{for} \mu \ge 5\hfill \\ 2,\mathrm{for} (\mu =3)\vee (\mu =4)\hfill \end{array}\right\}$
t	t	$\{3,4\}$	$n+\left\{\begin{array}{c}0,\mathrm{for} \mu \ge 5\hfill \\ 1,\mathrm{for} (\mu =3)\vee (\mu =4)\hfill \end{array}\right\}$
t	s	$\{\pm 3,\pm 4\}$	$n+3$
t	{b, a}	$\{-3,-4\}$	$2n$
t	{b, a}	$\{3,4\}$	$2n-1$
t	{t, s, b, a}	$\{\pm 1,\pm 2\}$	$nA+B(\epsilon :=\frac{1}{\mu }\to 0\Rightarrow (A\to 1)\wedge (B\to 0))$

.

It is clear that without normalisation of the residuals (Dantzig), several instances show an exponential number of iterations. Normalization provides for all combinations a linear number of iterations of an order between n and $2n$. The influence of the position taken by adding the dual slack variables is also interesting. With the setting o=‘tnnns’ for $\mu \ge 5$, with mode = −3, $n+2^n-1$ iterations are required. Using mode = 3 only requires n. Such differences may disappear with more sophisticated tie-breaking activation criteria. We use for tie-breaking the minimum index criterion (Bland).

Goldfarb [54] designed another deformation of the cube such that steepest-edge rules also require an exponential number of iterations. The parametric problem of Goldfarb [54], with $n\ge 3$, $\beta >2$ and $\delta >2\beta$, is given by

$$\begin{array}{ccc}\hfill max& (\beta {\alpha }_n-{\alpha }_{n-1})x_{n-1}+{\alpha }_n{x}_n\hfill & ,x\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& 0\le x_1\le 1\hfill & \\ & \beta x_1\le x_2\le \delta -\beta x_1\hfill & \\ & \beta x_j-x_{j-1}\le x_{j+1}\le {\delta }^j-\beta x_j+x_{j-1}\hfill & ,j\in \{2,\dots ,n-1\},\hfill \end{array}$$

with ${\alpha }_1=1$, ${\alpha }_2=\beta$ and ${\alpha }_{j+1}=\beta {\alpha }_j-{\alpha }_{j-1}$, $j\in \{2,\dots ,n-1\}$. The generation of the problem is provided by the procedure GoldfaSp in the Appendix A. The results obtained by Sagitta without scaling and with ‘dmperm’ as ordering method, option setting ‘tnnnt’, and tolerance $\sqrt{\epsilon }$ do not provide an exponential growth in the dimension of the number of iterations (see

Table 6. Results for the parametrized Goldfarb [54] problems.

n	$\mathit{\beta }$	$\mathit{\delta }$	Iterats	CntSecs
3	2	5	5	22
6	2	9	20	28
6	3	9	20	33
6	4	9	20	33
8	2	10	32	38
8	3	10	32	38
10	2	8	46	60
10	2	10	46	61
12	2	8	62	82
12	2	10	62	82

). The computational time is given in centiseconds (CntSecs), which represent hundredths of a second. This illustrates the requirement of careful numerical procedure and the recoveries.

As a final extreme case, we consider the work of Clausen [55] to design an exponential LP,

$$\begin{array}{ccc}\hfill max& \sum _{j=1}^n{\alpha }^j{y}_j\hfill & ,y\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& 2\sum _{j=1}^{i-1}{\beta }^{i-j}{y}_j+y_i\le {\gamma }^{i-1}\hfill & ,i\in \{1,\dots ,n\}\hfill \\ & y\ge 0,\hfill \end{array}$$

with $\alpha =4/5$, $\beta =5/4$, and $\gamma =5$. The idea of the designed problem is to have an exponential number of iterations when using the slack basis for both the simplex method for LPs in standard form and the simplex method for LPs in inequality form applied to the dual of the given problem. The procedure ClauseSp in the Appendix A generates the parameters. Let c, A, and b be the data of the original problem $max\{b^{T}y:Ay\le c,y\ge 0\}$ and let $\overline{c}$, $\overline{A}$, and $\overline{b}$ be the data of the problem $max\{{\overline{b}}^T\overline{y}:\overline{A}\overline{y}=\overline{c},\overline{y}\ge 0\}$ at the output of the routine; then we can select the following variants:

• If mode=1, then $\overline{b}={(b^T,0)}^T$, $\overline{A}=[A,I_n]$ and $\overline{c}=c$.
• If mode=0, then $\overline{b}={(-c^T,0)}^T$, $\overline{A}=[-A^T,I_n]$ and $\overline{c}=-b$.
• If mode=2, then $\overline{b}={(-c^T,0)}^T$, $\overline{A}=[-A^T,I_n]$ and $\overline{c}=b$.
• If mode=3, then $\overline{b}={(-b^T,0)}^T$, $\overline{A}=[A,I_n]$ and $\overline{c}=c$.

The variants facilitate applying Sagitta to problem $min\{{\overline{c}}^{T}x:{\overline{A}}^{T}x\ge \overline{b}\}$ in unfavourable circumstances for the simplex method for LP in standard form (if mode=1) and for the simplex method for LP problems in the inequality form (if mode=0). The other modes play the same role with the original problem of minimization instead of maximization, as noticed by Murty [56].

The results obtained by neither scaling nor preordering and with tolerances to $\sqrt{\epsilon }$ are shown in

Table 7. Settings for Clausen [55] problems, $u_n$ and $\Delta u_n$ according to (5).

o(5)	o(1)	Mode	Number of Iterations (Iterats)
{s, t}	{s, t, a, b}	0, 2	$n,n$
t	t	1, 3	$n+1,n+2$
t	{a, b}	1, 3	$2n-1,2n$
s	t	1, 3	$u_n-1,u_n$
s	{a, b}	1, 3	$1+u_{n-1}+\Delta u_{n-1}$, $u_n+\Delta u_n$

for varying values of o(5) and o(1). We use for the number of iterations

$$u_t:=2^{t-1}+t,\Delta u_t:=\frac{1+\left(\right(t+1\left)\mathrm{mod}2\right)}{3}\left(4^{\lfloor \frac{t-3}{2}\rfloor +1}-1\right).$$

(5)

The number of iterations for modes 0 and 2 on the one hand, and 1 and 3 on the other hand are very similar, separated by a comma in the last column. The last two experiment rows of the table reflect exponential behaviour. For the first three rows, the behaviour of Sagitta with o(3)=n was surprisingly polynomial, requiring at most $n+2$ iterations by using option setting ‘tnnnt’ in each mode.

3.3. Dealing with Sparse Instances

Preliminary experiments were carried out on classical problems with sparse matrices to compare the Sagitta sparse treatment with the one used by TomLab LpSolve v3.0. Initial problems of interest at the time of the appearance of the simplex method [57] (first published in) in 1947 were diet problems, such as the favorite Stigler 9-variable, 77-constraint [58] (p. 3), [59] (pp. 551–567) problem. Instead, we focus on the problems by Andrus and Schäferkotter [60] (p. 582), consisting of

$$\begin{array}{ccc}\hfill max& \sum _{i\in \{1}^n(3i-1)x_i\hfill & ,x\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& 1\le 2x_i+x_{i+1}\le 2\hfill & ,i\in \{1,\dots ,n-1\}\hfill \\ & 1\le 2x_n\le 2.\hfill \end{array}$$

Procedure AndSchSp in the Appendix A generates the problem data given the chosen input parameters. The number of iterations and the computation time (in centiseconds) for different values of n can be found in

Table 8. Andrus and Schäferkotter [60] diet problems.

	TomLab v3.0		Sagitta v1.1
n	Iterats	CntSecs	Iterats	CntSecs
50	51	258	50	60
100	101	807	100	126
200	201	4048	200	379
500	501	52,531	500	1972
1000	•	•	1000	8634
2500	•	•	2500	79,296

. We chose ‘dmperm’ as the ordering option as it allows A to be banded, and we did not scale ‘noscale’ the problem, using default options for Sagitta with all tolerances to $\sqrt{\epsilon }$. LpSolve needs to perform $n+1$ iterations and was unable to solve problems with $n=1000$ and $n=2500$ after 9 h of execution due to time limits and memory problems. Sagitta uses typically n iterations and took 20 s compared to almost 6 min that LpSolve needed to solve the problem with $n=500$. The problems with $n=1000$ and $n=2500$ were solved in almost a one and a half minutes and just over 13 min, respectively. Note that Sagitta needed n iterations, whereas TomLab needed $n+1$.

A widely used class of problems to test the performance of a new LP method is due to Quandt and Kuhn [61]:

$$\begin{array}{ccc}\hfill max& e^{T}y\hfill & ,y\in {\mathbb{R}}^n\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Qy\le {10}^{4}e\hfill & ,Q\in {\mathbb{N}}^{n\times n}\hfill \\ & y\ge 0,\hfill \end{array}$$

with e the all-ones vector and $q_{ij}\in {\mathbb{N}}^{+}$ randomly chosen in $\{1,2,\dots ,{10}^3\}$. This causes matrix Q to be dense. The procedure KunQuaSp in the Appendix A generates the data of the problem from a nonsingular sparse Q of density ${10}^{-5}$ and condition number 20 by means of round(1000*sprand(n,n,1e-5,1/20)), such that $\left|Q\right|\approx n$. Running the Sagitta routine, we choose ‘colmmd’ as the ordering option and not to scale ‘noscale’ the problem and used default option settings. For each value of n, 10 problems were solved; the average number of iterations and the average computation time (in hundredths of a second) for different values of n are reported in

Table 9. Sparse Quandt and Kuhn [61] problems.

	TomLab v3.0		Sagitta v1.1
n	Iterats	CntSecs	Iterats	CntSecs
50	51	215	50	43
100	101	835	100	86
250	251	8199	250	400
500	501	60,136	500	1531
1000	•	•	1000	6660
2500	•	•	2500	59,098

. Note that as before, Sagitta requires n iterations whereas TomLab needs $n+1$.

Chen et al. [62] (§8) presented a new primal-dual method and coded it in Matlab. To test it, they generated sparse random testing problems where the constraints are tangential to the unit ball. Consider a matrix $A\in {\mathbb{R}}^{n\times m}$ with Gaussian-distributed elements generated in Matlab according to sprandn(n,m,dens,1). Consequently, A is a nonsingular, well-conditioned matrix with random normally distributed elements. Moreover, let index number g be uniformly selected from $\{1,2,\dots ,m\}$. In the Appendix A, procedure ChPaSaSp generates a sparse problems of the form

$$\begin{array}{ccc}\hfill max& y_g\hfill & ,y\in {\mathbb{R}}^m\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \sum _{j=1}^m{A}_{ij}{y}_j=1\hfill & ,i\in \{1,\dots ,n\}\hfill \\ & y\ge 0.\hfill \end{array}$$

As in Chen et al. [62] (§8), for each of the combinations of interest of number of rows n, number of columns, m and density d, we generate blocks of 20 problems and average the number of required iterations. Moreover, we also average the elapsed time (in hundredths of a second). In Sagitta, we set ‘colmmd’ as the ordering option, and we do not scale ‘noscale’ the problem by using default options.

As combinations of interest we first chose the same combinations as Chen et al. [62] §8, and obtained the results shown in

Table 10. High-density sparse problems of Chen et al. [62].

			TomLab v3.0		Sagitta v1.1
m	n	d	Iterats	CntSecs	Iterats	CntSecs
50	25	0.3	39	95	39	117
100	25	0.3	46	110	46	162
150	25	0.3	53	133	50	191
200	25	0.3	54	145	49	196
100	50	0.1	71	271	67	381
150	50	0.1	83	328	79	594
200	50	0.1	92	379	89	774
150	100	0.1	137	1041	128	2686
200	100	0.1	158	1237	153	3975
200	150	0.1	210	2673	185	8547

. One can observe that, although Sagitta had a slight advantage in terms of number of iterations, TomLab is clearly faster. This is due to the fact that the problems are not large ($n\le 150$) and above all due to the fact that the maximum skeleton L is not sparse at all, i.e. $\left|L\right|$ is between $75\%$ and $85\%$.

To know the behavior for sparser combinations, i.e., $\left|L\right|$ does not exceed $10\%$, we reran the experiments for such instances. The results are given in

Table 11. Low-density sparse problems of Chen et al. [62].

			TomLab v3.0		Sagitta v1.1
m	n	d	Iterats	CntSecs	Iterats	CntSecs
50	25	0.06	27	73	27	24
100	25	0.06	30	85	30	40
150	25	0.06	31	92	30	53
200	25	0.06	32	101	31	68
100	50	0.02	51	222	50	37
150	50	0.02	51	233	51	40
200	50	0.02	51	243	51	41
150	100	0.02	103	885	102	288
200	100	0.02	107	938	103	357
200	150	0.02	161	2339	154	1865
500	250	0.001	251	8333	250	400
750	500	0.001	•	•	500	1374
1500	1000	0.0001	•	•	1000	5706

. The results of these sparse experiments show that that Sagitta has a slight advantage in terms of number of iterations, but is also faster. The dimension $n=250$ is the largest of those solved with TomLab. It took Sagitta 4 s to solve the problem compared to the 83 s required by TomLab. The two largest problems were solved by Sagitta in n iterations requiring 14 and 57 s, respectively.

4. Discussion

A code has been presented for linear programming to take care of sparsity and basis deficiency. Extreme cases have been taken from the literature, and the performance has been investigated by using a commercial Matlab solver Tomlab as benchmark. We structured this article as shown in Figure 1 and found that

• the code is numerically robust (cf. Section 3.2), despite the fact that it dispenses with the orthogonal factor of the QR factorisation and in no case uses more than one iterative refinement step;
• the computational effort in choosing suitable pivoting rules is quite encouraging; and
• the code reveals advantages for problems that are really sparse (cf. Section 3.3), both in terms of iterations and computational time, compared to a commercial software with comparable features.

Questions on the simplex method originate from its beginning in 1947. However, there is still a lot of ongoing research; in the last three years one can find many publications, including [1,2,3,4,5,6,7,8], just to cite a few. Although they mainly focus on the simplex method (the second volume of forthcoming renewed edition of Pan’s book [8] being the exception), some of the ideas they pointed out could be of interest to be incorporated into a deficient basis framework, thus reducing even more the necessary number of iterations.