AN OBJECT-ORIENTED APPROACH TO SEMIDEFINITE PROGRAMMING

An object-oriented design and implementation of a primal-dual algorithm for solving the semidefinite programming problem is pn~sented. The advantages of applying the object-oriented methodology to numerical computations, in particular to an interior point algorithm for semidefinite programming, or for solving other types of linear matrix inequalities are discussed. One object-oriented design of the primal-dual algorithm and its implementation using C++ is presented. The performance of the C++ implementation is compared with that of a procedural C implementation, and while the performance of the C++ implementation is comparable to that of the C implementation, the resulting code is easier to read, modify, and maintain. Object-oriented

The third advantage is code reuse.Inheritance enables the definition of a new object, which can be viewed as a subclass of an existing object, without having to repeat some of the details.The new object can inherit attributes or operations from its ancestor.Inheritance is one way to support reuse of existing objects.There are different kinds of reuse in object-oriented programming; inheritance is only one of them.
One of the most popular object-oriented programming languages is C++ [11], which is used to implement the algorithm of this paper.Some of the reasons why C++ is so widely used are upward compatibility with C, design emphasis on efficiency and performance, and the availability of many useful libraries and tools.For instance, the Gnu C++ compiler and other tools are available on a wide range of platforms and provide good performance, programming environments, and reasonable compliance with ANSI standards.
There are many available libraries such as IML++ [6]' SparseLib++ [5] [9], STL[10] [8], and others which emphasize numerical computation.One notable package is LAPACK++, developed by Dongarra et al. [4], which is a C++ interface to LAPACK and BLAS.[4] has show~,that performance of programs using the package is comparable to calling LAPACK and BLAS directly, and can at the same time reap the benefits of object-oriented programming.
This paper contains the result of object-oriented design and implementation of an algorithm for semidefinite programming.Semidefinite programming refers to minimizing a linear function subject to a linear matrix inequality [12].That is, Many problems in controls engineering can be cast in terms of a semidefinite programming problem [12].Since a semidefinite programming problem is a convex optimization problem, which can be solved by interior point methods [7], it has attracted the attention of many researchers in interior point methods.There is a C implementation of a primal-dual algorithm for solving the semidefinite programming problem [13].A C++ implementation of that primal-dual algorithm for the semidefinite programming problem is developed here to explore the possible benefits of object-oriented design.Because of the similarity of the primal-dual algorithm with other interior point algorithms for solving the semidefinite programming prbblem, the design and implementation methodology developed here can be easily modified and applied to other interior point algorithms.
The performance of a C++ implementation of the primal-dual algorithm for semidefinite programming is compared with the existing C implementation of the same algorithm from [13].While the CPU times of the two implementations are comparable to each other, the C++ version offers the advantages mentioned earlier in this section.Segments of the code will be used to illustrate object-oriented features of the implementation.
Section 2 briefly sketches the primal-dual algorithm for semidefinite programming that will be used to illustrate the object-oriented design methodology.In Section 3, the details of an object-oriented design of the primal-dual algorithm will be given.The implementation using C++ will be described in Section 4. Comparison and discussion of the C and C++ results will be given in Section 5.
The primal-dual algorithm for solving the semidefinite programming program given in detail in [12] will be described briefly here.The dual problem associated with the semidefinite program (1) is maXImIze -tr FoZ ZER"X" subjecttotrFiZ=ci, i=1, ... ,m Z = ZT, Z ?:: o.
The primal-dual method can be inter-preted as solving the primal-dual optimization problem mllllmlze c T x + tr FaZ xERm,ZER"X" subject to tr FiZ = Ci, i = 1, ... , m F(x) ?:: 0, Z ?:: 0, Z = ZT, where the objective function c T x + tr FoZ == 1] is ca.lled the duality gap, which has the known optimum value of zero for a convex problem.The advantage of using the primaldual formulation is that at each step information from the dual problem can be used to obtain a good update for the primal variables.
One of the methods to solve the primal-dual optimization problem is the potential reduction method.Define a potential function where 1/ ?:: 1 is a weighting parameter in the potential.The duality gap is ry ~exp (l/~) , which approaches 0 as the potential function </> approaches -00.
The whole algorithmic process can be described as follows.Starting from a strictly feasible Xo and Zo, find Xk and Zk so that the potential </> is reduced at each step by at least a fixed amount 8 > 0, until the duality gap ry is smaller than some specified € > 0. The first Xk and Zk which make ry < € constitute the approximate numerical solution of the primal-dual problem.
The primal-dual potential reduction method for solving the semidefinite program (1) can be summarized as follows.Given strictly feasible x and Z, while duality gap ry = cTx + tr PoZ,> € do 1.compute a direction 8x for the primal variable x and a direction 8Z for the dual variable Z; 2. find p, q that minimize </>(x+ p8x, Z4+ q8Z), where (p, q) are constrained to some search rectangle• in the plane; 3. update x := x + p8x and Z := Z + q8Z.
Details for each of these steps are given in [12].
In C++ terminology, the word "class" is used to mean a type of object.For example, a class Symm can be defined for sylllllwtric matrices, while a specific symmetric matrix is an object of type Symm or an instanv' or the class Symm.C++ terminology will be followed in the rest of the paper.
Object-oriented analysis and design is one of the most active research areas for both academics and industrial practitioners.When applied in different circumstances, different analysis and design techniques may be emphasized.One of the most influential analysis and design techniques is due to Booch[l), wllose conventions will be loosely followed in this paper.
First we will describe the classes used in the design and their relationship.The class diagram for the problem is shown in Fig. 1, where all the classes are defined with their aHributes and functions.For clarity, only main attributes and functions are shown and the functiolls narnes may be different from that used in real implementation.ln Fig.This code declares prime as an instance of the class SpPrime, and dual as an instance of the class SpDual.
An arrow -+ from SpSym to SpDual denotes that SpSym is a subclass of SpDttal and inherits the public and protected attributes and functions from SpDual.An upside-down triangle with A ill SpDual denotes that SpDual is an abstract class.Many functions in SpDual are defined as virtual so that dynamic binding is used for these functions, Le., the decision Oll which implerpentation to use is made at run time, as it cannot be determined at compile Lime., Consequently, if a class other than SpSym is used to implement SpDual, then the run time system will choose the right function depending on the parameters passed.
The symmetry between the primal and dual spaces in the objective function in (3) suggests that these two spaces should be fundarr:ental cla'3ses in the problem, and so two classes, SpPrime and SpDual, are defined.SpPrime is relatively simple.The primal variable x is the independent variable.The only important actions on the class are to calculate c T x and update x.The class SpDual will contain more functions and is more complicated.It will not only calculate trFoZ, but also compute oZ, p, q, and update Z.The dual variable Z is a symmetric matrix.In the future we may want to exploit any special structure the matrix Z may have.For example, we could exploit the sparsity of the matrix Z by defining and Ilsing another class SpSparse.So SpDual is designed as an "envelope" class [3] to provide a generic dual space interface and to isolate it from the "letter" class that implements the dual space for a particular representation.
As of this writing, we implemented SpSym, a symmetric matrix representation not exploiting any other special structure that the matrix may have.' SpAlgo is the base class for algorithm implementation.
It mediates the communication between SpPrime and SpDual, executes the computational tasks, and changes the "states" of the SpPrime and SpDual objects.
Finally main is a utility class, a "free" program not tied to any particular class, that stores the convergence criterion and is the driver for the algorithm.It is denoted with a dotted cloud with a shadow in Fig. 2 and Fig. 3.
The majo~advantage of this way of thinking is that the implementation of the dual space is completely independent of the program structure and state transition.This is desirable because the dual space implementation is where algorithmic changes are likely to occur, and this separation of the interface and implementation localizes changes in the program.The envelope class SpDual provides a clean and effective interface for the dual space; SpSym is one of the possible subclasses of ;SpDual that does nothing but implement the specification of SpDual.The matrix and vector classes are from LAPACK++, and are not shown in the diagram.
There are two execution scenarios, whose outlines are shown in Figs. 2 and 3, corresponding to different stages of the primal-dual algorithm.
A small square with F inside on the border of SpDual and SpPrime on the line from SpAlgo denotes that SpPrime and SpDual are part of the client object, SpAlgo.The two square boxes with L inside on the border of SpAlgo on the line from main denote that SpAlgo is a locally declared object in main.An arrow with an empty circle indicates that results or services are returned to the object pointed to by the arrow.An arrow denotes the direction of a message from one object to another.The number in front of the message denotes the execution sequence of the message.
In the first scenario (Fig. 2): Step 1.If a search rectangle exists, main sends a message to SpAlgo to do a plane search, Le., find p and q such that ¢(x + pox, Z + qoZ) is minimized, where p and q are constrained in the se~rch rectangle.Otherwise main sends a message to SpAlgo to calculate the duality gap.
Step 2. SpAlgo sen,ds a message to SpPrime to get the primal variable.
Step 3. SpAlgo sends the primal variable to SpDual along with a message to calculate the duality gap.Step 4. main chec~s the convergence criterion.If the criterion is met, execution stops.Otherwise, go to the second scenario.
In the second scenario (Fig. 3): Step 1. main sends a message to SpAlgo to do a potential reduction calculation.
Step 2. SpAlgo sends a message to SpDual, which returns dx, the increment for the primal variable.
Step 3. SpAlgo seri'ds dx to SpPrime to store it in SpPrime.
Step 4. main sends a message to SpAlgo to calculate the search rectangle.
Step 5. SpAlgo sends a message to SpPr'ime to calculate c T dx.
Step 6. SpAlgo sends c T dx along with message SearchRect( ... ) to SpDual to calculate and return several intermediate variables that mark the search rectangle.
Step 7. main checks the convergence criterion using the corners of the search rectangle.If the convergence criterion is met, execution stops.Otherwise, go to the first scenario .

THE C:+'+ IMPLEMENTATION
The program is built upon the LAPACK++ v1.0 package, especially the LaVectorDouble, LaGenMat, LaSymmMat classes, and BLAS++ is used extensively.Several special purpose routines are added to the LAPACK++ package to accommodate the primal-dual algorithm for semidefinite programming.The program also uses the iterator object in STL (Standard Template Library)[lO] [8] to traverse arrays of objects.The first major difference between the C and C++ implementations is the way the initial data is read in.Unli~te C /Fortran style subroutL1!::.3,in which one can pass a pointer/address for a piece of storage and let the subroutine split the storage into pieces to get the data., C++ objects' constructors have no such scheme.Initialization is done by reading a data -file.The second major difference is that because C++'s objects are higher level abstractions, the implementation in C++ is less dependent upon pointer arithmetic, as shown by the code segments in C and C++ (Figs. 4 and 5) for doing the same computation.There is overhead.assoCiated with this higher level of abstraction, but we will show that the effect on performance is negligible.
Two sets of data.are obta.inedby randomly generating all the matrices Fa, F1, .. " l'~n, and the vector c.Strictly feasible init.icd points :1;0 and Zo are also generated.The timing results are shown in Table L All the timings ,ue done on a HP 712/60 workstation.Both the C irnplementatioll froln [13] and the C++ implementation are compiled using the Gnu C/C++ compiler version 2.7.2 with the same compiler options.It is clear from Table 1 that the performance penalty [or using C++ is only a few percent and decreases as the problem size increases.We have shown an objected-oriented design and implementation of a semidefinite programming algorithm.Even though object-oriented technology is being used more a.nd more widely in industry now, there are not many realistic applications to numerical computation.The programming environments and tools seem to be mature enough to apply this new methodology, and the performance seems to be comparable to a non-object-oriented implementation.
However, :there are other considerations that have to be taken into account when applying object-oriented technology.First, it takes time and effort to learn the new methodology.
Second, it is not a trivial task to set up the environment: compiling all the C++ packages, and verifying that they work correctly, especially when most of the C++ packages for numerical computation are still in the testing stage.Third, the resulting code size, Le., the size of the C++ executable, is about 2.5 times that of the C executable.With continuing development of object-oriented technology and of compilers for object-oriented languages, the second problem, will likely be alleviated.
The hardware considerations of the third problem are becoming less of a hindrance with the advances in the computer industry.We do believe that the benefits of using object-oriented methodology outweight the currently existing disadvantages.' The design and analysis in this paper can be generalized to apply to object-oriented design and implementation of other interior point algorithms which use potential reduction to find the optimum.
i=l c E Rm, and Fo, ... , Fm E Rnxn are symmetric matrices.F(x) ~0 means that F(x) is positive semidefinite.

} 1 *
/scal +rho*scal) * sq2; loop over 1'1, compute V' * Fi * V, store it in Fsc *1 for ( vector < LaVectorDouble>::iterator i = Fi->begin()+l; i < Fi->end(); i++, n++) LaVectorDouble tmp(Fsc.addr()+pd_sz*n, 1, pd_sz); DualScale(O, *i, vecx, trnp); 1* correct diagonal elements *1 for ( int k = 0, pos=O; k < j; pos += j-k, k++) tmp(pos) *= sq2; } LaLeastSquare(dx, Fsc, &:n); 1, each dotted cloud-shaped figure describes a class, with the private members uf the cla.':ispreceded by II and the protected members of the class preceded by I. Protected properties are inherited by and accessible to subclasses, whereas p1'ivate properties are not.All other properties are considered public, i.e., accessible by other classes.Protected and private properties are not accessible to other classes.For example, in the class SpPrime, the private attributes are prime_variable x, matrix c, and increment dx, the main public functions are cTx() which computes c T x, cTdx() which computes c T dx, Dx() which computes dx, and update() to update x.The connection e--.from SpAlgo to SpPrime denotes the physical containment of SpPrime in SpAlgo, while the connection 8--0 from SpAlgo to SpDttal denotes a pointer reference to the class SpDual by SpAlgo, which can be illustrated by the corresponding part of the definition of SpAlgo