A Projection and Contraction Method for Nonparallel Hyperplane Circular Cone Programming Support Vector Machine

Abstract

A second-order cone programming support vector machine (SOCP-SVM) method is proposed to realize the binary classification faster and better. This method combines the advantages of better classification performance of nonparallel hyperplane second-order cone programming support vector machine (SOCP-NHSVM), and represents SOCP-NHSVM as a convex quadratic nonparallel hyperplane circular cone programming support vector machine (CQCCP-NHSVM), and finally uses the projection and contraction method to solve it. Experiments on the benchmark dataset of the UCI repository show that the proposed method can improve the computational efficiency and obtain the accuracy value, F-measure value and G-mean value, which are similar to the SOCP-NHSVM of the linear classifier. In the case of kernel-based nonlinear classification, the proposed method can achieve similar performance under three model evaluation indexes, and greatly shorten the running time of the program.

1. Introduction

Recently, not only has second-order cone programming (SOCP) been used as an alternative optimization scheme for support vector machine (SVM) [1], but a robust formula following the principle of maximum margin of SVM has been derived [2,3]. The goal of this work is to construct a model that can correctly classify most instances of each training mode, given the mean and covariance matrix, even under the worst distribution of class conditional density. These methods have been proved to be very effective in classification performance [2]. In order to solve the SOCP-SVM problem conveniently, some scholars package the classical algorithm of the SOCP problem as a solver, such as the SeDuMi toolbox in MATLAB R2023a [4], and its principle is to solve the SOCP problem by using the primal-dual interior point method.

At the same time, in order to further improve the classification accuracy of the SOCP-SVM model, twin support vector machine, nonparallel support vector machine and other models are proposed as optimization and improvement schemes for SVM. Maldonado and others extended the traditional twin SVM and put forward a twin SOCP-SVM [5], gave the linear and nonlinear twin SOCP-SVM models, and proved their dual formula. Reference [6] extends the twin SVM and obtains a nonparallel hyperplane support vector machine (NHSVM). By solving a single problem instead of two problems, two classifiers are obtained, thus obtaining better prediction performance. At the same time, reference [7] extends the idea of NHSVM to the second-order cone and obtains better classification performance. In these papers, both the new SOCP-SVM formula and the improvement of the existing SOCP-SVM formula is shown to improve the model classification accuracy to a certain extent, but these methods always use the primal-dual interior point method to solve the objective problem [8,9]. We need to find another way to solve the SOCP-SVM formula.

To solve this problem, this paper reconstructs the linear SOCP-NHSVM formula and the kernel-based SOCP-NHSVM formula into two CQCCCP-NHSVM formulas, and solves CQCCCP-NHSVM by the projection and contraction method. Although the new solution needs to reformulate the SOCP problem as a circular cone programming problem, the transformed circular cone programming problem reduces the scale of the original optimization problem, that is, the scale of CQCCP-NHSVM solved by the projection and contraction method is smaller than that of SOCP-NHSVM solved by SeDuMi. For large-scale data classification problems, the new solution improves the computational efficiency of SOCP-NHSVM.

The structure of this paper is as follows: Section 2 introduces the basic formula of NHSVM and its second-order cone form. In Section 3, the SOCP-NHSVM formula is reconstructed into a nonparallel hyperplane circular cone programming support vector machine. Section 4 gives the projection and contraction method of CQCCP-NHSVM. Section 5 gives the experimental results of relevant benchmark datasets. The main conclusion of this paper is in Section 6.

2. Basic Formulation

2.1. NHSVM

SVM is a classification algorithm; for the linearly divisible case, it constructs a classification hyperplane $f\left(\mathbf{x}\right)={\mathbf{w}}^{\mathrm{T}}\mathbf{x}+b$ by learning from the training data. The new data point $\overline{\mathbf{x}}$ is then classified. If $f\left(\overline{\mathbf{x}}\right)>0$, it belongs to a positive example, i.e., $\overline{y}=+1$; if $f\left(\overline{\mathbf{x}}\right)<0$, it belongs to a negative example, i.e., $\overline{y}=-1$. Suppose the training set is $T=\{({\mathbf{x}}_1,y_1),\dots ,({\mathbf{x}}_m,y_m)\}$, where ${\mathbf{x}}_i\in {\Re }^n,y_i\in \{-1,+1\},i=1,\dots ,m$, m denote the number of data points. The set of positive classes is represented by $A\in {\Re }^{m_1\times n}$, and the set of negative classes is represented by $B\in {\Re }^{m_2\times n}$, where $m_1+m_2=m$.

The linear NHSVM formula finds two hyperplanes in ${\Re }^n$, which makes each classifier close to one of the training modes and as far away from the other as possible, and the geometric explanation of each training mode can be represented by a simplified convex hull [7]. Compared with twin support vector machine, NHSVM only needs to solve a single quadratic programming problem and construct two nonparallel hyperplanes for classification. The linear NHSVM formula is as follows

$$\begin{array}{cc}\hfill \underset{{\displaystyle \genfrac{}{}{0pt}{}{{\mathbf{w}}_i,b_i,{\xi }_i\hfill }{i=1,2\hfill }}}{min}& {\displaystyle \frac{1}{2}}{∥A{\mathbf{w}}_1+{\mathbf{e}}_1{b}_1∥}^2+{\displaystyle \frac{1}{2}}{∥B{\mathbf{w}}_2+{\mathbf{e}}_2{b}_2∥}^2\hfill \\ \hfill & +{\displaystyle \frac{c_1}{2}}({∥{\mathbf{w}}_1∥}^2+b_1^2+{∥{\mathbf{w}}_2∥}^2+b_2^2)\hfill \\ & +{\displaystyle \frac{c_2}{2}}({\mathbf{e}}_1^{\mathrm{T}}{\xi }_1+{\mathbf{e}}_2^{\mathrm{T}}{\xi }_2)\hfill \\ \hfill s.t.& A{\mathbf{w}}_1+{\mathbf{e}}_1{b}_1-A{\mathbf{w}}_2-{\mathbf{e}}_1{b}_2\ge {\mathbf{e}}_1-{\xi }_1,\hfill \\ \hfill & B{\mathbf{w}}_2+{\mathbf{e}}_2{b}_2-B{\mathbf{w}}_1-{\mathbf{e}}_2{b}_1\ge {\mathbf{e}}_2-{\xi }_2,\hfill \\ & {\xi }_1\ge 0,{\xi }_2\ge 0,\hfill \end{array}$$

(1)

where ${\xi }_1\in {\Re }^{m_1},{\xi }_2\in {\Re }^{m_2}$ is the relaxation variable of their respective classes, $c_1,c_2>0$ is the regularization parameter, and ${\mathbf{e}}_1,{\mathbf{e}}_2$ is the vector of appropriate dimension. According to the proximity of the new data point $\mathbf{x}\in {\Re }^n$ to the hyperplane, the new data point $\mathbf{x}$ is assigned to ${\mathbf{k}}^{\ast }$, ${\mathbf{k}}^{\ast }=\underset{i=1,2}{argmin}\left\{d_i\left(\mathbf{x}\right):={\displaystyle \frac{\left|{\mathbf{w}}_i^{\mathrm{T}}\mathbf{x}+b_i\right|}{∥{\mathbf{w}}_i∥}}\right\}$, where $d_i\left(\mathbf{x}\right)$ corresponds to the vertical distance between the data sample $\mathbf{x}$ and the hyperplane ${\mathbf{w}}_i^{\mathrm{T}}\mathbf{x}+b_i=0,i=1,2$.

Through the calculation of the Lagrange condition and the Karush–Kuhn–Tucker(KKT) condition, the dual formula of Formula (1) can be obtained [7]:

$$\begin{array}{cc}\hfill \underset{\mathsf{\alpha }}{max}& {\mathbf{e}}^{\mathrm{T}}\mathsf{\alpha }-{\displaystyle \frac{1}{2}}{\mathsf{\alpha }}^{\mathrm{T}}{\tilde{A}}^{\mathrm{T}}{(H^{\mathrm{T}}H+c_{1}I)}^{-1}\tilde{A}\mathsf{\alpha }\hfill \\ & -{\displaystyle \frac{1}{2}}{\mathsf{\alpha }}^{\mathrm{T}}{\tilde{A}}^{\mathrm{T}}{(G^{\mathrm{T}}G+c_{1}I)}^{-1}\tilde{A}\mathsf{\alpha }\hfill \\ \hfill s.t.& 0\le \mathsf{\alpha }\le c_2\mathbf{e},\hfill \end{array}$$

(2)

where $H=[A,{\mathbf{e}}_1]\in {\Re }^{m_1\times (n+1)}$, $G=[B,{\mathbf{e}}_2]\in {\Re }^{m_2\times (n+1)}$, ${\tilde{A}}^{\mathrm{T}}=[H^{\mathrm{T}},-G^{\mathrm{T}}]\in {\Re }^{(n+1)\times m}$. Using this formula, the values of ${\mathbf{w}}_i$ and $b_i$ are calculated as ${\mathbf{v}}_1={(H^{\mathrm{T}}H+c_{1}I)}^{-1}{\tilde{A}}^{\mathrm{T}}\mathsf{\alpha }$ and ${\mathbf{v}}_2={(G^{\mathrm{T}}G+c_{1}I)}^{-1}{\tilde{A}}^{\mathrm{T}}\mathsf{\alpha }$, where ${\mathbf{v}}_i={[{\mathbf{w}}_i^{\mathrm{T}},b_i]}^{\mathrm{T}}\in {\Re }^{n+1},i=1,2$.

2.2. The Linear SOCP-NHSVM

Before introducing SOCP-NHSVM, we need to explain the traditional SOCP-SVM model proposed by Nath and Bhattacharyya [3]. Assuming that ${\mathbf{X}}_1$ and ${\mathbf{X}}_2$ are random vectors that generate positive and negative samples respectively, in order to construct a maximum marginal linear classifier, so that the probability of correctly classifying positive and negative samples is not less than ${\eta }_1\in (0,1)$ and ${\eta }_2\in (0,1)$, the following probability constrained optimization problems are considered:

$$\begin{array}{cc}\hfill \underset{w,b}{min}& {\displaystyle \frac{1}{2}}{∥\mathbf{w}∥}^2\hfill \\ \hfill s.t.& Pr\{({\mathbf{w}}^{\mathrm{T}}{\mathbf{X}}_1+b)\ge 1\}\ge {\eta }_1,\hfill \\ \hfill & Pr\{({\mathbf{w}}^{\mathrm{T}}{\mathbf{X}}_2+b)\le -1\}\ge {\eta }_2.\hfill \end{array}$$

(3)

Given its mean and covariance $({\mu }_i,{\mathsf{\Sigma }}_i),i=1,2$, where ${\mathsf{\Sigma }}_i\in {\Re }^{n\times n}$ is a symmetric positive definite matrix, Equation (3) can be expressed as

$$\begin{array}{cc}\hfill \underset{{\mathbf{X}}_1\sim ({\mu }_1,{\mathsf{\Sigma }}_1)}{inf}& Pr\{({\mathbf{w}}^{\mathrm{T}}{\mathbf{X}}_1+b)\ge +1\}\ge {\eta }_1,\hfill \\ \hfill \underset{{\mathbf{X}}_2\sim ({\mu }_2,{\mathsf{\Sigma }}_2)}{inf}& Pr\{({\mathbf{w}}^{\mathrm{T}}{\mathbf{X}}_2+b)\le -1\}\ge {\eta }_2,\hfill \end{array}$$

(4)

where $\mathbf{X}\sim (\mu ,\mathsf{\Sigma })$ refers to a distribution family with a common average $\mu$ and covariance $\mathsf{\Sigma }$.

Due to the proper application of multivariate Chebyshev inequality [8], Formula (4) is equivalent to

$$\begin{array}{c}\hfill {\mathbf{w}}^{\mathrm{T}}{\mu }_1+b\ge 1+{\kappa }_1\sqrt{{\mathbf{w}}^{\mathrm{T}}{\mathsf{\Sigma }}_1\mathbf{w}},\\ \hfill -({\mathbf{w}}^{\mathrm{T}}{\mu }_2+b)\ge 1+{\kappa }_2\sqrt{{\mathbf{w}}^{\mathrm{T}}{\mathsf{\Sigma }}_2\mathbf{w}},\end{array}$$

(5)

where ${\kappa }_i=\sqrt{{\displaystyle \frac{{\eta }_i}{1-{\eta }_i}}},k=1,2$. Substituting the constraint in Equation (3) leads to the following deterministic problem

$$\begin{array}{cc}\hfill \underset{w,b}{min}& {\displaystyle \frac{1}{2}}{∥\mathbf{w}∥}^2\hfill \\ \hfill s.t.& {\mathbf{w}}^{\mathrm{T}}{\mathbf{\mu }}_1+b\ge 1+{\kappa }_1∥S_1^{\mathrm{T}}\mathbf{w}∥,\hfill \\ \hfill & -({\mathbf{w}}^{\mathrm{T}}{\mathbf{\mu }}_2+b)\ge 1+{\kappa }_2∥S_2^{\mathrm{T}}\mathbf{w}∥,\hfill \end{array}$$

(6)

where ${\mathsf{\Sigma }}_i=S_i{S}_i^{\mathrm{T}},i=1,2$. We can also introduce a new variable t and constraint $∥\mathbf{w}∥\le t$, and transform Formula (6) into a constraint with linear objective function and three second-order cones [9]. These linear SOCP formulas can be effectively solved by the interior point method [9,10].

Using the above knowledge, in order to get two linear nonparallel hyperplanes, consider the following quadratic chance constrained programming formula

$$\begin{array}{cc}\hfill \underset{{\displaystyle \genfrac{}{}{0pt}{}{{\mathbf{w}}_i,b_i\hfill }{i=1,2\hfill }}}{min}& {\displaystyle \frac{1}{2}}{∥A{\mathbf{w}}_1+{\mathbf{e}}_1{b}_1∥}^2+{\displaystyle \frac{1}{2}}{∥B{\mathbf{w}}_2+{\mathbf{e}}_2{b}_2∥}^2+\hfill \\ & {\displaystyle \frac{\theta }{2}}({∥{\mathbf{w}}_1∥}^2+b_1^2+{∥{\mathbf{w}}_2∥}^2+b_2^2)\hfill \\ \hfill s.t.& \underset{{\mathbf{X}}_1\sim ({\mu }_1,{\mathsf{\Sigma }}_1)}{inf}Pr\{{\mathbf{X}}_1\in H^{+}({\mathbf{w}}_1-{\mathbf{w}}_2,b_1-b_2)\}\ge {\eta }_1,\hfill \\ \hfill & \underset{{\mathbf{X}}_2\sim ({\mu }_2,{\mathsf{\Sigma }}_2)}{inf}Pr\{{\mathbf{X}}_2\in H^{-}({\mathbf{w}}_1-{\mathbf{w}}_2,b_1-b_2)\}\ge {\eta }_2,\hfill \end{array}$$

(7)

where $\theta \ge 0$, $H^{+}(\mathbf{w},b):=\{\mathbf{x}:{\mathbf{x}}^{\mathrm{T}}\mathbf{w}+b\ge 1\}$ and $H^{-}(\mathbf{w},b):=\{\mathbf{x}:{\mathbf{x}}^{\mathrm{T}}\mathbf{w}+b\le -1\}$. Note that Equation (7) is equivalent to hard margin NHSVM. If the meaning of other parameters is the same as before, then we can get the following deterministic questions:

$$\begin{array}{cc}\hfill \underset{{\displaystyle \genfrac{}{}{0pt}{}{{\mathbf{w}}_i,b_i\hfill }{i=1,2\hfill }}}{min}& {\displaystyle \frac{1}{2}}{∥A{\mathbf{w}}_1+{\mathbf{e}}_1{b}_1∥}^2+{\displaystyle \frac{1}{2}}{∥B{\mathbf{w}}_2+{\mathbf{e}}_2{b}_2∥}^2+\hfill \\ & {\displaystyle \frac{\theta }{2}}({∥{\mathbf{w}}_1∥}^2+b_1^2+{∥{\mathbf{w}}_2∥}^2+b_2^2)\hfill \\ \hfill s.t.& {({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_1+(b_1-b_2)\ge 1+{\kappa }_1∥S_1^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2)∥,\hfill \\ \hfill & -{({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_2-(b_1-b_2)\ge 1+{\kappa }_2∥S_2^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2)∥.\hfill \end{array}$$

(8)

According to the symbolic representation mentioned in the dual formula in Section 2.1, Equation (8) can be written as follows:

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2}}{∥A{\mathbf{w}}_1+{\mathbf{e}}_1{b}_1∥}^2+{\displaystyle \frac{\theta }{2}}({∥{\mathbf{w}}_1∥}^2+b_1^2)={\displaystyle \frac{1}{2}}{\mathbf{v}}_1^{\mathrm{T}}(H^{\mathrm{T}}H+\theta I){\mathbf{v}}_1,\\ \hfill {\displaystyle \frac{1}{2}}{∥B{\mathbf{w}}_2+{\mathbf{e}}_2{b}_2∥}^2+{\displaystyle \frac{\theta }{2}}({∥{\mathbf{w}}_2∥}^2+b_2^2)={\displaystyle \frac{1}{2}}{\mathbf{v}}_2^{\mathrm{T}}(G^{\mathrm{T}}G+\theta I){\mathbf{v}}_2.\end{array}$$

(9)

By introducing new variables $t_1,t_2$ and constraints $∥{(H^{\mathrm{T}}H+\theta I)}^{1/2}{\mathbf{v}}_1∥\le t_1$ and $∥{(G^{\mathrm{T}}G+\theta I)}^{1/2}{\mathbf{v}}_2∥\le t_2$, Equation (8) can be transformed into the following linear SOCP problem:

$$\begin{array}{cc}\hfill \underset{{\displaystyle \genfrac{}{}{0pt}{}{{\mathbf{w}}_i,b_i,t_i\hfill }{i=1,2\hfill }}}{min}& t_1+t_2\hfill \\ \hfill s.t.& ∥{(H^{\mathrm{T}}H+\theta I)}^{1/2}{\mathbf{v}}_1∥\le t_1,\hfill \\ \hfill & ∥{(G^{\mathrm{T}}G+\theta I)}^{1/2}{\mathbf{v}}_2∥\le t_2,\hfill \\ \hfill & {({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_1+(b_1-b_2)\ge 1\hfill \\ & +{\kappa }_1∥S_1^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2)∥,\hfill \\ \hfill & -\left({({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_2+(b_1-b_2)\right)\ge 1\hfill \\ & +{\kappa }_2∥S_2^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2)∥.\hfill \end{array}$$

(10)

The decision function of Formula (8) is similar to that of the NHSVM method; that is, a new sample $\mathbf{x}\in {\Re }^n$ belongs to class ${\mathbf{k}}^{\ast }$ if and only if ${\mathbf{k}}^{\ast }=\underset{i=1,2}{argmin}\left\{d_i\left(\mathbf{x}\right):={\displaystyle \frac{\left|{\mathbf{w}}_i^{\mathrm{T}}\mathbf{x}+b_i\right|}{∥{\mathbf{w}}_i∥}}\right\}$.

The dual formula of SOCP-NHSVM is as follows [7]

$$\begin{array}{cc}\hfill \underset{{\displaystyle \genfrac{}{}{0pt}{}{{\mathbf{z}}_i,{\mathbf{u}}_i\hfill }{i=1,2\hfill }}}{min}& -{\displaystyle \frac{1}{2}}{\mathbf{e}}^{\mathrm{T}}{\left(Z^{\mathrm{T}}[{(H^{\mathrm{T}}H+\theta I)}^{-1}+{(G^{\mathrm{T}}G+\theta I)}^{-1}]Z\right)}^{-1}\mathbf{e}\hfill \\ \hfill s.t.& {\mathbf{z}}_1\in \mathbf{B}({\mu }_1,S_1,-{\kappa }_1),\hfill \\ \hfill & {\mathbf{z}}_2\in \mathbf{B}({\mu }_2,S_2,{\kappa }_2),\hfill \end{array}$$

(11)

where $\mathbf{B}(\mu ,S,\kappa )=\{\mathbf{z}:\mathbf{z}=\mu +\kappa S\mathbf{u},∥\mathbf{u}∥\le 1\}$. Set $\mathbf{B}(\mu ,S,\kappa )$ is an ellipsoid centered on $\mathsf{\mu }$, whose shape is determined by S and size is determined by $\kappa$ [8].

2.3. The Kernel-Based SOCP-NHSVM

In this section, we extend SOCP-NHSVM to kernel functions to obtain nonlinear classifiers. The weight vector of each nonparallel hyperplane can be written as ${\mathbf{w}}_i=\mathbb{Z}{\mathbf{s}}_i+M{\mathbf{r}}_i$, where M is a matrix whose column vectors are orthogonal to the training data points; ${\mathbf{s}}_i$ and ${\mathbf{r}}_i$ are vectors with appropriate dimensional series; $\mathbb{Z}=[A^{\mathrm{T}},B^{\mathrm{T}}]\in {\Re }^{n\times m}$ is the data matrix containing these two training modes. The empirical estimates of the mean ${\mathsf{\mu }}_i$ and covariance ${\mathsf{\Sigma }}_i$ are given by the following formula:

$$\begin{array}{c}\hfill {\mu }_1={\displaystyle \frac{1}{m_1}}{A}^{\mathrm{T}}{\mathbf{e}}_1,{\mu }_2={\displaystyle \frac{1}{m_2}}{B}^{\mathrm{T}}{\mathbf{e}}_2,\end{array}$$

(12)

$$\begin{array}{c}\hfill {\mathsf{\Sigma }}_1={\displaystyle \frac{1}{m_1}}{A}^{\mathrm{T}}(I_1-{\displaystyle \frac{{\mathbf{e}}_1{\mathbf{e}}_1^{\mathrm{T}}}{m_1}})(I_1-{\displaystyle \frac{{\mathbf{e}}_1{\mathbf{e}}_1^{\mathrm{T}}}{m_1}})A,\\ \hfill {\mathsf{\Sigma }}_2={\displaystyle \frac{1}{m_2}}{B}^{\mathrm{T}}(I_2-{\displaystyle \frac{{\mathbf{e}}_2{\mathbf{e}}_2^{\mathrm{T}}}{m_2}})(I_2-{\displaystyle \frac{{\mathbf{e}}_2{\mathbf{e}}_2^{\mathrm{T}}}{m_2}})B.\end{array}$$

(13)

Therefore, for each class i, $A{\mathbf{w}}_1=\left[K_{11},K_{12}\right]{\mathbf{s}}_1=K_{1•}{\mathbf{s}}_1$, $B{\mathbf{w}}_2=\left[K_{21},K_{22}\right]{\mathbf{s}}_2=K_{2•}{\mathbf{s}}_2$ and ${\mathbf{w}}_i^{\mathrm{T}}{\mu }_i={\mathbf{s}}_i^{\mathrm{T}}{\mathbf{g}}_i,{\mathbf{w}}_i^{\mathrm{T}}{\mathsf{\Sigma }}_i{\mathbf{w}}_i={\mathbf{s}}_i^{\mathrm{T}}{\mathsf{\Xi }}_i{\mathbf{s}}_i,i=1,2$. where $g_i={\displaystyle \frac{1}{m_i}}\left[\begin{array}{c}{K}_{1i}{\mathbf{e}}_i\hfill \\ K_{2i}{\mathbf{e}}_i\hfill \end{array}\right]$, ${\mathsf{\Xi }}_i={\displaystyle \frac{1}{m_i}}\left[\begin{array}{c}{K}_{1i}\hfill \\ K_{2i}\hfill \end{array}\right]\left(I_{m_i}-{\displaystyle \frac{1}{m_i}}{\mathbf{e}}_i{\mathbf{e}}_i^{\mathrm{T}}\right)\left[K_{1i},K_{2i}\right]$. $K_{11}=A{A}^{\mathrm{T}},K_{12}=K_{21}^{\mathrm{T}}=A{B}^{\mathrm{T}},K_{22}=B{B}^{\mathrm{T}}$ is a matrix, and its elements are the inner product of training data points. For example, matrix $K_{i{i}^{\prime }}$ follows ${\left(K_{i{i}^{\prime }}\right)}_{l,s}={({\mathbf{x}}_l^i)}^{\mathrm{T}}{\mathbf{x}}_s^{i^{\prime }}$ under entry $(l,s)$, and the form is ${\left(K_{i{i}^{\prime }}\right)}_{l,s}=k({\mathbf{x}}_l^i,{\mathbf{x}}_s^{i^{\prime }})$ when the kernel function is adopted. In this paper, Gaussian kernel is used, which usually leads to the best empirical results [11,12], and its formula is $k({\mathbf{x}}_l,{\mathbf{x}}_s)=exp\left(-{\displaystyle \frac{∥{\mathbf{x}}_l-{\mathbf{x}}_s∥}{2{\sigma }^2}}\right)$, and $\sigma$ is a parameter to control the kernel width [12].

Note that M has nothing to do with constraints; in other words, ${\mathbf{w}}_i=\mathbb{Z}{\mathbf{s}}_i$ in the optimal state. Therefore, we can get the SOCP-NHSVM model based on the above information

$$\begin{array}{cc}\hfill \underset{{\displaystyle \genfrac{}{}{0pt}{}{{\mathbf{s}}_i,b_i\hfill }{i=1,2\hfill }}}{min}& {\displaystyle \frac{1}{2}}{∥K_{1•}{\mathbf{s}}_1+{\mathbf{e}}_1{b}_1∥}^2+{\displaystyle \frac{1}{2}}{∥K_{2•}{\mathbf{s}}_2+{\mathbf{e}}_2{b}_2∥}^2+\hfill \\ & {\displaystyle \frac{\theta }{2}}({∥{\mathbf{s}}_1∥}^2+b_1^2+{∥{\mathbf{s}}_2∥}^2+b_2^2)\hfill \\ \hfill s.t.& {({\mathbf{s}}_1-{\mathbf{s}}_2)}^{\mathrm{T}}{\mathbf{g}}_1+(b_1-b_2)\ge 1+{\kappa }_1∥{\mathsf{\Lambda }}_1^{\mathrm{T}}({\mathbf{s}}_1-{\mathbf{s}}_2)∥,\hfill \\ \hfill & -\left({({\mathbf{s}}_1-{\mathbf{s}}_2)}^{\mathrm{T}}{\mathbf{g}}_2+(b_1-b_2)\right)\ge 1+{\kappa }_2∥{\mathsf{\Lambda }}_2^{\mathrm{T}}({\mathbf{s}}_1-{\mathbf{s}}_2)∥,\hfill \end{array}$$

(14)

where ${\mathsf{\Xi }}_i={\mathsf{\Lambda }}_i{\mathsf{\Lambda }}_i^{\mathrm{T}},i=1,2$ is a semi-positive definite matrix. ${\mathsf{\Lambda }}_i,i=1,2$ are derived from the eigenvalue decomposition matrix of the matrix ${\mathsf{\Xi }}_i$. The solution of this model can produce the following kernel plane:

$$\begin{array}{c}\hfill k(\mathbf{x},\mathbb{Z}){\mathbf{s}}_1+b_1=0,k(\mathbf{x},\mathbb{Z}){\mathbf{s}}_2+b_2=0.\end{array}$$

(15)

Therefore, for a given $\mathbf{x}\in {\Re }^n$, the row vector $k(\mathbf{x},\mathbb{Z})$ can be defined as

$$\begin{array}{c}\hfill k(\mathbf{x},\mathbb{Z})=\left[k(\mathbf{x},{\mathbb{Z}}_{•1}),k(\mathbf{x},{\mathbb{Z}}_{•2}),\dots ,k(\mathbf{x},{\mathbb{Z}}_{•m})\right],\end{array}$$

(16)

where ${\mathbb{Z}}_{•j}\in {\Re }^n$ represents the jth column of matrix $\mathbb{Z}$. Based on this, a new data point $\mathbf{x}\in {\Re }^n$ belongs to class ${\mathbf{k}}^{\ast }$ [7] if and only if

$$\begin{array}{c}\hfill {\mathbf{k}}^{\ast }=\underset{i=1,2}{argmin}{\displaystyle \frac{\left|k(\mathbf{x},\mathbb{Z}){\mathbf{s}}_i+b_i\right|}{\sqrt{{\mathbf{s}}_i^{\mathrm{T}}\mathbf{K}{\mathbf{s}}_i}}},\end{array}$$

(17)

where $\mathbf{K}=\left[K_{11},K_{12};K_{21},K_{22}\right]\in {\Re }^{m\times m}$.

3. Nonparallel Hyperplane Circular Cone Programming Support Vector Machine

To improve the computational efficiency and solve the model more stably, we convert the Second-Order Cone Programming Nonparallel Hyperplane Support Vector Machine (SOCP-NHSVM) into a standard convex quadratic cone programming support vector machine. Nonparallel hyperplane circular cone programming support vector machine (CQCCP-NHSVM) is a modified support vector machine algorithm based on nonparallel hyperplanes and circular cone programming, which has significant advantages in handling nonlinearly separable data.

In the original SOCP-NHSVM model, the conic constraints contain complex variable terms that are not compatible with the form of standard convex quadratic cone programming. To address this issue, we perform variable substitution on the variables involved in the conic constraints. Specifically, we introduce new auxiliary variables to replace the nonlinear combination terms of the original variables, which simplifies the form of the conic constraints and makes them conform to the standard form required for subsequent solution steps. This substitution process ensures that the equivalence of the original model is maintained, i.e., the optimal solution of the new model after substitution is consistent with that of the original SOCP-NHSVM model.

Let $u_0={({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_1+(b_1-b_2)-1$, $v_0=-{({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_2-(b_1-b_2)-1$, $u_1=S_1^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2)$ and $v_1=S_2^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2)$; then Equation (8) can be rewritten as:

$$\begin{array}{cc}\hfill min& {\displaystyle \frac{1}{2}}{\mathbf{v}}_1^{\mathrm{T}}(H^{\mathrm{T}}H+\theta I){\mathbf{v}}_1+{\displaystyle \frac{1}{2}}{\mathbf{v}}_2^{\mathrm{T}}(G^{\mathrm{T}}G+\theta I){\mathbf{v}}_2\hfill \\ \hfill s.t.& u_1=S_1^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2),\hfill \\ \hfill & u_0={({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_1+(b_1-b_2)-1,\hfill \\ \hfill & v_1=S_2^{\mathrm{T}}({\mathbf{w}}_1-{\mathbf{w}}_2),\hfill \\ \hfill & v_0=-{({\mathbf{w}}_1-{\mathbf{w}}_2)}^{\mathrm{T}}{\mathsf{\mu }}_2-(b_1-b_2)-1,\hfill \\ \hfill & ∥u_1∥\le {\displaystyle \frac{1}{{\kappa }_1}}·u_0,∥v_1∥\le {\displaystyle \frac{1}{{\kappa }_2}}·v_0.\hfill \end{array}$$

(18)

By the singular value decomposition of the matrix in Equation (18), can get:

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2}}{\mathbf{v}}_1^{\mathrm{T}}(H^{\mathrm{T}}H+\theta I){\mathbf{v}}_1={\displaystyle \frac{1}{2}}{M}_1^{\mathrm{T}}{M}_1,\\ \hfill M_1={(H^{\mathrm{T}}H+\theta I)}^{1/2}{\mathbf{v}}_1,\\ \hfill {\displaystyle \frac{1}{2}}{\mathbf{v}}_2^{\mathrm{T}}(G^{\mathrm{T}}G+\theta I){\mathbf{v}}_2={\displaystyle \frac{1}{2}}{M}_2^{\mathrm{T}}{M}_2,\\ \hfill M_2={(G^{\mathrm{T}}G+\theta I)}^{1/2}{\mathbf{v}}_2.\end{array}$$

(19)

Then block the matrix $M_i$ of the above formula, and we can obtain $M_1=\left[m_{11}{m}_{111}\right]\left[\begin{array}{c}{\mathbf{w}}_1\hfill \\ b_1\hfill \end{array}\right]$ and $M_2=\left[m_{22}{m}_{222}\right]\left[\begin{array}{c}{\mathbf{w}}_2\hfill \\ b_2\hfill \end{array}\right]$, where $m_{11},m_{22}\in {\Re }^{(n+1)\times n}$ and $m_{111},m_{222}\in {\Re }^{(n+1)\times 1}$. $m_{11},m_{111}$ respectively represent the blocks of ${(H^{\mathrm{T}}H+\theta I)}^{1/2}$ with dimensions $(n+1)\times n$ and $(n+1)\times 1$. $m_{22},m_{222}$ respectively represent the blocks of ${(G^{\mathrm{T}}G+\theta I)}^{1/2}$ with dimensions $(n+1)\times n$ and $(n+1)\times 1$.

Based on the above information, Equation (19) can be simplified as

$$\begin{array}{cc}\hfill min& {\displaystyle \frac{1}{2}}{x}^{\mathrm{T}}{Q}_{1}x\hfill \\ \hfill s.t.& A_{1}x=b^{\prime },x\in {\mathsf{\Omega }}_1,\hfill \end{array}$$

(20)

where $x={(b_1,b_2,{\mathbf{w}}_1^{\mathrm{T}},{\mathbf{w}}_2^{\mathrm{T}},u_0,u_1^{\mathrm{T}},v_0,v_1^{\mathrm{T}})}^{\mathrm{T}}$ is a vector with dimension $(4\times n)+4$, $A_1=\left[\begin{array}{c}1-1{\mathsf{\mu }}_1^{\mathrm{T}}-{\mathsf{\mu }}_1^{\mathrm{T}}-1000\hfill \\ 1-1{\mathsf{\mu }}_2^{\mathrm{T}}-{\mathsf{\mu }}_2^{\mathrm{T}}0010\hfill \\ 00S_1^{\mathrm{T}}-S_1^{\mathrm{T}}0-I_{n}00\hfill \\ 00S_2^{\mathrm{T}}-S_2^{\mathrm{T}}000-I_n\hfill \end{array}\right]$, $Q_1=\left[\begin{array}{c}{m}_{111}^{\mathrm{T}}{m}_{111}0m_{111}^{\mathrm{T}}{m}_{11}00000\hfill \\ 0m_{222}^{\mathrm{T}}{m}_{222}0m_{222}^{\mathrm{T}}{m}_{22}0000\hfill \\ m_{11}^{\mathrm{T}}{m}_{111}0m_{11}^{\mathrm{T}}{m}_{11}00000\hfill \\ 0m_{22}^{\mathrm{T}}{m}_{222}0m_{22}^{\mathrm{T}}{m}_{22}0000\hfill \\ 00000000\hfill \\ 00000000\hfill \\ 00000000\hfill \\ 00000000\hfill \end{array}\right]$, $b^{\prime }=\left[\begin{array}{c}1\\ -1\\ 0\\ 0\end{array}\right]$, ${\mathsf{\Omega }}_1\in {\Re }^{2\times n+2}\times L_{{\theta }_1}\times L_{{\theta }_2}$. $L_{{\theta }_1},L_{{\theta }_2}$ represents two cones [13]

$$\begin{array}{c}\hfill L_{{\theta }_1}=\left\{(u_0,u_1)|∥u_1∥\le tan{\theta }_1·u_0\right\},\\ \hfill L_{{\theta }_2}=\left\{(v_0,v_1)|∥v_1∥\le tan{\theta }_2·v_0\right\},\end{array}$$

(21)

where $tan{\theta }_i={\displaystyle \frac{1}{{\kappa }_i}},i=1,2$.

Equation (20) is a CQCCP problem with two conic constraints, and its linear equation is $(2\times n+2)\times (4\times n+4)$ dimensions. Equation (10) is a linear second-order cone programming problem with four second-order cone constraints, and its dimensions are $(4\times n+6)\times (6\times n+10)$. Obviously, the scale of Formula (20) is smaller than that of Formula (10), which means that the calculation time of the program is saved.

Similarly, the kernel-based SOCP-NHSVM is rewritten as a conic programming form. Firstly, the objective function in Equation (14) is rewritten as

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2}}{∥K_{1•}{\mathbf{s}}_1+{\mathbf{e}}_1{b}_1∥}^2+{\displaystyle \frac{\theta }{2}}({∥{\mathbf{s}}_1∥}^2+b_1^2)={\displaystyle \frac{1}{2}}{{\mathbf{v}}^{\prime }}_1^{\mathrm{T}}({H^{\prime }}^{\mathrm{T}}{H}^{\prime }+\theta I){{\mathbf{v}}^{\prime }}_1,\\ \hfill {\displaystyle \frac{1}{2}}{∥K_{2•}{\mathbf{s}}_2+{\mathbf{e}}_2{b}_2∥}^2+{\displaystyle \frac{\theta }{2}}({∥{\mathbf{s}}_2∥}^2+b_2^2)={\displaystyle \frac{1}{2}}{{\mathbf{v}}^{\prime }}_2^{\mathrm{T}}({G^{\prime }}^{\mathrm{T}}{G}^{\prime }+\theta I){{\mathbf{v}}^{\prime }}_2,\end{array}$$

(22)

where $H^{\prime }=[K_{1•},{\mathbf{e}}_1]\in {\Re }^{m_1\times (m+1)}$, $G^{\prime }=[K_{2•},{\mathbf{e}}_2]\in {\Re }^{m_2\times (m+1)}$, ${{\mathbf{v}}^{\prime }}_i={[{\mathbf{s}}_i^{\mathrm{T}},b_i]}^{\mathrm{T}}\in {\Re }^{m+1}$.

Let ${u^{\prime }}_1={\mathsf{\Lambda }}_1^{\mathrm{T}}({\mathbf{s}}_1-{\mathbf{s}}_2)$, ${v^{\prime }}_1={\mathsf{\Lambda }}_2^{\mathrm{T}}({\mathbf{s}}_1-{\mathbf{s}}_2)$, ${u^{\prime }}_0={({\mathbf{s}}_1-{\mathbf{s}}_2)}^{\mathrm{T}}{\mathbf{g}}_1+(b_1-b_2)-1$ and ${v^{\prime }}_0=-{({\mathbf{s}}_1-{\mathbf{s}}_2)}^{\mathrm{T}}{\mathbf{g}}_2-(b_1-b_2)-1$, then that constraints in Equation (14) can be rewritten as:  

$$\begin{array}{cc}\hfill min& {\displaystyle \frac{1}{2}}{{\mathbf{v}}^{\prime }}_1^{\mathrm{T}}({H^{\prime }}^{\mathrm{T}}{H}^{\prime }+\theta I){{\mathbf{v}}^{\prime }}_1+{\displaystyle \frac{1}{2}}{{\mathbf{v}}^{\prime }}_2^{\mathrm{T}}({G^{\prime }}^{\mathrm{T}}{G}^{\prime }+\theta I){{\mathbf{v}}^{\prime }}_2\hfill \\ \hfill s.t.& {u^{\prime }}_1={\mathsf{\Lambda }}_1^{\mathrm{T}}({\mathbf{s}}_1-{\mathbf{s}}_2),\hfill \\ \hfill & {u^{\prime }}_0={({\mathbf{s}}_1-{\mathbf{s}}_2)}^{\mathrm{T}}{\mathbf{g}}_1+(b_1-b_2)-1,\hfill \\ \hfill & {v^{\prime }}_1={\mathsf{\Lambda }}_2^{\mathrm{T}}({\mathbf{s}}_1-{\mathbf{s}}_2),\hfill \\ \hfill & {v^{\prime }}_0=-{({\mathbf{s}}_1-{\mathbf{s}}_2)}^{\mathrm{T}}{\mathbf{g}}_2-(b_1-b_2)-1,\hfill \\ \hfill & ∥{u^{\prime }}_1∥\le {\displaystyle \frac{1}{{\kappa }_1}}·{u^{\prime }}_0,∥{v^{\prime }}_1∥\le {\displaystyle \frac{1}{{\kappa }_2}}·{v^{\prime }}_0.\hfill \end{array}$$

(23)

Similar to the linear case, the singular value of Equation (22) is decomposed into ${\displaystyle \frac{1}{2}}{{\mathbf{v}}^{\prime }}_1^{\mathrm{T}}(H^{\prime \mathrm{T}}{H}^{\prime }+\theta I){{\mathbf{v}}^{\prime }}_1={\displaystyle \frac{1}{2}}{M^{\prime }}_1^{\mathrm{T}}{M^{\prime }}_1$ and ${\displaystyle \frac{1}{2}}{{\mathbf{v}}^{\prime }}_2^{\mathrm{T}}(G^{\prime \mathrm{T}}{G}^{\prime }+\theta I){{\mathbf{v}}^{\prime }}_2={\displaystyle \frac{1}{2}}{M^{\prime }}_2^{\mathrm{T}}{M^{\prime }}_2$. Then block it to get ${M^{\prime }}_1=\left[{m^{\prime }}_{11}{m^{\prime }}_{111}\right]\left[\begin{array}{c}{\mathbf{s}}_1\hfill \\ b_1\hfill \end{array}\right]$ and ${M^{\prime }}_2=\left[{m^{\prime }}_{22}{m^{\prime }}_{222}\right]\left[\begin{array}{c}{\mathbf{s}}_2\hfill \\ b_2\hfill \end{array}\right]$, where ${m^{\prime }}_{11},{m^{\prime }}_{22}\in {\Re }^{(m+1)\times m}$ and ${m^{\prime }}_{111},{m^{\prime }}_{222}\in {\Re }^{(m+1)\times 1}$. ${m^{\prime }}_{11},{m^{\prime }}_{111}$ respectively represent the blocks of ${(H^{\prime \mathrm{T}}{H}^{\prime }+\theta I)}^{1/2}$ with dimensions $(m+1)\times m$ and $(m+1)\times 1$. ${m^{\prime }}_{22},{m^{\prime }}_{222}$ respectively represent the blocks of ${(G^{\prime \mathrm{T}}{G}^{\prime }+\theta I)}^{1/2}$ with dimensions $(m+1)\times m$ and $(m+1)\times 1$.

Then Equation (23) can be simplified as follows

$$\begin{array}{cc}\hfill min& {\displaystyle \frac{1}{2}}{x}^{\mathrm{T}}{Q}_{2}x\hfill \\ \hfill s.t.& A_{2}x=b^{″},x\in {\mathsf{\Omega }}_2,\hfill \end{array}$$

(24)

where $x={(b_1,b_2,{\mathbf{s}}_1^{\mathrm{T}},{\mathbf{s}}_2^{\mathrm{T}},{u^{\prime }}_0,{u^{\prime }}_1^{\mathrm{T}},{v^{\prime }}_0,{v^{\prime }}_1^{\mathrm{T}})}^{\mathrm{T}}$ is a vector with dimension $(4\times m)+4$, $A_2=\left[\begin{array}{c}1-1{\mathbf{g}}_1^{\mathrm{T}}-{\mathbf{g}}_1^{\mathrm{T}}-1000\hfill \\ 1-1{\mathbf{g}}_2^{\mathrm{T}}-{\mathbf{g}}_2^{\mathrm{T}}0010\hfill \\ 00{\mathsf{\Lambda }}_1^{\mathrm{T}}-{\mathsf{\Lambda }}_1^{\mathrm{T}}0-I_{m}00\hfill \\ 00{\mathsf{\Lambda }}_2^{\mathrm{T}}-{\mathsf{\Lambda }}_2^{\mathrm{T}}000-I_m\hfill \end{array}\right]$, $Q_2=\left(\begin{array}{cc}{Q}_{22}& 0_{4\times 4}\\ 0_{4\times 4}& 0_{4\times 4}\end{array}\right)$, $Q_2=\left[\begin{array}{c}{m^{\prime }}_{111}^{\mathrm{T}}{m^{\prime }}_{111}0{m^{\prime }}_{111}^{\mathrm{T}}{m^{\prime }}_{11}0\hfill \\ 0{m^{\prime }}_{222}^{\mathrm{T}}{m^{\prime }}_{222}0{m^{\prime }}_{222}^{\mathrm{T}}{m^{\prime }}_{22}\hfill \\ {m^{\prime }}_{11}^{\mathrm{T}}{m^{\prime }}_{111}0{m^{\prime }}_{11}^{\mathrm{T}}{m^{\prime }}_{11}0\hfill \\ 0{m^{\prime }}_{22}^{\mathrm{T}}{m^{\prime }}_{222}0{m^{\prime }}_{22}^{\mathrm{T}}{m^{\prime }}_{22}\hfill \end{array}\right]$, $b^{″}=\left[\begin{array}{c}1\\ -1\\ 0\\ 0\end{array}\right]$, ${\mathsf{\Omega }}_2\in {\Re }^{2\times m+2}\times {L^{\prime }}_{{\theta }_1}\times {L^{\prime }}_{{\theta }_2}$. ${L^{\prime }}_{{\theta }_1},{L^{\prime }}_{{\theta }_2}$ represents two cones [13]:

$$\begin{array}{c}\hfill {L^{\prime }}_{{\theta }_1}=\left\{({u^{\prime }}_0,{u^{\prime }}_1)|∥{u^{\prime }}_1∥\le tan{\theta }_1·{u^{\prime }}_0\right\},\\ \hfill {L^{\prime }}_{{\theta }_2}=\left\{({v^{\prime }}_0,{v^{\prime }}_1)|∥{v^{\prime }}_1∥\le tan{\theta }_2·{v^{\prime }}_0\right\}.\end{array}$$

(25)

Similarly, Equation (24) is a CQCCP problem with two conic constraints, and its dimensions are $(2\times m+2)\times (4\times m+4)$. Equation (14) can obviously be rewritten as a linear second-order cone programming problem with four second-order cone constraints, and its dimensions are $(4\times m+6)\times (6\times m+10)$. Similarly, the scale of Formula (24) is smaller than that of Formula (14), which saves the calculation time of the program.

Equations (20) and (24) are a special convex quadratic circular cone programming problem [14]

$$\begin{array}{cc}\hfill min& {\displaystyle \frac{1}{2}}{x}^{\mathrm{T}}Qx+c^{\mathrm{T}}x\hfill \\ \hfill s.t.& \mathbf{A}x=d,x\in L_{\theta },\hfill \end{array}$$

(26)

where $L_{\theta }={\Re }^n\times L_{{\theta }_1}\times L_{{\theta }_2}$.

When ${\theta }_i={\displaystyle \frac{\pi }{4}},i=1,2$, two conic programs $L_{{\theta }_1},L_{{\theta }_2}$ are transformed into two second-order cone constraints:

$$\begin{array}{c}\hfill K_1=\left\{(u_0,u_1)|∥u_1∥\le u_0\right\},\\ \hfill K_2=\left\{(v_0,v_1)|∥v_1∥\le v_0\right\}.\end{array}$$

(27)

Based on this, $K={\Re }^n\times K_1\times K_2$. The dual form of Equation (26) can be directly derived by the Lagrange method

$$\begin{array}{cc}\hfill max& d^{\mathrm{T}}y-{\displaystyle \frac{1}{2}}{x}^{\mathrm{T}}Qx+c^{\mathrm{T}}x\hfill \\ \hfill s.t.& {\mathbf{A}}^{\mathrm{T}}y+z-Qx=c,z\in L_{\theta }^{\ast },\hfill \end{array}$$

(28)

where $y\in {\Re }^n$ is a lagrange multiplier, $L_{\theta }^{\ast }={\left\{0\right\}}^n\times L_{{\theta }_1}^{\ast }\times L_{{\theta }_2}^{\ast }$ is a dual cone of $L_{\theta }$, $L_{{\theta }_i}^{\ast }$ is a dual cone of $L_{{\theta }_i}$, and $L_{{\theta }_i}^{\ast }=\left\{z_i|z_i^{\mathrm{T}}{x}_i\ge 0,x_i\in L_{{\theta }_i}\right\}$.

According to Slater’s condition, the KKT condition of the strong duality theorem for general cone programming problems is [15]:

$$\begin{array}{cc}\hfill & \mathbf{A}x=d,x\in L_{\theta },\hfill \\ \hfill & {\mathbf{A}}^{\mathrm{T}}y+z-Qx=c,z\in L_{\theta }^{\ast },\hfill \\ \hfill & x^{\mathrm{T}}z=0.\hfill \end{array}$$

(29)

4. The Projection and Contraction Method of CQCCP-NHSVM

Let $T_i=\left[\begin{array}{cc}{I}_{n_i-1}\hfill & \hfill 0\\ 0\hfill & \hfill tan{\theta }_i\end{array}\right],i=1,2$; from the definition of cones $L_{{\theta }_i}$ and $L_{{\theta }_i}^{\ast }$, we can know [13]:  

$$\begin{array}{c}\hfill x_i\in L_{{\theta }_i}\iff T_i{x}_i\in K_i,z_i\in L_{{\theta }_i}^{\ast }\iff T_i^{-1}{x}_i\in K_i.\end{array}$$

(30)

Let $T=\left[\begin{array}{c}{I}_{n}00\hfill \\ 0T_{1}0\hfill \\ 00T_2\hfill \end{array}\right]$; from the Formula (30), the following relational formula can be obtained:

$$\begin{array}{c}\hfill x\in L_{\theta }\iff Tx\in K,z\in L_{\theta }^{\ast }\iff T^{-1}x\in K.\end{array}$$

(31)

References [16,17] gives the projection of vector x on the second-order cone $K_i$ as follows

$$\begin{array}{c}\hfill P_{K_i}\left(x\right)={\left(z_0-∥z_1∥\right)}_{+}·{\displaystyle \frac{1}{2}}\left[\begin{array}{c}-w\hfill \\ 1\hfill \end{array}\right]\\ \hfill +{\left(z_0+∥z_1∥\right)}_{+}·{\displaystyle \frac{1}{2}}\left[\begin{array}{c}w\hfill \\ 1\hfill \end{array}\right],\end{array}$$

(32)

where, when $z_1\ne 0$, $w={\displaystyle \frac{-z_1}{∥z_1∥}}$, when $z_1=0$, w is an arbitrary vector of ${\Re }^{n-1}$ and satisfies $∥w∥=1$; for any real number $s\in \Re$, there is $s_{+}=max(0,s)$.

Therefore, the projection of vector x on the whole cone K can be expressed as

$$\begin{array}{c}\hfill P_{K_i}\left(z\right)=\left[P_{{\Re }^n}\left(x_0\right),P_{K_1}\left(x_1\right),P_{K_2}\left(x_2\right)\right]\in {\Re }^n\times {\Re }^n\times {\Re }^n,\end{array}$$

(33)

where the vector $x=\left[x_1,x_2,x_3\right]\in {\Re }^n\times {\Re }^n\times {\Re }^n$ can be regarded as a ${\Re }^{3\times n}$ dimensional column vector.

Theorem 1.

Suppose $\overline{x}\in L_{\theta },\overline{z}\in L_{\theta }^{\ast }$, if ${\overline{x}}^{\mathrm{T}}\overline{z}=0$, if and only if $e\left(\overline{x},\overline{z}\right)=0$, where $e\left(\overline{x},\overline{z}\right)=T\overline{x}-P_K(T\overline{x}-T^{-1}\overline{z})$.

Proof. 

From Equation (31), we can see that $T\overline{x},T^{-1}\overline{z}\in K$. From reference [18], it is known that ${\left(T\overline{x}\right)}^{\mathrm{T}}\left(T^{-1}\overline{z}\right)=0$ holds if and only if $e\left(\overline{x},\overline{z}\right)=T\overline{x}-P_K(T\overline{x}-T^{-1}\overline{z})=0$.    □

Let $u=\left[\begin{array}{c}x\hfill \\ y\hfill \end{array}\right],W=\left[\begin{array}{c}T0\hfill \\ 0I_m\hfill \end{array}\right],M=\left[\begin{array}{c}Q-A^{\mathrm{T}}\hfill \\ A0\hfill \end{array}\right]$, $q=\left[\begin{array}{c}c\hfill \\ d\hfill \end{array}\right]$, and then we can define the residual function

$$\begin{array}{c}\hfill e\left(u\right)=Wu-P_{\mathsf{\Omega }}(Wu-W^{-1}(Mu+q)),\end{array}$$

(34)

where $P_{\mathsf{\Omega }}(·)$ represents the projection on the set $\mathsf{\Omega }=K\times {\Re }^m$. Through Theorem 1, the optimality condition Formula (29) is equivalent to $e\left(u\right)=0$.

The dumped Newton projection and contraction method was first proposed and applied to solve linear monotone variational inequality problems [19]. Reference [14] also applies it to the problem of force optimization. In this paper, the dumped Newton projection contraction algorithm can be used to solve convex quadratic programming problems. For solving Equations (20) and (24), it is only necessary to change the dimensions of the corresponding vector x and set $\mathsf{\Omega }$.

Let $u^{\ast }\in {\mathsf{\Omega }}_{\theta }^{\ast }$, where ${\mathsf{\Omega }}_{\theta }^{\ast }$ is the solution set of problem (30). The ellipsoid norm distance function is defined as follows

$$\begin{array}{c}\hfill F\left(u\right)={\displaystyle \frac{1}{2}}{∥u-u^{\ast }∥}_{\mathrm{G}}^2={\displaystyle \frac{1}{2}}{(u-u^{\ast })}^{\mathrm{T}}G(u-u^{\ast }),\end{array}$$

(35)

where G is a symmetric positive definite matrix.

If $G={(W+W^{-1}M)}^{\mathrm{T}}(W+W^{-1}M)$ and $d\left(u\right)=(W+W^{-1}{M}^{\mathrm{T}})e\left(u\right)$, then we can know that $-G^{-1}d\left(u^i\right)$ is a descending direction of the function $F\left(u\right)$ at point $u^i$ [14].

The steps of the dumped Newton projection and contraction (DNPC) algorithm (Algorithm 1) are presented:

Algorithm 1 Dumped Newton Projection and Contraction Method (DNPC) [14]

1: Given the initial point $u^0$ and parameter $r\in (0,2),\epsilon >0$, set $i=0$; 2: if $∥e\left(u^i\right)∥<\epsilon$ then 3:    the iteration is stopped. 4: else 5:    Calculate the descending direction of function $F\left(u\right)$ by the following formula: $$\begin{array}{c}\hfill -g\left(u^i\right)=-G^{-1}d\left(u^i\right)=-{(W+W^{-1}M)}^{-1}e\left(u^i\right);\end{array}$$ 6:    Update iteration point: $u^{i+1}=u^i-rg\left(u^i\right)$; 7:    Let $i=i+1$; 8: end if

The dumped Newton projection and contraction method only needs to calculate the projection on the cone K once, which is simple and requires less calculation time. Similar to the convergence analysis in reference [14], the convergence of the dumped Newton projection and contraction method for solving convex quadratic cone programming support vector machine can be proven.

Theorem 2.

The point sequence $\left\{u^i\right\}$ generated by the dumped Newton projection and contraction method satisfies that ${∥u^{i+1}-u^{\ast }∥}_G^2\le {∥u^i-u^{\ast }∥}_G^2-r(2-r){∥e\left(u^i\right)∥}^2$ exists for any $u^{\ast }\in {\mathsf{\Omega }}_{\theta }^{\ast }$ [14].

Theorem 3.

Every point sequence $\left\{u^i\right\}$ generated by the dumped Newton projection and contraction method converges to $u^{\ast }\in {\mathsf{\Omega }}_{\theta }^{\ast }$ [14].

Theorems 2 and 3 follow directly from [14] under the convex conic structure, monotonicity, and strong duality of the CQCCP-NHSVM problem. A sketch of the proof relies on: (1) the convexity of the objective; (2) the Lipschitz continuity of the residual mapping; and (3) the fixed step–size condition $r\in (0,2)$. Full proof is omitted because it duplicates [14] under our problem’s conditions.

5. Experimental Results

5.1. Data Information and Evaluation Metrics

In this paper, the following seven benchmark datasets from the UCI repository are selected for experimental verification [20]: Australian Credit (AUS), Wisconsin Breast Cancer (WBC), Heart/Statlog (HEART), Iono sphere (IONO), Seeds-1_vs_2-3 (SEED) and Musk (Version 1). Table 1 shows the relevant information of the selected standard dataset, including the number of dataset variables, dataset sample size and dataset imbalance rate. More information about datasets can be found on the official website, the UCI resource library.

Table 1. Number of variables, number of examples and imbalanced ratio for all six datasets.

Dataset	Variables	Examples	Imbalance Ratio
Australian Credit (AUS)	14	690	1.2
Musk (Version 1)	166	476	1.3
Seeds-1_vs_2-3 (SEED)	7	210	2
Heart/Statlog (HEART)	13	270	1.25
Iono sphere (IONO)	34	351	1.8
Wisconsin Breast Cancer (WBC)	30	569	1.7
Fisher’s Iris Dataset ( IRIS)	2	50	1.1

In machine learning, there are various performance evaluation metrics used by scholars, such as the traditional Accuracy (AUC), as well as the currently most used F-measure [21] and G-mean [22]. Their specific expressions are as follows

$$\begin{array}{cc}\hfill & AUC={\displaystyle \frac{TP+TN}{TP+TF+FP+FN}},\hfill \\ & F-measure={\displaystyle \frac{2\ast Precision\ast Recall}{Precision+Recall}},\hfill \\ & G-mean=\sqrt{TPR\ast TNR},\hfill \end{array}$$

(36)

where $TN$ = true negative, $FP$ = false positive, $FN$ = false negative, $TP$ = true positive, $Precision=TP/(TP+FP),Recall=TPR=TP/(TP+FN),TNR=TN/(TN+FP)$.

To better validate the classification effect of the model, the experiments conducted in this section divide the dataset into a training subset and a test subset by a ten-fold cross-validation method. In addition, by calculating the average CPU time, the difference in running time between the two methods is compared. In the second-order cone programming support vector machine model, the value range of probability parameter ${\eta }_1,{\eta }_2$ is $\{0.1,0.2,0.3,0.4\}$, and the value range of kernel function parameter $\sigma$ is $\{2^{-7},2^{-6},2^{-5},2^{-4},2^{-3},2^{-2},2^{-1},2^0\}$.

The kernel parameter $\sigma$ was determined via a grid search with cross-validation (cyclic optimization) over the candidate set $\{2^{-7},2^{-6},2^{-5},2^{-4},2^{-3},2^{-2},2^{-1},2^0\}$. For each dataset, we evaluated all candidate values and selected the one that achieved the best average performance in terms of AUC, G-mean, and F-measure under 10-fold cross-validation. The same optimal $\sigma$ was used for both the DNPC and SeDuMi methods to ensure a fair comparison.

For solving mathematical problems of SOCP-NHSVM (Equations (10) and (23)), this chapter uses the SeDuMi toolbox [4]. The projection and contraction method proposed in this paper for solving the CQCCP-NHSVM model is solved by the DNPC method (Equations (20) and (24)).

5.2. Numerical Results and Analysis of Linear Classifier

This section summarizes the average calculation time and classification performance of linear classifiers, and gives the corresponding analysis. Choose one representative for each dataset in all parameter ranges. Table 2 shows the average CPU time required to solve the linear SOCP-NHSVM model by using the SeDuMi method and DNPC method.

Table 2. Average CPU time of solving linear SOCP-NHSVM by DNPC method and SeDuMi method.

Dataset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	DNPC	SeDuMi
AUS	0.2	0.4	0.0256 ± 0.0012	0.0802 ± 0.0011
Musk	0.4	0.1	0.4430 ± 0.0010	1.7068 ± 0.0010
HEART	0.4	0.4	0.0318 ± 0.0011	0.0701 ± 0.0012
IONO	0.3	0.4	0.1096 ± 0.0009	0.1474 ± 0.0009
SEED	0.2	0.2	0.0021 ± 0.0011	0.0602 ± 0.0011
WBC	0.1	0.2	0.0312 ± 0.0012	0.1664 ± 0.0013

The data in Table 2 shows that if a linear SOCP-NHSVM is solved, the SeDuMi method will need more CPU time than the DNPC method. Therefore, in the same time, the DNPC method can train more support vector models of nonparallel hyperplane second-order cone programming. Compared with the SeDuMi method, the DNPC method greatly improves the computational efficiency of the model.

Table 3, Table 4 and Table 5 show the classification performance of the SeDuMi method for solving the linear SOCP-NHSVM problem and the DNPC method for solving the linear CQCCP-NHSVM problem. Among them, Table 3 and Table 4 take the AUC value and G-mean value as the evaluation index of classification performance, and Table 5 takes the F-measure value as the evaluation index of classification performance. In the table, “max”, “min” and “mean” respectively represent the maximum value, minimum value and average value in the results of multiple runs.

Table 3. Test results of the linear SOCP-NHSVM model with the AUC value as the evaluation index.

Dateset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	DNPC			SeDuMi
			Max	Min	Mean	Max	Min	Mean
AUS	0.2	0.4	0.8729	0.8612	0.8661	0.8736	0.8634	0.8674
Musk	0.4	0.1	0.7965	0.7856	0.7918	0.8023	0.7872	0.7933
HEART	0.4	0.4	0.8547	0.8124	0.8343	0.8547	0.8136	0.8353
IONO	0.3	0.4	0.8226	0.8030	0.8096	0.8250	0.7994	0.8105
SEED	0.2	0.2	0.9452	0.9289	0.9369	0.9471	0.9326	0.9387
WBC	0.1	0.2	0.9293	0.9260	0.9274	0.9226	0.9207	0.9216
IRIS	0.2	0.4	0.8690	0.8654	0.8660	0.8689	0.8653	0.8660

Table 4. Test results of the linear SOCP-NHSVM model with G-mean value as evaluation index.

Dateset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	DNPC			SeDuMi
			Max	Min	Mean	Max	Min	Mean
AUS	0.2	0.4	0.8711	0.8636	0.8667	0.8705	0.8636	0.8664
Musk	0.4	0.1	0.7727	0.7667	0.7696	0.7830	0.7648	0.7734
HEART	0.4	0.4	0.8464	0.8240	0.8380	0.8467	0.8240	0.8383
IONO	0.3	0.4	0.9048	0.8862	0.8980	0.9055	0.8801	0.8963
SEED	0.2	0.2	0.9550	0.9081	0.9362	0.9560	0.9097	0.9364
WBC	0.1	0.2	0.9577	0.9517	0.9545	0.9572	0.9488	0.9518

Table 5. Test results of linear SOCP-NHSVM model with F-measure value as evaluation index.

Dateset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	DNPC			SeDuMi
			Max	Min	Mean	Max	Min	Mean
AUS	0.2	0.4	0.8715	0.8462	0.8588	0.8715	0.8462	0.8586
Musk	0.4	0.1	0.7718	0.7496	0.7628	0.7805	0.7559	0.7689
HEART	0.4	0.4	0.8567	0.8103	0.8338	0.8535	0.8060	0.8303
IONO	0.3	0.4	0.9194	0.8898	0.9039	0.9213	0.8917	0.9045
SEED	0.2	0.2	0.9137	0.8959	0.9038	0.9137	0.8932	0.9027
WBC	0.1	0.2	0.9231	0.9067	0.9147	0.9171	0.8973	0.9093

By observing Table 3, Table 4 and Table 5, can find that the SeDuMi method and DNPC method have very similar maximum and minimum values on six datasets. The numbers in black and bold in the table indicate that the corresponding method is better in the results of multiple runs under the corresponding parameters, but the classification performance of the two methods is very small. When there is no bold on both sides, it means that the two methods get the same result. Thus, in the case of linear separability, there is little difference between the SeDuMi method and DNPC method in three classification performance evaluation indexes, and the classification accuracy is very similar.

Generally speaking, both the SeDuMi method and DNPC method can ensure the classification performance of the second-order cone programming support vector machine when performing linear separable classification tasks, but the DNPC method only needs less time in solving the model. For the task that requires both classification performance and classification efficiency, it is a better choice to use the DNPC method to solve SOCP-NHSVM.

5.3. Numerical Results and Analysis of Kernel-Based Classifier

Table 6 shows the average CPU time of the SeDuMi method for solving nonlinear nonparallel hyperplane second-order cone programming support vector machine and the DNPC method for solving the nonlinear nonparallel cone programming support vector machine.

Table 6. Average CPU time of solving nonlinear SOCP-NHSVM by DNPC method and SeDuMi method.

Dataset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	$\mathit{\sigma }$	DNPC	SeDuMi
AUS	0.2	0.1	$2^{-6}$	2.8393 ± 0.0012	78.1674 ± 0.0010
Musk	0.3	0.4	$2^{-6}$	1.1026 ± 0.0011	25.2245 ± 0.0012
HEART	0.1	0.3	$2^{-6}$	0.4197 ± 0.0010	4.8026 ± 0.0013
IONO	0.3	0.1	$2^{-7}$	0.7177 ± 0.0008	11.9632 ± 0.0010
SEED	0.4	0.2	$2^{-4}$	0.2549 ± 0.0014	2.4372 ± 0.0011
WBC	0.2	0.2	$2^{-6}$	1.8937 ± 0.0011	42.4503 ± 0.0010

By observing Table 6, it can be found that the DNPC method needs less solving time than the SeDuMi method on six datasets. Specifically, taking the dataset AUS as an example, when ${\eta }_1$ is 0.2, ${\eta }_2$ is 0.1, and $\sigma$ is $2^{-6}$, it takes 78.1674 s for the SeDuMi method to solve a nonlinear SOCP-NHSVM problem, and only 2.8393 s for the DNPC method to solve a CQCCP-NHSVM problem. For small datasets such as HEART and SEED, the time difference multiple of the two methods is smaller than that of AUS and WBC with large datasets. It can be seen that for large-scale datasets, the DNPC method needs less calculation time than the SeDuMi method, and the DNPC method has a great time advantage. Whether for small-scale data classification or large-scale data classification, using the DNPC method is beneficial to improve the computational efficiency of the nonlinear SOCP-NHSVM model.

Table 7, Table 8 and Table 9 show the classification performance of the SeDuMi method for solving the nonlinear SOCP-NHSVM problem and the DNPC method for solving the nonlinear CQCCP-NHSVM problem. Among them, Table 7 and Table 8 take the AUC value and G-mean value as evaluation criteria of classification performance, respectively, while Table 9 takes the F-measure value as evaluation criteria of classification performance.

Table 7. Test results of nonlinear SOCP-NHSVM model with AUC value as evaluation index.

Dateset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	$\mathit{\sigma }$	DNPC			SeDuMi
				Max	Min	Mean	Max	Min	Mean
AUS	0.2	0.1	$2^{-6}$	0.8779	0.8714	0.8754	0.8720	0.8697	0.8706
Musk	0.3	0.4	$2^{-6}$	0.8799	0.8567	0.8720	0.8783	0.8610	0.8711
HEART	0.1	0.3	$2^{-6}$	0.8496	0.8351	0.8401	0.8439	0.8325	0.8375
IONO	0.3	0.1	$2^{-7}$	0.9413	0.9057	0.9243	0.9346	0.9014	0.9206
SEED	0.4	0.2	$2^{-4}$	0.9369	0.9136	0.9252	0.9317	0.9061	0.9203
WBC	0.2	0.2	$2^{-6}$	0.9668	0.9533	0.9592	0.9689	0.9526	0.9590
IRIS	0.2	0.4	$2^{-6}$	0.9180	0.9142	0.9158	0.9180	0.9141	0.9154

Table 8. Test results of nonlinear SOCP-NHSVM model with G-mean value as evaluation index.

Dateset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	$\mathit{\sigma }$	DNPC			SeDuMi
				Max	Min	Mean	Max	Min	Mean
AUS	0.2	0.1	$2^{-6}$	0.8841	0.8637	0.8766	0.8828	0.8572	0.8741
Musk	0.3	0.4	$2^{-6}$	0.8691	0.8631	0.8652	0.8695	0.8622	0.8667
HEART	0.1	0.3	$2^{-6}$	0.8530	0.8384	0.8477	0.8448	0.8310	0.8365
IONO	0.3	0.1	$2^{-7}$	0.9560	0.9418	0.9484	0.9537	0.9383	0.9458
SEED	0.4	0.2	$2^{-4}$	0.8951	0.8684	0.8849	0.8907	0.8549	0.8764
WBC	0.2	0.2	$2^{-6}$	0.9744	0.9603	0.9670	0.9750	0.9603	0.9668

Table 9. Test results of nonlinear SOCP-NHSVM model with F-measure value as evaluation index.

Dateset	${\mathit{\eta }}_1$	${\mathit{\eta }}_2$	$\mathit{\sigma }$	DNPC			SeDuMi
				Max	Min	Mean	Max	Min	Mean
AUS	0.2	0.1	$2^{-6}$	0.8737	0.8487	0.8595	0.8681	0.8445	0.8553
Musk	0.3	0.4	$2^{-6}$	0.8483	0.8258	0.8342	0.8492	0.8272	0.8357
HEART	0.1	0.3	$2^{-6}$	0.8663	0.8288	0.8496	0.8611	0.8207	0.8432
IONO	0.3	0.1	$2^{-7}$	0.9499	0.9306	0.9381	0.9489	0.9311	0.9373
SEED	0.4	0.2	$2^{-4}$	0.8814	0.8563	0.8720	0.8789	0.8477	0.8663
WBC	0.2	0.2	$2^{-6}$	0.9688	0.9608	0.9645	0.9676	0.9597	0.9638

From the experimental data in Table 7, Table 8 and Table 9, it can be found that, similar to the linear case, the SeDuMi method and DNPC method have similar maximum and minimum values on six datasets, integrating three classification performance standards. Thus, in the case of nonlinear separability, there are small differences between the SeDuMi method and DNPC method in three classification performance evaluation indexes.

We have added the comparison results of classification accuracy between the Naive Bayes (NB) Classifier, linear CQCCP-NHSVM, and nonlinear CQCCP-NHSVM by the DNPC method in Table 10. The results show that the classification accuracy of the nonlinear CQCCP-NHSVM is superior to that of the Naive Bayes (NB) Classifier; however, for some datasets, the linear CQCCP-NHSVM performs slightly worse than the Naive Bayes (NB) Classifier.

Table 10. Test results of mean AUC value as evaluation index by Naive Bayes and DNPC method.

Dataset	Naive Bayes	Linear SOCP-NHSVM	Nonlinear SOCP-NHSVM
AUS	0.8152	0.8661	0.8754
Musk	0.7982	0.7918	0.8720
HEART	0.8370	0.8343	0.8401
IONO	0.8857	0.8096	0.9243
SEED	0.8714	0.9369	0.9252
WBC	0.9313	0.9274	0.9592

In order to improve the classification efficiency of the nonlinear SOCP-NHSVM model, the time for solving the nonlinear SOCP-NHSVM problem can be shortened on the premise of ensuring the classification performance. By comparing the DNPC method and SeDuMi method to solve the nonlinear SOCP-NHSVM model, it can be found that the DNPC method can not only obtain better classification performance, but also requires less calculation time. Especially in large-scale data classification, the time advantage of the DNPC method is more obvious. Therefore, the DNPC method can be used instead of the SeDuMi method in solving the support vector machine of nonparallel hyperplane second-order cone programming.

It should be noted that this study also has certain limitations, which also point out the direction for future research. First, the experimental verification of this study is mainly carried out on UCI benchmark datasets, which are relatively standard and low-dimensional. In future work, it is necessary to test the performance of the proposed method on more complex and high-dimensional real-world datasets (such as high-resolution image datasets and large-scale text datasets) to further verify its generalization ability. Second, this study only compares the proposed method with the interior point method, and does not compare it with other emerging solution methods (such as gradient descent-based methods and the alternating-direction method of multipliers). Future research can expand the comparison scope to clarify the advantages and disadvantages of the proposed method in different solution frameworks. Third, the kernel function used in the kernel-based CQCCCP-NHSVM in this study is mainly the traditional RBF kernel. Future work can explore the adaptability of the proposed method to different kernel functions (such as polynomial kernel, sigmoid kernel) and optimize the selection strategy of kernel parameters to further improve the classification performance and computational efficiency of the model. In addition, exploring the application of the projection and contraction method in other types of nonparallel hyperplane SVM models (such as weighted NHSVM and robust NHSVM) is also a valuable research direction.

6. Discussion

From the perspective of previous studies, nonparallel hyperplane support vector machines (NHSVMs) have been widely usedin the field of machine learning due to their advantages in handling nonlinearly separable data and improving classification robustness compared with traditional parallel hyperplane SVMs. However, most existing solution methods for SOCP-NHSVM rely on interior point methods, which often suffer from high computational complexity and long running time, especially when dealing with large-scale datasets or high-dimensional feature spaces. Early studies have pointed out the inefficiency of interior point methods in solving large-scale SOCP problems, and proposed some improved interior point algorithms to reduce computational cost, but the effect is still limited. In contrast, the projection and contraction method proposed in this study avoids the inherent defects of interior point methods in terms of computational efficiency. By reformulating the linear and kernel-based SOCP-NHSVM into CQCCCP-NHSVM models and applying the projection and contraction strategy, the algorithm can quickly converge to the optimal solution with lower computational overhead, which is consistent with the research hypothesis that “a non-interior point method can effectively improve the solution efficiency of SOCP-NHSVM without sacrificing classification accuracy”.

The findings of this study have important theoretical and practical implications. Theoretically, this study enriches the solution methods of nonparallel hyperplane SOCP-NHSVM, provides a new idea for solving such second-order cone programming problems, and expands the application scope of projection and contraction methods in the field of support vector machines. Different from the existing improved interior point methods, the proposed CQCCP-NHSVM projection and contraction method fundamentally changes the solution framework, which not only enriches the theoretical system of SVM solution algorithms but also provides a reference for solving other similar convex optimization problems. Practically, the significant reduction in average CPU time makes the SOCP-NHSVM model more applicable to large-scale practical tasks, such as image classification, text recognition, and bioinformatics data processing, where computational efficiency and real-time performance are critical. For example, in scenarios where large-scale UCI benchmark datasets are used for real-time classification, the proposed method can ensure the accuracy of classification results while reducing the time cost of model training and prediction, which has high practical value.

7. Conclusions

In this paper, aiming at the problem of solving support vector machine for nonparallel hyperplane second-order cone programming, a solution scheme different from the interior point method, namely the CQCCP-NHSVM projection contraction method, is proposed. Firstly, the linear SOCP-NHSVM and kernel-based SOCP-NHSVM are reformulated as linear CQCCCP-NHSVM and kernel-based CQCCCP-NHSVM respectively. Then, the projection and contraction method is used to solve these two models. Finally, test and compare the classification performance of the two methods on a benchmark dataset of the UCI database under accuracy value, F-measure value and G-mean value. The experimental results show that the projection and contraction method can greatly reduce the average CPU time with similar classification accuracy when solving linear SOCP-NHSVM or kernel-based SOCP-NHSVM.