Pancreatic Cancer Early Detection Using Twin Support Vector Machine Based on Kernel

Abstract

Early detection of pancreatic cancer is difficult, and thus many cases of pancreatic cancer are diagnosed late. When pancreatic cancer is detected, the cancer is usually well developed. Machine learning is an approach that is part of artificial intelligence and can detect pancreatic cancer early. This paper proposes a machine learning approach with the twin support vector machine (TWSVM) method as a new approach to detecting pancreatic cancer early. TWSVM aims to find two symmetry planes such that each plane has a distance close to one data class and as far as possible from another data class. TWSVM is fast in building a model and has good generalizations. However, TWSVM requires kernel functions to operate in the feature space. The kernel functions commonly used are the linear kernel, polynomial kernel, and radial basis function (RBF) kernel. This paper uses the TWSVM method with these kernels and compares the best kernel for use by TWSVM to detect pancreatic cancer early. In this paper, the TWSVM model with each kernel is evaluated using a 10-fold cross validation. The results obtained are that TWSVM based on the kernel is able to detect pancreatic cancer with good performance. However, the best kernel obtained is the RBF kernel, which produces an accuracy of 98%, a sensitivity of 97%, a specificity of 100%, and a running time of around 1.3408 s.

1. Introduction

According to the World Health Organization (WHO), the second leading cause of death in the world is cancer [1]. In 2018, around 17 million new cases of cancer occurred in the world and 9.6 million of the world’s population died from cancer [2]. Cancer has many types depending on location. If cancer occurs in the pancreas, then the cancer is called pancreatic cancer. Pancreatic cancer is the seventh leading cause of cancer deaths in the world and ranks as the 14th most common cancer [3]. Based on the Global Cancer Observatory in 2018, the estimated number of diagnoses of this cancer in the world is 458,918 and the estimated number of deaths is 432,242 [4]. This cancer is expected to be the second leading cause of cancer deaths in the world in 2030 [5].

Pancreatic cancer is a type of cancer that is difficult to detect physically. This is because the pancreas is an organ deep in the body; there are no external lumps or external skin changes such as in cases of breast lesions [6]. In addition, non-specific symptoms such as nausea, anorexia, jaundice, weight loss, and abdominal pain are also factors in the difficulty of detecting pancreatic cancer early [7]. The difficulty in detecting pancreatic cancer early results in many cases of pancreatic cancer being diagnosed late. When pancreatic cancer is detected, cancer is usually well developed, whereas in one study, only 7% of pancreatic cancer was diagnosed as localized disease [6]. If pancreatic cancer is detected too late, then the cancer can develop properly and spread to other body parts so that the cancer is difficult to treat. Therefore, early detection of pancreatic cancer is an important problem.

The problem of detecting pancreatic cancer early is a classification problem in machine learning. Machine learning has been accepted for various classification problems in various fields, one of which is in the field of medicine. In the field of medicine, several machine learning methods are used to detect several types of cancer, namely breast cancer [8,9,10,11], cervical cancer [12,13], ovarian cancer [14,15], colon cancer [16], prostate cancer [17], and lung cancer [18].

Machine learning has also been implemented to detect pancreatic cancer. Qiu et al. [19] implemented several methods of machine learning, namely the decision tree, the $k$-nearest neighbor, and the support vector machine. From the results of the implementation, the support vector machine method is the method that produces the best performance, which is 70% sensitivity and 70% specificity.

The support vector machine (SVM) is one of the well-known machine learning methods; it uses the concept of “maximum margin”, and this concept reduces generalization errors by maximizing margins between two half-separated planes [20]. Many researchers have proposed the development of SVM to obtain better performance. One such development is the twin support vector machine proposed by Jayadeva et al. in 2007.

The twin support vector machine (TWSVM) aims to find two symmetry planes such that each plane has a distance close to one data class and as far as possible from another data class [20]. On several benchmark data sets, TWSVM is not only fast, but shows good generalization [20]. At present, TWSVM has become one of the popular methods because of its excellent learning performance [21]. However, both SVM and TWSVM indirectly need a kernel method to classify data. The kernel method is a method that uses a kernel function that allows an algorithm to operate in a feature space that has a higher dimension by using product operations between images of all data pairs in the feature space [22]. The kernel functions commonly used for SVM methods are the linear kernel, polynomial kernel, and radial basis function (RBF) kernel.

This paper proposes the TWSVM method as a novel approach for early detection of pancreatic cancer. The kernel functions used are the linear kernel, the polynomial kernel, and the RBF kernel. This paper compares the performance of TWSVM with each kernel to get the best kernel for early detection of pancreatic cancer.

2. Materials and Methods

2.1. Kernel Method

The kernel method uses a kernel function that allows an algorithm to operate in a feature space that has a higher dimension by using product operations between images of all data pairs in the feature space [22].

Let ${\mathrm{X}}^{\mathrm{n}}$ be an input space; $\mathrm{F}$ is a feature space, and $\mathsf{\phi }:{\mathrm{X}}^{\mathrm{n}}\to \mathrm{F}$. Kernel function is defined by

$$\mathrm{K}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)=\mathsf{\phi }\left({\mathrm{x}}_{\mathrm{i}}\right).\mathsf{\phi }\left({\mathrm{x}}_{\mathrm{j}}\right)$$

(1)

where ${\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\in {\mathrm{X}}^{\mathrm{n}}$.

Kernel functions that are often used are the linear kernel, the polynomial kernel, and the radial basis function (RBF) kernel. The formulas for the kernel functions are listed below.

2.1.1. Linear Kernel

Linear kernels are the simplest kernel functions, represented by the product $\langle \mathrm{x}, \mathrm{y}\rangle$ [23]. This kernel is the basic kernel that is most often used by SVM because with this kernel, SVM divides data linearly. The linear kernel formula is presented in Equation (2) [24].

$$\mathrm{K}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)={\mathrm{x}}_{\mathrm{i}}^{\mathrm{T}}{\mathrm{x}}_{\mathrm{j}}+\mathrm{c}$$

(2)

2.1.2. Polynomial Kernel

The polynomial kernel is a kernel that is suitable for problems where training data is normalized [24]. From Equation (3), $\mathsf{\sigma }$ is the parameter that must be settled. Variable $\mathrm{c}$ is the constant that is set and variable $\mathrm{d}$ is the degree of polynomial that is set.

$$\mathrm{K}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)={\left({\mathsf{\sigma }\mathrm{x}}_{\mathrm{i}}^{\mathrm{T}}{\mathrm{x}}_{\mathrm{j}}+\mathrm{c}\right)}^{\mathrm{d}};\mathsf{\sigma }>0$$

(3)

2.1.3. Radial Basis Function (RBF) Kernel

The RBF kernel is a kernel family where distance measurements are smoothed by radials function (exponential function) [23]. The RBF kernel is denoted as in Equation (4),

$$\mathrm{K}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)=\exp \left\{-\mathsf{\sigma }{\left|\left|{\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{j}}\right|\right|}^2+\mathrm{c}\right\};\mathsf{\sigma }>0$$

(4)

where $\mathsf{\sigma }$ is the adjustable parameter.

2.2. Twin Support Vector Machine (TWSVM)

The twin support vector machine (TWSVM) is one of the developments of the support vector machine proposed by Jayadeva et al. in 2007. TWSVM aims to find two symmetry planes such that each plane has a distance close to one data class and as far as possible from another data class. [20]. Figure 1 shows an illustration of TWSVM.

Let $\mathrm{D}=$ $\{\left(x_i,{\mathrm{y}}_{\mathrm{i}}\right){|\mathrm{x}}_{\mathrm{i}}\in {\mathrm{X}}^{\mathrm{n}},{\mathrm{y}}_{\mathrm{i}}\in \left\{-1,+1\right\},\mathrm{i}=1,2,\dots ,\mathrm{N}\}$ is the training data. Then let there be ${\mathrm{d}}_1$ data points in class +1 and ${\mathrm{d}}_2$ data points in class −1 such that ${\mathrm{d}}_1+{\mathrm{d}}_2=\mathrm{d}$. We form $\left({\mathrm{d}}_1\times \mathrm{n}\right)$ matrix A, which contains the data points in class +1, and $\left({\mathrm{d}}_2\times \mathrm{n}\right)$ matrix B, which contains the data points in class −1. The two non-parallel hyperplanes are [20]:

$$x^{\mathrm{T}}{w}_1+{\mathrm{b}}_1=0$$

(5)

$$x^{\mathrm{T}}{w}_2+{\mathrm{b}}_2=0$$

(6)

where $x$ is data vector, $w_1$ is weight parameter for first hyperplane, ${\mathrm{b}}_1$ is bias parameter for first hyperplane, $w_2$ is weight parameter for second hyperplane, and ${\mathrm{b}}_2$ is bias parameter for second hyperplane.

The TWSVM method is obtained by solving the following pair of quadratic programming problems [20]:

TWSVM 1:

$$\underset{\begin{array}{c}{w}_1,{\mathrm{b}}_1,{\mathsf{\xi }}_2\\ \end{array}}{\min }\frac{1}{2}{\left|\left|\mathrm{A}{w}_1+{\mathrm{e}}_1{\mathrm{b}}_1\right|\right|}^2+{\mathrm{c}}_1{\mathrm{e}}_2^{\mathrm{T}}{\mathsf{\xi }}_2$$

(7)

subject to

$$-\left(\mathrm{B}{w}_1+{\mathrm{e}}_2{\mathrm{b}}_1\right)\ge {\mathrm{e}}_2-{\mathsf{\xi }}_2$$

(8)

$${\mathsf{\xi }}_2\ge 0$$

(9)

and

TWSVM 2:

$$\underset{\begin{array}{c}{w}_2,{\mathrm{b}}_2,{\mathsf{\xi }}_1\\ \end{array}}{\min }\frac{1}{2}{\left|\left|\mathrm{B}{w}_2+{\mathrm{e}}_2{\mathrm{b}}_2\right|\right|}^2+{\mathrm{c}}_2{\mathrm{e}}_1^{\mathrm{T}}{\mathsf{\xi }}_1$$

(10)

subject to

$$-\left(\mathrm{A}{w}_2+{\mathrm{e}}_2{\mathrm{b}}_2\right)\ge {\mathrm{e}}_1-{\mathsf{\xi }}_1$$

(11)

$${\mathsf{\xi }}_1\ge 0$$

(12)

where ${\mathrm{c}}_1>0$ and ${\mathrm{c}}_2>0$ are penalty parameters, ${\mathsf{\xi }}_1$ and ${\mathsf{\xi }}_2$ are slack variables, and ${\mathrm{e}}_1$ and ${\mathrm{e}}_2$ are vectors of ‘ones’, i.e., each component is ‘one’ only [20].

The two hyperplanes of TWSVM with kernel [20]:

$$\mathrm{K}\left({\mathrm{x}}^{\mathrm{T}},{\mathrm{C}}^{\mathrm{T}}\right)u_1+{\mathrm{b}}_1=0$$

(13)

$$\mathrm{K}\left({\mathrm{x}}^{\mathrm{T}},{\mathrm{C}}^{\mathrm{T}}\right)u_2+{\mathrm{b}}_2=0$$

(14)

where ${\mathrm{C}}^{\mathrm{T}}={\left[\mathrm{A},\mathrm{B}\right]}^{\mathrm{T}}$, $u_1,u_2\in {\mathrm{R}}^{\mathrm{d}}$, and $\mathrm{K}$ is the kernel matrix corresponding to an appropriately chosen kernel function [11].

The kernel TWSVM can be obtained by solving the optimization problems [20]:

KTWSVM 1:

$$\underset{\begin{array}{c}{w}_1,{\mathrm{b}}_1,{\mathsf{\xi }}_2\\ \end{array}}{\min }\frac{1}{2}{\left|\left|\mathrm{K}\left(\mathrm{A},{\mathrm{C}}^{\mathrm{T}}\right)u_1+{\mathrm{e}}_1{\mathrm{b}}_1\right|\right|}^2+{\mathrm{c}}_1{\mathrm{e}}_2^{\mathrm{T}}{\mathsf{\xi }}_2$$

(15)

subject to

$$-\left(\mathrm{K}\left(\mathrm{B},{\mathrm{C}}^{\mathrm{T}}\right)u_1+{\mathrm{e}}_2{\mathrm{b}}_1\right)\ge {\mathrm{e}}_2-{\mathsf{\xi }}_2$$

(16)

$${\mathsf{\xi }}_2\ge 0$$

(17)

and

KTWSVM 2:

$$\underset{\begin{array}{c}{w}_2,{\mathrm{b}}_2,{\mathsf{\xi }}_1\\ \end{array}}{\min }\frac{1}{2}{\left|\left|\mathrm{K}\left(\mathrm{B},{\mathrm{C}}^{\mathrm{T}}\right)u_2+{\mathrm{e}}_2{\mathrm{b}}_2\right|\right|}^2+{\mathrm{c}}_2{\mathrm{e}}_1^{\mathrm{T}}{\mathsf{\xi }}_1$$

(18)

subject to

$$-\left(\mathrm{K}\left(\mathrm{A},{\mathrm{C}}^{\mathrm{T}}\right)u_2+{\mathrm{e}}_2{\mathrm{b}}_2\right)\ge {\mathrm{e}}_1-{\mathsf{\xi }}_1$$

(19)

$${\mathsf{\xi }}_1\ge 0$$

(20)

where ${\mathrm{c}}_1>0$ and ${\mathrm{c}}_2>0$ are penalty parameters, ${\mathsf{\xi }}_1$ and ${\mathsf{\xi }}_2$ are slack variables, ${\mathrm{e}}_1$ and ${\mathrm{e}}_2$ are vectors of ‘ones’, i.e., each component is ‘one’ only, ${\mathrm{C}}^{\mathrm{T}}={\left[\mathrm{A},\mathrm{B}\right]}^{\mathrm{T}}$, $u_1,u_2\in {\mathrm{R}}^{\mathrm{d}}$, and $\mathrm{K}$ is the kernel matrix corresponding to an appropriately chosen kernel function [20].

2.3. k-Fold Cross Validation

In this paper, to obtain a model and evaluate the model obtained, the dataset was divided into training data and testing data. Training data are data used by machines to recognize and study pancreatic cancer data patterns, while testing data are data used to evaluate models obtained after a machine learns data patterns. The dataset was divided into training data and testing data using the $k$-fold cross validation method. $k$-fold cross validation is a method for selecting training data samples. The $k$-fold cross validation method divided the dataset into $k$ sections of equal size [25]. Each subsample was taken as validation data to test the model and repeat the process $k$ times [25]. The advantage of this method is the repetition of random samples as training data and validation [25].

2.4. Proposed Method

In this paper, the method proposed for early detection of pancreatic cancer consisted of four stages. In the first stage, the data were divided into training data and testing data using k-fold cross validation. The k value chosen was 10. This means that the dataset was divided into 10 samples of the same size. In this dataset, 9 samples were used as training data, and 1 sample was used as testing data. In the second stage, the training data were used by the TWSVM method based on linear kernels, polynomial kernels, and RBF kernels to study data patterns and build classification models. In the third stage, the classification model obtained was evaluated based on the parameters of accuracy, sensitivity, specificity, and required running time. This evaluation used testing data. After that, evaluation parameters generated by each kernel were compared to find out the best kernel to detect pancreatic cancer early using the TWSVM method. The stages carried out in this paper used the Python 3 programming language.

3. Results and Discussions

3.1. Data

In this paper, the data used were pancreatic cancer data obtained from Al Islam Hospital Bandung, Indonesia. The data consisted of 203 samples and six features. The six features were the diagnosis of patient, namely pancreatic cancer (Y) and not pancreatic cancer (N), and blood tests which consisted of cancer antigens, hemoglobin, leukocytes, hematocrit, and platelets. The diagnosis feature became a target feature in detecting pancreatic cancer.

Table 1. Part of the pancreatic data.

No	CA (U/mL)	Hemoglobin (g/dL)	Leukocytes (sel/uL)	Hematocrit (%)	Platelets (sel/uL)	Diagnosis
1	5.73	12.1	10,200	36.7	143,000	N
2	8.05	11.8	11,300	35.9	222,000	N
3	86.21	10.1	12,800	34.1	346,000	Y
4	87.13	12	11,700	36.7	612,000	Y

shows part of the data.

In the data shown in Table 1, the diagnosis feature is a categorical feature. This feature must be changed into a numeric feature for the proposed method to work. Therefore, based on the TWSVM method, the Y category in the diagnostic feature was transformed into +1 and the N category in the diagnostic feature was transformed into −1.

3.2. Confusion Matrix

In this paper, a confusion matrix was used to assist in calculating the evaluation parameters of the classification model.

Table 2. Confusion matrix.

		Predict
		Pancreatic Cancer (Y)	Not Pancreatic Cancer (N)
Actual	Pancreatic Cancer (Y)	True Positive (TP)	False Negative (FN)
	Not Pancreatic Cancer (N)	False Positive (FP)	True Negative (TN)

shows the confusion matrix used to evaluate the TWSVM classification model based on the kernel for early detection of pancreatic cancer.

Explanation:

• TP = Many cases of pancreatic cancer are predicted to be correct
• TN = Many cases of not pancreatic cancer are predicted to be correct
• FP = Many cases of not pancreatic cancer are predicted to be wrong (predicted as pancreatic cancer)
• FN = Many pancreatic cancer cases are predicted to be wrong (predicted as not pancreatic cancer)

3.3. Evaluation Parameters

The parameters to evaluate the performance of the TWSVM classification model were accuracy, sensitivity, specificity, and required running time.

Table 3. Parameter formulae to evaluate the classification model.

Parameter	Formula	Explanation
Accuracy	$\frac{\left(\mathrm{TN}+\mathrm{TP}\right)}{\left(\mathrm{FN}+\mathrm{TP}+\mathrm{FP}+\mathrm{TN}\right)}\times 100\%$	Comparison between the number of cases of pancreatic cancer and not pancreatic cancer that identified correctly with the total number of all cases
Sensitivity	$\frac{\mathrm{TP}}{\left(\mathrm{FN}+\mathrm{TP}\right)}\times 100\%$	Proportion of pancreatic cancer cases identified correctly
Specificity	$\frac{\mathrm{TN}}{\left(\mathrm{FP}+\mathrm{TN}\right)}\times 100\%$	Proportion of not pancreatic cancer cases identified correctly

shows the formula for accuracy, sensitivity, and specificity.

3.4. Results

In this section, we discuss the performance evaluation of the TWSVM classification model with a linear kernel, polynomial kernel, and RBF kernel. The TWSVM classification model based on kernel that is proposed in this paper refers to research conducted by [20] which detects hepatitis using TWSVM with a linear kernel and a RBF kernel. In research conducted by the authors of [20], the accuracy produced by the RBF kernel is superior to that of a linear kernel. This indicates that the RBF kernel is the appropriate kernel in detecting hepatitis using TWSVM.

In this paper, we have built the TWSVM classification model with a linear kernel, a polynomial kernel, and a RBF kernel in detecting pancreatic cancer.

Table 4. Results of TWSVM classification model based on kernel.

Classification Model	Accuracy (%)	Sensitivity (%)	Specificity (%)	Running Time (seconds)
TWSVM with Linear Kernel	92%	86%	95%	1.2811
TWSVM with Polynomial Kernel with $\mathrm{d}=4$	80%	75%	83%	1.2040
TWSVM with RBF Kernel with $\mathsf{\sigma }=0.05$	98%	97%	100%	1.3408

presents a comparison of TWSVM performance with linear kernels, polynomial kernels with $\mathrm{d}=4$, and RBF kernels with $\mathsf{\sigma }=0.05$. The performance evaluation parameters compared are accuracy, sensitivity, specificity, and running time.

From Table 4, it can be seen that for accuracy, TWSVM models with RBF kernel have the most superior accuracy compared to the linear kernel and the polynomial kernel, reaching 98%. That is, the TWSVM model with the RBF kernel correctly detected 98% of the total cases. The lowest accuracy is the TWSVM model with a polynomial kernel, with a resulting percentage of 80%. In addition, for consideration of sensitivity and specificity, TWSVM models with RBF kernel have percentages that are also superior than the linear kernel and polynomial kernel, which are 97% sensitivity and 100% specificity. This means that the TWSVM model with the RBF kernel is able to detect 98% of cases correctly, with a truth of 97% of all cases of pancreatic cancer, and 100% of all cases of non-pancreatic cancer.

Based on consideration of accuracy, sensitivity, and specificity produced by the TWVM model with a linear kernel, a polynomial kernel, and a RBF kernel, overall the TWSVM model with RBF kernel is the most superior kernel. This means that the pancreatic cancer dataset can be separated almost precisely by RBF function. However, for consideration of running time, the TWSVM model with RBF kernel has the longest running time compared to linear and polynomial kernels, which is around 1.3408 s. The TWSVM model with the polynomial kernel actually produces the fastest running time, which is around 1.2040 s. Even so, the running time produced by the RBF kernel is quite good and acceptable for detecting pancreatic cancer early. Thus, the RBF kernel is the best kernel for TWSVM in detecting pancreatic cancer early.

4. Conclusions

Early detection of pancreatic cancer is very important so that the handling of pancreatic cancer does not occur too late, before the cancer spreads to other organs in the body. However, early detection of pancreatic cancer is difficult because this cancer has non-specific symptoms. The twin support vector machine method based on the kernel can help detect pancreatic cancer early, based on blood tests. The most appropriate kernel for the TWSVM method in detecting pancreatic cancer is the RBF kernel which produces an accuracy of 98%, sensitivity of 97%, and 100% specificity, and the required running time is 1.3408 s.