Deep Transfer Learning in Diagnosing Leukemia in Blood Cells

Abstract

Leukemia is a fatal disease that threatens the lives of many patients. Early detection can effectively improve its rate of remission. This paper proposes two automated classification models based on blood microscopic images to detect leukemia by employing transfer learning, rather than traditional approaches that have several disadvantages. In the first model, blood microscopic images are pre-processed; then, features are extracted by a pre-trained deep convolutional neural network named AlexNet, which makes classifications according to numerous well-known classifiers. In the second model, after pre-processing the images, AlexNet is fine-tuned for both feature extraction and classification. Experiments were conducted on a dataset consisting of 2820 images confirming that the second model performs better than the first because of 100% classification accuracy.

1. Introduction

Diagnosis is performed by a physician to detect the presence or absence of a certain disease in a patient according to a particular dataset, which may include signs, symptoms, medical images, and exams. An incorrect diagnosis can have adverse consequences, for example, prescription of drugs with side effects, on a patient’s health. As well as increasing the costs of treatment, incorrect diagnoses may complicate treatment procedures [1]. To help physicians achieve high diagnostic accuracy, many assistant systems were proposed. Many diseases, including glaucoma [2], skin cancer [3], breast cancer [4], and leukemia [5], are already addressed by such systems. Early and accurate diagnoses could effectively reduce treatment costs, increase the probability of remission, or even prolong the lives of patients [1].

Leukemia is a common fatal disease that threatens the lives of many teenagers and children. Infants younger than five years of age are at increased risk [1]. A 2012 study showed that about 352,000 adults and children all over the world develop leukemia, which starts in the bone marrow and is distinguished by the number of white cells increasing in an abnormal manner [1]. This disease has several causes, such as exposure to radiation and certain chemicals, as well as family history [6]. Diagnoses can be performed via a variety of tests, such as physical examination, blood test, blood count, and bone marrow biopsy. Microscopic analysis is considered the most cost-effective procedure for initial diagnoses, but it is usually performed manually by an operator who is vulnerable to fatigue that could result from having to perform many tests in a single day. Moreover, such manual diagnoses are unreliable in themselves, as they are tedious, time-consuming, and subject to inter-observer variations. Hence, there is a need to build automated, low-cost systems that can differentiate between healthy and unhealthy blood smear images with high accuracy but without manual intervention [1,7].

Many traditional computer-aided systems use image processing and machine-learning techniques that usually involve several steps, including pre-processing, segmentation, feature extraction, and classification. However, the success of each step depends on the success of the preceding step. For example, the success of classification depends on the success of the preceding feature extraction, which itself depends on the success of the preceding segmentation. Hence, high classification accuracy requires the success of all steps, each of which is non-trivial and problem-dependent [1,7].

Recently, deep learning achieved many breakthroughs in different fields such as computer vision, natural language processing, and object recognition. Deep neural networks, such as convolutional neural networks (CNNs), can be used effectively to build computer-aided diagnostic systems. However, the design and training of deep neural networks are non-trivial and time-consuming tasks. Hence, instead of building a deep neural network from scratch, we use the concept of transfer learning, in which a deep network that achieved success in solving a certain problem is tuned to solve another problem.

This paper proposes two classification models that are based on transfer learning and can distinguish between healthy and unhealthy blood smear images with high accuracy. These models employ AlexNet, which is a deep CNN that achieved huge success in the image classification challenge, ImageNet 2012.

The remaining sections of this paper are arranged as follows: Section 2 covers some of the traditional and end-to-end-based methods described in the literature. Section 3 presents our two proposed classification models. Section 4 discusses their implementation in our experiments. Section 5 discusses the results obtained. Finally, Section 6 concludes the paper.

2. Related Studies

This section covers some of the research conducted in the field of diseases, especially blood disease detection and diagnosis. Some of these studies pertain to traditional methods [8,9], which consist of several steps such as pre-processing, segmentation, feature extraction, and classification, whereas other methods pertain to deep-learning-based methods [6,10,11,12], which use deep neural networks for end-to-end learning tasks.

2.1. Traditional Methods

Sajjad et al. [8] proposed a mobile-cloud-assisted framework that segments and classifies leukocytes into five classes. The framework firstly segmented white blood cells (WBCs) by a color k-means algorithm, which removed irrelevant components via morphological operations. Various types of features, including geometric, statistical, and textural, were extracted by principal component analysis. Finally, classification was performed by an ensemble multi-class support vector machine (SVM). Experiments on a dataset of 1030 blood smear WBC images produced an average accuracy of 98.6% for this framework. Kumar et al. [13] presented an automated detection system for acute leukemia. The system started with the pre-processing of noise and blurring in microscopic digital images. A variety of features, including color, geometric, textural, and statistical, were extracted and classified as benign or malignant. Two classification models, k-nearest neighbor (K-NN) and naïve Bayes, were used. Experiments on a dataset of 60 blood samples revealed the superiority of the K-NN classifier with its 92.8% classification accuracy.

Supardi et al. [14] introduced a classification system that differentiates between two types of acute leukemia: acute myelogenous leukemia (AML) and acute lymphocytic leukemia (ALL). Twelve features were manually extracted from image samples. Finally, a K-NN classifier was used for classification. Experiments on a dataset of 1500 images produced 86% accuracy. Madhukar et al. [15] proposed an AML classification system that enhanced image contrasting and extracted five features. An SVM classifier performed the classification. Experiments on a dataset of 50 images produced 93.5% classification accuracy.

Setiawan et al. [16] introduced a system that could classify the cells in AML of subtypes M4, M5, and M7. Firstly, the cells were segmented by a color k-means algorithm. Then, six statistical features were extracted and input into a multi-class SVM classifier. The results produced about 87% segmentation accuracy and 92.9% classification accuracy in the best case. Faivdullah et al. [17] proposed a three-layered framework with feature extraction, coding, and classification. Given a blood smear image of a certain patient, the objective of this framework was to decide whether a patient has leukemia and to identify which type. Dense scale-invariant feature transform was used in feature extraction. Then, the dimensionality of the extracted feature vectors was reduced in the coding layer. Finally, a multi-class SVM classifier performed the classification. Experiments on a dataset of 400 samples produced 79.38% classification accuracy.

Laosai and Chamnongthai [18] presented an AML classification system that segments nuclei by k-means and contour signature approaches. Then, feature extraction for cell size, cell color, etc., was performed via morphology. Experiments on a dataset of 100 images showed that the SVM classifier had an accuracy of up to 92%.

Patel and Mishra [19] presented an automated leukemia system based on microscopic images. The system started with noise and blurring removal during pre-processing. Then, the WBCs were segmented by k-means and Zack algorithms. Then, several features, including color, statistical, geometric, and textural, were extracted. Finally, an SVM classifier distinguished between normal and abnormal images. Experiments on a dataset of 27 images produced 93.57% accuracy.

Dwivedi [20] presented a system to differentiate between ALL and AML by using the microarray gene profile and an artificial neural network for classification. The system was evaluated with a dataset of 46 samples. With 98% accuracy, the artificial neural network-based classifier achieved the best results of all the classification models. Further work to detect AML was suggested by Abdeldaim et al. [9]. WBCs were segmented by a combination of approaches, including histogram equalization and the Zack algorithm. Then, various features, including color, shape, and texture, were extracted and normalized. Finally, a number of classifiers were used for classification. The system was evaluated with a dataset of 260 images. With 96.01% accuracy, the best result was achieved by the K-NN classifier. Finally, Sahlol et al. [21] presented an automated system to diagnose ALL. The system segmented the WBCs by histogram equalization and the Zack algorithm. Then, several features, including color, texture, shape, and hybrid features, were extracted. Then, a social spider optimization algorithm selected the most significant features. Finally, a number of classifiers were used for classification. The system was evaluated with a dataset of 260 images. With 95.67% accuracy, the K-NN classifier produced the best results.

2.2. Deep-Learning-Based Methods

Yu et al. [11] presented a classification system for WBCs by applying a number of deep learning nets. This system was evaluated with a dataset of 2000 microscopic images of seven kinds of WBCs and compared to several traditional methods. With an average accuracy of 88.5%, the obtained results confirmed the superiority of using a CNN. Thanh et al. [6] proposed a CNN-based method to differentiate between normal and abnormal blood cell images to detect leukemia at an early stage. Evaluated with a dataset of 1188 images, the proposed method achieved 96.6% classification accuracy.

Vogado et al. [10] introduced a leukemia diagnosing system performed on 377 images. This system used transfer learning and a CNN network to extract the discriminant features. Then, feature selection used information gain. Finally, an SVM-based classifier performed the classification. With three heterogeneous datasets used for validation, this system achieved approximately 99% classification accuracy. Zhao et al. [22] proposed a system to detect and classify the WBCs, which were firstly detected in microscopic images by a number of approaches. Then, a CNN was used for feature extraction. Finally, an SVM and a random forest classifier were jointly used for classification. The proposed system was evaluated by a dataset that was a collection of some standard datasets. The results produced an average classification accuracy of 92.8%.

Habibzadeh et al. [23] presented a classification model for WBCs based on both transfer learning and deep learning. The proposed method started with a pre-processing step, and then employed transfer learning for feature extraction. Finally, classification was performed using Inception and ResNet deep networks. The method was evaluated with a dataset of 1244 WBCs. The results achieved 99.84% accuracy in the best case. Lin et al. [24] presented a leukocyte classification system. Firstly, an improved method extracted complex leukocytes according to a variation of the k-means algorithm. Then, classification was performed by a CNN. A dataset of 368 images was used for evaluation. The results achieved 98.96% classification accuracy.

Rehman et al. [25] proposed a classification system for ALL and its subtypes. Firstly, the region of interest (lymphoblast) was segmented from bone marrow images using a simple threshold method. Then, AlexNet was employed for classification. The evaluation used a dataset of 330 images. The results achieved 97.78% classification accuracy. Shafique and Tehsin [26] classified ALL and its subtypes using a pre-trained AlexNet. The ALL-IDB2 dataset, which included 260 images, was used for evaluation. The results achieved an average classification accuracy of 96.06%.

Wang et al. [27] provided a method to detect and classify WBCs according to an ensemble classifier that fuses the output of a number of CNNs. Experiments with a dataset of 3000 images for each class produced an average classification accuracy of 99.37% in the best case. Pansombut et al. [12] proposed another method to classify ALL and its subtypes using a CNN network called ConVNet. A dataset of 363 images was used for evaluation and 80% accuracy was achieved. Experiments confirmed the superiority of this method over numerous traditional methods.

Sawada et. al. [28] suggested a deep learning approach to change goal vectors. Learning to move may offer a solution to this issue. The standard method increases the efficiency of the classification, but it does not consider the relationship between the vectors of the relation. From this point of view, the suggested modification of the relationship vector is based on restricted repulsive forces in pairs. The results of their proposed method indicated that the task-specific layer can be reused by appropriately estimating the relation vectors.

Gu et. al. [29] suggested two approaches for a novel task of understanding cross-domain skin disease. The dataset was initially pre-trained with ImageNet; then, using a completely supervised deep convolution neural network classifier, they analyzed a two-step progressive transfer learning technique by fine-tuning the network on two skin disease datasets. Next, they suggested implementing adversarial learning as a technique for domain adaptation to perform invariant attribute translation from the source to target domain in order to boost the output in recognition. Their experiments showed how effective the proposed approach was in solving the problem of domain shifts.

Zamir et. al. [30] proposed a fully analytical approach to visual task space structure modeling. This was achieved by discovering (first-order and higher-order) learning dependencies through a dictionary of 26 two-dimensional (2D), 2.5D, 3D, and semantic tasks in latent space. The result was a taxonomic mapping for role transition learning. They studied the implications of this structure, e.g., nontrivial relationships that arose, before manipulating them to reduce the demand for branded data. They had a collection of tools to compute and check this taxonomic structure, including a user who could use it to find supervisory policies.

The quality of results mainly depends on the quality of the used features. The previous approaches depended on traditional machine learning and image processing methods, which take a long time and introduce a relatively high recognition error rate, particularly for the rare kinds of leukocytes [1].

To support state-of-the-art representation learning techniques, deep learning and strong solutions for generic behavior learning, through reinforcement learning, can be used to create models for learning processes that cover both the appearance of the object and the search strategy. Initial results in the task of landmark detection proved that this type of approach is very promising in terms of accuracy and robustness. In addition, the speed is improved by avoiding typical inefficient scanning routines [31].

The development of deep neural networks contributed to the evolution of a second category of leukemia detection methods called end-to-end learning-based methods. The idea behind this category is to design and build a deep neural network that takes the input image and returns the output class without undergoing the different challenging tasks involved in traditional methods such as region of interest (ROI) segmentation and feature extraction. A block diagram of end-to-end learning-based trademark image retrieval methods is shown in Figure 1.

Figure 1. A block diagram of leukemia detection method based on end-to-end learning.

3. Proposed Method

Two classification models are proposed here to distinguish between microscopic images depicting healthy tissue and leukemia. Transfer learning was adopted for both models, which employed pre-trained deep neural networks. Transfer learning eliminates the time and effort needed to design and train such networks from scratch. According to Castelluccio et al. [32], there are two methods to apply transfer learning. The first method includes obtaining features extracted from the input images by obtaining the values of the last fully connected layer (FC) of the net [33], before using another classifier for classification. The second method involves modifying the structure of the network by eliminating the high-level layers. This process is known as network fine-tuning. In this study, both methods were adopted and implemented by our proposed models.

As shown in Figure 2, the first classification model comprises three main steps: image pre-processing, feature extraction, and classification. The pre-processing step includes many operations such as converting blood images into a red–green–blue (RGB) model, resizing the images to fixed sizes, and performing data augmentation to overcome the lack of large datasets. In feature extraction, a pre-trained AlexNet is employed to extract a set of features from each image for later use in classification to differentiate between healthy and leukemia-affected images. The classification uses a number of well-known classifiers, such as SVMs, linear discriminants (LDs), decision trees (DTs), and K-NNs.

Figure 2. Diagram of first classification model.

The second classification model consists of only two steps: image pre-processing, and feature extraction and classification (see Figure 3). AlexNet is used due to the huge computing power needed when using GoogLeNet and VGG-19, and it is employed in this work for both feature extraction and classification of blood microscopic images. Detailed descriptions of both models are presented below.

Figure 3. Diagram of second classification model.

3.1. First Classification Model

Each step of this model is described below

3.1.1. Image Pre-Processing

In this step, a number of operations are applied to the input blood microscopic images, which are firstly converted into an RGB color model. Then, their sizes are fixed to 227 × 227. Finally, data augmentation is performed to overcome the absence of a large dataset, because deep neural networks require large datasets to accomplish their training and testing phases. Data augmentation consists of three operations: translation, reflection, and rotation. In translation, the images are shifted along the X-axes and Y-axes with selected values being randomly bounded by the interval [15–25]. In the reflection process, the images are mirrored along the vertical axis. Finally, in the rotation process, the images are rotated right or left with a random rotation angle of values bounded by the interval [25–125] with a step equal to five. Examples of data augmentation are shown in Figure 4, in which columns (a–d) depict original, translated, reflected, and rotated images, respectively.

Figure 4. Data augmentation, in which columns (a–d) depict original, translated, reflected, and rotated images, respectively.

3.1.2. Feature Extraction

CNN is one of the main network architectures employed effectively for machine-learning processes. The main reason behind the success of CNNs is their ability to perform tasks regardless of tilting, translation, and scaling [11]. As shown in Figure 5, three main types of layers are included in CNN architecture: convolutional, pooling, and fully connected layers. Convolutional layers calculate the output of neurons by adding the bias to the weighted sum and by applying an activation function called a rectified linear unit (ReLu), which can be computed by Equation (1).

ReLu (x) = max (0, x).

(1)

Figure 5. General architecture of convolution neural networks (CNNs) [1].

The pooling layers control overfitting by reducing the number of features resulting from the convolutional layers. The fully connected layer gathers all the features of the descriptors that are to be classified by the last layer [6].

In our first classification model, feature extraction is executed according to the concept of transfer learning as implemented by AlexNet, which was recommended by Krizhevsky et al. [34] to compete in the ILSVRC-2010 challenge for classifying the ImageNet database. AlexNet includes five convolutional and three fully connected layers, as well as max-pooling layers. All eight layers must be trained. In AlexNet, the overfitting problem is addressed in many ways, including normalizing the local response, data augmentation, and the dropout approach, which sets the output of hidden neurons to zero, with a probability of 0.5. Dropout is performed on the first two fully connected layers. In our first model, features were extracted by obtaining the values of the last fully connected layers. The length of each feature vector was 4096.

3.1.3. Classification Approaches

In our first model, a number of classifiers were used to classify the feature vectors, which were obtained from the previous step, into one of two classes: healthy or unhealthy. The classifiers used included a DT with a max-split equal to 20 [35], an LD [36], an SVM with different kernel functions [37], and a K-NN using Euclidian distance with k = 1 [38].

3.2. Second Classification Model

As shown in Figure 3, our second proposed classification model included two main steps: pre-processing, and feature extraction and classification. Pre-processing was exactly the same as that in the first model, whereas feature extraction and classification were performed by AlexNet, whose architecture was fine-tuned to suit our problem. The last three layers, which were the last fully connected, SoftMax, and output layers of the original AlexNet, were frozen and replaced with three other layers that suited our classification problem. The resulting network was trained with the collected blood microscopic images.

4. Implementation and Experiments

This section describes the implementation of our models, the experiments conducted, and the dataset used.

4.1. Dataset Description

Our dataset consisted of 564 (282 healthy and 282 unhealthy) blood microscopic images. Samples from the original dataset are shown in Figure 6, in which the first row includes leukemia-free samples, while the second row includes leukemia-affected samples. The healthy images were collected from Reference [39] while the unhealthy images were collected from Reference [40]. Samples were captured with an optical laboratory microscope coupled with a camera, and their images were prepared for use as an appropriate dataset in the learning process. After data augmentation, the number of images reached 2820.

Figure 6. Samples from the used dataset.

4.2. Implementation and Experiments

MATLAB 2018a was used to implement both models and several experiments were conducted to evaluate their performance in terms of precision, recall, accuracy, and specificity, which are defined below.

Precision = TP/(TP + FP),

(2)

Recall = TP/(TP + FN),

(3)

Accuracy = (TP + TN)/(TP + TN + FP + FN),

(4)

Specificity = TN/(TN + FP),

(5)

where TP is true positive, TN is true negative, FP is false positive, and FN is false negative. For the first and second models, the classifiers were evaluated by a 10-fold cross-validation approach. The K-NN classifier was implemented by setting k = 1, and the maximum number of iterations was set to 30. Gaussian, linear, and cubic kernel functions were used with the SVM classifier. For the second model hold out, the dataset was divided into 80% training data and 20% test data. During training, the batch size was set to five, the number of epochs was set to six, and the learning rate was set to 1 × 10⁻⁴. All the experiments were conducted with a graphics processing unit (GPU) NVIDIA GE FORCE 920M 4 GDDRAM. The results obtained for both models are shown in Table 1.

Table 1. Performance of the proposed models. DT—decision tree; LD—linear discriminant; SVM—support vector machine; K-NN—k-nearest neighbor.

Methods		Performance Metrics
		Precision	Recall	Accuracy	Specificity
First Model	DT	95.69%	95.96%	95.82%	95.67%
	LD	99.64%	97.38%	98.51%	99.65%
	SVM-Linear	99.93%	98.72%	99.33%	99.93%
	SVM-Gaussian	99.93%	99.43%	99.68%	99.93%
	SVM-Cubic	99.93%	99.65%	99.79%	99.93%
	K-NN	99.64%	98.44%	99.04%	99.65%
Second Model	CNN (Alex Net): Cross fold	99.65%	100.00%	99.82%	99.65%
	CNN (Alex Net)	100%	100%	100%	100%

As can be seen in Table 1, for the first model, the SVM classifier with the cubic kernel function scored the best values in all the metrics, while the LD and K-NN classifiers achieved close results. DT performed the worst. The second model, which employed AlexNet, scored higher than the first model in all metrics. A graphical representation of the results is shown in Figure 7.

Figure 7. Samples from the used dataset.

Screenshots for the training and the confusion matrix of the second model are shown in Figure 8 and Figure 9, respectively.

Figure 8. Screenshot for training of the second model.

Figure 9. Confusion matrix of the second model.

We compared this system to others proposed in the literature and the results are tabulated in Table 2.

Table 2. Comparison of accuracy obtained by the second model with that of other models in the literature.

Work	Number of Images	Methodology		Accuracy
		Feature Extraction	Classifier
Thanh et al. [6]	1188	CNN	FC	96.6%
Vogado et al. [10]	377	CNN	SVM	99%
Rehman et al. [25]	330	CNN	FC	97.78%
Shafique and Tehsin [26]	260	CNN	FC	96.06%
Pansombut et al. [12]	363	CNN	FC	80%
Proposed work	2820	CNN	FC	100%

5. Discussion

Nearly all the works listed in Table 2 used CNNs for both feature extraction and classification. Only Vogado et al. [10] used a CNN for feature extraction but an SVM-based classifier for classification. Our second model had higher classification accuracy than these works in terms of a larger dataset of 2820 images.

6. Conclusions and Future Studies

The early detection of leukemia can help effectively in its treatment. This study proposed two classification models distinguishing between leukemia-free and leukemia-affected blood microscopic images. Both models employ transfer learning. In the first model, a pre-trained CNN known as AlexNet is employed to extract the discriminant features and other well-known classifiers, such as DT, LD, SVM, and K-NN, are employed for classification. Experiments demonstrated the superiority of the SVM classifier. The second model employs AlexNet for both feature extraction and classification. Experiments for this model demonstrated its superiority to the first model with respect to various performance metrics. A future study could be extended to differentiate among the different types of leukemia rather than simply marking images as leukemia-free or leukemia-affected.