A New Deep Hybrid Boosted and Ensemble Learning-Based Brain Tumor Analysis Using MRI

Brain tumor analysis is essential to the timely diagnosis and effective treatment of patients. Tumor analysis is challenging because of tumor morphology factors like size, location, texture, and heteromorphic appearance in medical images. In this regard, a novel two-phase deep learning-based framework is proposed to detect and categorize brain tumors in magnetic resonance images (MRIs). In the first phase, a novel deep-boosted features space and ensemble classifiers (DBFS-EC) scheme is proposed to effectively detect tumor MRI images from healthy individuals. The deep-boosted feature space is achieved through customized and well-performing deep convolutional neural networks (CNNs), and consequently, fed into the ensemble of machine learning (ML) classifiers. While in the second phase, a new hybrid features fusion-based brain-tumor classification approach is proposed, comprised of both static and dynamic features with an ML classifier to categorize different tumor types. The dynamic features are extracted from the proposed brain region-edge net (BRAIN-RENet) CNN, which is able to learn the heteromorphic and inconsistent behavior of various tumors. In contrast, the static features are extracted by using a histogram of gradients (HOG) feature descriptor. The effectiveness of the proposed two-phase brain tumor analysis framework is validated on two standard benchmark datasets, which were collected from Kaggle and Figshare and contain different types of tumors, including glioma, meningioma, pituitary, and normal images. Experimental results suggest that the proposed DBFS-EC detection scheme outperforms the standard and achieved accuracy (99.56%), precision (0.9991), recall (0.9899), F1-Score (0.9945), MCC (0.9892), and AUC-PR (0.9990). The classification scheme, based on the fusion of feature spaces of proposed BRAIN-RENet and HOG, outperform state-of-the-art methods significantly in terms of recall (0.9913), precision (0.9906), accuracy (99.20%), and F1-Score (0.9909) in the CE-MRI dataset.


Introduction
The brain is a complex and one 8th vital organ of the human body, controlling the nervous system.Irregular and uncontrolled growth of cells in the brain can cause a brain tumor.A brain tumor is usually categorized into primary and secondary tumors.The creation of brain tumors is not identifiable along with the growth rate, and it has the world's highest mortality ratio of cancers.
Primary brain tumors are devised in the brain tissues, whereas secondary tumors produce in some other part of the body and are shifted to the brain through blood flow.Among the primary-brain tumors, meningioma, glioma, and pituitary are harmful types of brain tumors and most challenging for their early detection and effective treatment.Furthermore, these may lead to critical conditions if not addressed timely [1].
Early detection and classification of brain tumors with high prognosis accuracy is the most critical step for diagnosis and treatment to save the patient's life.However, the manual analysis of brain MR images is laborious for radiologists and doctors to detect and localize the tumor and normal tissues and categorize the tumors from medical images [2].A computer-aided diagnosis (CADx) system is essential to overcome this problem.It needs to be implemented to relieve the workload and facilitate radiologists or doctors in medical images analysis.In the past, numerous researchers proposed several robust and accurate solutions to automate the brain tumor detection and classification task.
Conventional Machine learning (ML) based approaches have been employed for brain tumor analysis.However, ML-based techniques entail manual features extraction and classification and also are used on limited data.Deep learning (DL) has combined feature extraction and classification into a self-learning manner on a significant amount of labeled data, which considerably improved the performance.Moreover, CNN is a branch of DL, specially designed for image or two-dimensional (2D) data.It only takes datasets with minimal preprocessing and captures various features from MR images without human intervention [3].Deep CNN models are largely used for brain tumor detection.However, brain tumor analysis is highly challenging because of variable morphological structure, tumor appearance in an image, and illumination effects which needs an efficient DL-based brain tumor analysis system to strengthen the radiologist's decision.
In this regard, we develop a deep boosted hybrid learning-based approach to overcome these limitations by customizing the CNN models to exploit brain tumor-specific patterns from the brain MRI dataset.CNNs have shown admirable performance for identifying tumors infected from normal individuals and segregation of tumor types using medical images.Moreover, deep feature boosting, ensemble learning, and ML classifiers help to improve performance considerably.
Experimental results suggest that the proposed deep learning-based approaches would assist radiologists in diagnosing tumors and other irregularities from medical imaginings.The key contributions of the work are listed as follows: 1.An automated two-phase deep hybrid learning-based detection and classification (DHL-DC) framework is proposed for brain tumor analysis using MR images.

A novel deep boosted features, and ensemble classifiers (DBF-EC) based scheme is
proposed to detect brain tumors.In this scheme, deep boosted feature space is accomplished using outperforming customized CNNs and provided to an ensemble of ML classifiers.

For the classification of brain tumors, a new deep hybrid features space-based brain tumor
classification approach is proposed to categorize tumor types.In the proposed technique, the dynamic features are obtained from the proposed BRAIN-RENet and concatenated with a histogram of gradients(HOG) features to increase the feature space diversity and to improve the learning capacity of ML classifiers.Moreover, the proposed BRAIN-RENet carefully learns various tumor's heteromorphic and inconsistent behavior.
The paper is arranged like this, in section; 2 related work is discussed, Section; 3 articulates the proposed methodology.Section; 4 illustrates the experimental arrangements, and Section; 5 is dedicated to results and discussion, and conclusion is in Section; 6.

Related work
In medical image analysis, many research horizons are explored.These include various areas of medical imaging, such as detection, classification, and segmentation [4][5][6][7][8].As cohorts build for brain tumor classification, there is a gap for novel approaches related to feature extraction using limited and class-unbalanced MR images datasets of brain tumors and tumors from other parts of the human body [9,10].Binary classifications are primitively explored in the literature to detect benign and malignant instances of the tumor.Kharrat et al. [11] explored the features of support vector machine (SVM) and genetic algorithm (GA) for brain tumor classification into three classes, normal, benign, and malignant.The proposed approach is only for binary classification.It is limited because it necessitates fresh training whenever there is a change in the image database.
Abdolmaleki et al. [12] constructed a shallow neural network using thirteen distinct features to distinguish benign and malignant tumors.These features were chosen based on the visual understanding of radiologists.The classification accuracy obtained by their proposed method was 91% and 94% for the malignant and descriptive tumors, respectively.Papageorgiou et al. [13] used fuzzy cognitive maps (FCM) to bifurcate the low and high-grade gliomas.Their work achieved an accuracy of 93.22% for high-grade and 90.26% for low-grade brain tumors.Zacharaki et al. [14] proposed the feature selection scheme and then applied the conventional machine learning.They extracted the features like the shape of the tumor, tumor intensity, and invariant texture for this purpose.Feature selection and tumor classification are carried out using SVM.Their work achieved the highest accuracy of 88% for low-grade and high-grade gliomas classification.Many researchers used the challenging benchmark dataset [15] consisting of MRI brain tumor scans with meningioma, gliomas, and pituitary-tumors.Cheng et al. [16] proposed multi-phase brain tumor classification comprises image dilation used as ROI and augmentation of tumor region in ring form.They evaluated their proposed model using three different features and achieved 82.31% accuracy.In general, they improved their performance by using bag of the word (BOW) features, but the overall complexity of the model was increased.Sultan et al. [17] proposed a deep CNNbased brain tumor classification model and employed extensive data augmentation.They attained 96.13% accuracy for multi-class categorization.Ahmet and Muhammad [18] Employed various dep CNN models for brain tumor analysis and achieved an accuracy of 97.2% with modified ResNet50 architecture.Khwaldeh et al. [19] employed multiple CNNs for brain MRI image classification and attained satisfactory accuracy.They attain higher accuracy of 97.2% by using reformed pre-trained Alexnet CNN.
In general, previously reported work lakes to address the following points: 1. Most of the previously done works have been evaluated using accuracy on the validation dataset.However, precision, recall, and MCC are assessed for better performance evaluation on unbalanced datasets.Evaluation of such performance metrics is essential to measure the model's generalization on the test dataset.
2. Previous work is largely restricted to either detection or classification of brain tumors.However, only the detection of tumors puts radiologists in an ambiguous situation due to insufficient details of the tumor type.
3. Largely, normal individuals and tumors are classified in a single phase; this increased the overall complexity of models.Hence, isolating normal instances from tumor images for the classification phase may decrease the model's complexity.
In this manner, the proposed two-phase brain tumor identification and classification framework can improve the diagnostics model's performance using standard performance assessment metrics like Accuracy, Precision, Recall, F-score, MCC, and AUC-PR.

Anticipated Methodology
The detailed architecture of the proposed DHL-DC framework is explained in this part.The

Preprocessing and Data Augmentation
In DL-based models, a deficient amount of data tends to model overt fit.Image augmentation is employed for efficient training and improved generalization ability of models.Data augmentation techniques help to improve the performance of DL models [20,21].This work employs four augmentation methods, shown in Table 1.Our dataset has MR images of different widths and heights, but it is recommended to resize them in the same height and width for optimum performance attaining.In this proposed work, we resize the greyscale MR images into either 299 × 299 or 224 × 224 pixels.The Block diagram of the DBFS-EC approach is shown in Figure 2.
TL is employed to optimize well-established TL-B CNNs.We replace the input of the TL-B CNNs The convolutional operation is used in (Eq.1).Input feature-map have size m * n is illustrated by w, the filter of size m * n is denoted by k.The output feature map is illustrated by , m and n start from 1 to (M − r + 1) and (N − s + 1), correspondingly.As illustrated in (1) , (Eq. 2) and (Eq.3) determine the z-avg and z-max processes and are denoted by  Avg and  Max , respectively.In (Eq.2), and (Eq.3), W denotes the average-and max-window size.In (Eq.4), the HOG-descriptor emphasizes the shape or the structure of an object in images.[37].ML classifier, SVM is used to minimize the structural risk factors and, hence improves generalization with the help of increased inter-class margins [38].

Experimental Setup Dataset
We collect a brain tumor data set of normal and tumor images; normal images are collected from the open-source Kaggle website [39] and named as dataset1(DS-1).Furthermore, tumor images are taken from a publicly available CE-MRI figshare [15], titled dataset2(DS-2).We collected 5058 images containing 1994 healthy patients and 3064 tumor images; thus, the acquired dataset is imbalanced and called dataset3(DS-3).In the first phase, detection is performed on DS-

Performance Evaluation of Tumor Screening Stage
In the proposed framework, in the initial analysis, for categorizing all samples into the tumor or healthy brain image, a DL-based DBFS-EC approach is proposed.Optimization of this stage results in minimum numbers of false positives for identifying tumors.The detection rate is enhanced by using three improvements in the detection phase.In the first step, we evaluate customized TR-SC and TL-B -based CNN models and examined that TL-B models perform better than TR-SC models ((0.75_8.4%)improvement in accuracy (Table 3)).The performance of TL-B CNN models is improved by replacing the softmax layer with three ML classifiers after features extraction from fully connected layers of TL-B CNNs (0.24_0.65% improvement in accuracy (Table 5)).At last, our proposed novel DBFS-EC based detection approach further improves the performance (( (1.39_9.05% ) accuracy (Table 3& Table 6)) of the brain tumor detection compared to existing customized CNN models.

Distinction Competency of The Brain Tumor Detection Approach
Experimentation results for the proposed brain tumor detection scheme are obtained by using combined data set (DS-3).In the first experiment, we compare several customized TL-B and TR-SC deep CNN models using an end-to-end way to obtain the tumor-related features.Performance assessment advocates that TL-B fine-tuned models learn the tumor-related feature better than the deep CNN models, trained from scratch on brain MR images.It is because of pre-trained weights learned from extensive data set named ImageNet [46].The performance assessment based on Accuracy, sensitivity, precision, and AUC-ROC, MCC, F1-Score is shown in Table 3.
While for the second experiment, we employ a hybrid learning-based approach by extracting the dominant features and exploiting the learning capability of deep CNNs with the strong discrimination power of ML classifiers.For this, we extract deep features spaces from the end layers of four best performing TL-B deep CNNs (InceptionV3, ResNet18, GoogleNet, DenseNet201), and fed them into competitive ML classifiers (SVM, MLP, and AdaBoostM1).The deep features of DenseNet201 with all three ML classifiers perform better than extracted deep features of other pre-trained CNNs.However, the performance of the deep features extracted using inceptionV3 fell shorter than features extracted from other pre-trained CNN networks on DS-3.
The performance assessment is based on accuracy, recall, precision, F-score, and MCC (Table 4 and Table 5).
In the last experiment for the proposed DBFS-EC approach, the effectiveness of the deep boosted ensemble learning is evaluated.Hybrid feature space is formed by concatenating all four selected feature spaces and ensemble classifier using all three classifiers employed in experiment two.
Utilizing ensemble deep feature spaces from more than one TL-B deep CNNs is effective for ML classifiers.An ensemble of classifiers enhanced the overall performance of the proposed DBFS-EC approach for brain tumor detection.

ROC Curve Based Performance Exploration
Analyzing the ROC curve is crucial for achieving the optimum analytic threshold for a classifier.
It pictorially shows the differentiation capacity of the classifier at possible threshold values.As shown in Figure 6, our proposed DBFS-EC scheme for the brain MRI dataset has improved performance (AUC_ROC: 0.999).ROC-based statistical analysis also highlights that the proposed approach attained high sensitivity.

Performance Analysis of Brain Tumor Classification Stage
Tumor classification is essential for designing effective treatment and diagnosis processes.The performance of the proposed HFF -HL model using deep and static features individually with SVM (Poly.Order 3) is evaluated and demonstrated in Table 8 and Figure 7. Results show that the proposed FSF-HL using SVM (with poly.order 3) performs better than deep and static feature spaces separately and previously reported work.The proposed method achieves performance metrics of recall (0.9906), precision (0.9913), accuracy (99.2%), and F1-Score (0.9909).This improvement in the performance of the proposed DBFS-EC is attained by two employing two techniques.First, we concatenate the dynamic and static feature spaces and then use SVM as a classifier.The concentration of features enhances the feature space diversity, the inter-class association is maximized, and SVM helps in structural risk minimization.In addition, systematic usage of max-and avg-pooling in the proposed BRAIN-RENet enabled the model in identifying fine-grained details and features high discrimination in raw MRI images.for brain tumor classification on a benchmark dataset.The two-phase framework is expected to assist clinicians in decision-making in clinical practice and will be helpful for radiologists in brain tumors diagnosis.In the future, we will appraise our proposed framework's performance and a more optimized one on other medical image datasets.
proposed framework includes two phases.A deep boosted features and ensemble classifiers (DBF-EC) method for brain tumor detection is proposed in the first phase.In the second phase, a hybrid features fusion-based brain tumor classification (HFF-BTC) model is proposed to classify tumor brain MR images detected from the first phase into three different classes: meningioma, glioma, and pituitary.MR images are categorized into different types by employing the proposed novel BRAIN-RENet based technique.Feature space diversity is attained by composing a fusion feature space comprised of dynamic and static features.The proposed multi-stage brain tumor detection and classification framework is shown in Figure 1.

Figure 1 .
Figure 1.Overall description of the proposed two-phase brain tumor analysis framework scheme.It includes; (1) Softmax-based implementation of existing customized transfer learningbased (TL-B) and training from scratch (TR-SC) deep CNN models in an end-to-end mode to differentiate tumor and normal brain MRI images, (2).Secondly, a deep feature space-based hybrid learning (DFS-HL) technique is designed.In DFS-HL, feature spaces of the four best TL-B finetuned models are fed into three ML classifiers to enhance the discrimination ability and generalization of the proposed technique (3).In the third, we proposed the DBFS-EC based framework to exploit the benefits of ensemble deep feature spaces and ensemble of ML classifiers.
with a new one, which is the same as the size of the MRI images.The dimensions of the last fullyconnected layer of all deep CNNs are set the same as the number of the classes, i.e., two.The softmax layer is employed to get the class-specific probabilities, and weights optimization is attained using a backpropagation algorithm through minimizing the cross entropy-based loss function.Proposed Deep Feature Spaces Based Hybrid Learning Our proposed DFS-HL scheme selects four well-performing TL-B CNN models as feature extractors and feeds them individually into three competitive ML classifiers.MRI dataset used in this study is not enough for the training of the deep CNN model, and there are chances of overfitting.So we have incorporated TL by using the pre-trained weight of the CNN models for the ImageNet dataset.TL-B deep CNN models learn the most discriminative features efficiently.We employ three different ML classifiers: SVM[30], MLP[31], and AdaBoostM1[32].DFS-HL deep CNNs minimize the empirical risk and reduce training error during optimal hyper-parameter selection[33].In addition, ML classifiers aim to minimize the test error on the unseen data with a fixed distribution for the training set by exploiting the structural risk minimization principle and improving generalization.Proposed Deep Boosted Feature Space and Ensemble Classifier:Ensemble learning aspires to performance improvement and encourages combining multiple feature vectors of various models into one rich information feature vector, and avoids the risk of using a feature vector extracted from a single model with unsatisfactory performance[34].It can be applied into two schemes, i.e., feature ensemble and classifier ensemble, dependent on the fusion level.Features ensemble implicates concatenating feature sets provided to the ML classifier for the final result.In contrast, ensemble classifiers are based on voting strategy by integrating decisions from multiple classifiers.In the proposed DBFS-EC scheme, we use both features and classifiers ensemble techniques.We concatenate feature vectors of four best performing TL-B CNN models to compose deep boosted feature space (DBFS) and an ensemble classifier (EC) by integrating all three classifiers for final decision.Ensemble learning enhances the feature space diversity and discrimination power of the proposed DBFS-EC framework.

Figure 2 .Figure 3 .
Figure 2. Part A. and Part B. illustrate the brief details of the detection framework, while Section (C), explains the comprehensive details of the proposed deep learning-based Brain Tumor Detection methodology

Figure 4 .
Figure 4.The proposed BRAIN-RENet deep CNN for brain tumor classification The proposed BRAIN-RENet contains six convolutional blocks.Every block is comprised of one convolutional layer, ReLU, and batch-normalization.The convolution layer extracts the tumorspecific features while ReLU act as an activation function.At the end of each block, average-and max-pooling operation is applied to learn region uniformity and boundary features of brain tumors as illustrated in Eqs.(2 & 3).Region and boundary-based operators are implemented systematically to capture patterns of different types of brain tumors.The detailed architecture of the proposed BRAIN-RENet is illustrated in Figure 4.
output of the dense layer is defined by Q, which employ global operation on  c ; the output of the feature extraction phase.Neurons of FC layers are shown by  d .The FC layer at the ending obtains important features for the classification (Eq.4).The CNN also contains a dropout layer to reduce overfitting.In brain MR images, various patterns show the intensity value variations in different image partitions.The smoothness of the region, changing texture, and edges form the pattern's structure.MR images show the patterns of brain tumors, distinguished into different types of complexities and severity levels.In the proposed BRAIN-RENet, the systematic employment of edge-and region-based operations (Eq.2), and (Eq.3), with a combination of convolutional operation(Eq.1), facilitates the model for enhancing patterns-specific properties for brain tumors classification[35].The advantages of the systematic employment of edge-and region-based operations in proposed BRAIN-RENet are as follows: The proposed BRAIN-RENet improves imitating the image's sharpness and smoothing dynamically, also can fine-tune the magnitude of smoothening and sharpening according to the spatial content of the image without human intervention.Systematically employment of edge-and region-based operations after each convolutional block enhances the region homogeneity of different image segments.The region operator smooths variations by applying average-pooling and suppresses the noise added during the MRI acquisition process.In contrast, the edge operator inspires CNNs for learning highly discriminative features using a max-pooling operation.Hybrid Features Fusion-based Brain Tumor Classification (HFF-BTC)In our proposed hybrid features fusion-based brain tumor classification (HFF-BTC) model, we compose a hybrid feature space comprising of static and dynamic features.Static features are extracted using the HOG feature descriptor.Dynamic features are extracted using the proposed BRAIN-RENet from the second last layer.Features fusion with hybrid learning exploited the advantages of empirical and structural risk minimization to enhance the performance of the brain tumor classification stage[36].Deep CNNs contain strong learning ability and focus on reducing the empirical risk factor to minimize the training loss and to avoid overfitting[33].The HOG feature descriptor counts the illustration of gradient orientation in local segments of an image.
3, which contains (5058) MR images, 3064 are tumor images, and 1994 normal brain MR images.The classification stage categorizes tumor instances (3064) brain tumor MR images into different family classes, i.e., glioma, meningioma, and pituitary using DS-2.Sample images of brain normal and tumor images are shown in Figure 5.

Figure 5 .
Figure 5. Sample image from data set of normal and tumor images (A) Normal,(B) Glioma,(C)Meningioma,

Table 2 . 9 )
Assessment metrics Details Metric Description Precision (Pre.)The fraction of correctly detected class to an actual class Recall (Rec) The proportion of correctly identified class and actual negative class Accuracy (Acc.) % of the total number of correct detection MCC Matthews correlation coefficient F1-Score The harmonic mean of Pre and Rec.TN) − (FP × FN) √(TP + FP) * (FP + FN) * (TN + FP) * (TN + FN) (Results and discussion A two-phase DL-based framework is designed for brain tumor analysis in this proposed work.In the first phase, the detection of brain tumor individuals from normal instances is performed.In the second phase, the classification of tumor images into further family classes is accomplished.Only tumor detection is not completely beneficial for the successful cure process, hence, it is essential to classify tumors further into relevant classes for effective and efficient treatment.The empirical effectiveness of the proposed framework is evaluated by performing two experiments.In the first experiment, the brain tumor detection task is performed by assessing the performance of DL and DB-HL-based models.In the second phase, we evaluated the advantages of feature spaces fusion by combining dynamic-static feature spaces to discriminate patterns of different brain tumors.The suggested brain tumor analysis framework is validated on unseen data using Accuracy, Sensitivity, Precision, AUC-ROC, MCC, F1-Score.The experimental results of dual stages are deliberated below.
deep features from top-4 pre-trained CNN models and ML classifiers individually.This is because ensemble learning employs feature spaces from four best performing TL-B CNNs and concatenates them.Integrating these deep features results in a hybrid feature vector that increases the feature space diversity, and an ensemble of ML classifiers enhances the discrimination ability of the ML classifiers.
Thus, brain MR images recognized as the tumor in the detection stage using the proposed DBFS-EC are allocated to the brain tumor classification model for tumor categorization.In the classification phase, we have proposed a hybrid features fusion-based approach for tumor MR images into particular classes, namely, meningioma, glioma, and pituitary.Dynamic and static features are concatenated to enhance the feature space diversity, and classification ability enhancement of the model is achieved by employing an ML classifier.Deep features are extracted using the proposed novel BRAIN-RENet, and static features are extracted using the HOG descriptor.

Figure 6 .
Figure 6.ROC curve for the existing CNNs, TR-SC based in (A), TL-B models in(B), and (C) represent proposed (DBFS-EC) framework.Differentiation Proficiency of The Brain Tumor Classification Stage The proposed model's performance is assessed with the proposed BRAIN-RENet and several MLbased models.Table 7 shows the performance comparison of the proposed hybrid learning-based model containing fusion feature spaces with SVM, other HFF models, and the proposed BRAIN-RENet.Performance of the proposed HFF-BTC model is evaluated for standard metrics, recall (0.9913), precision (0.9906), accuracy (99.20%), and F1-Score (0.9909).This framework outperformed the existing techniques in recognizing the tumor in MRI images.

Figure 7 .
Figure 7. Performance comparison of HFF-BTC models in terms of F1-score, Accuracy, Recall, and Precision

Table 6
shows that DBFS-EC, an ensemble of deep feature spaces from top-4 TL-B CNN models and the ensemble of ML classifiers, achieves higher performance measures than the ensemble of

Table 6
suggests that our proposed approach outperforms compared to other models in terms of recall (0.9913), precision (0.9906), accuracy (99.2%), and F1-Score (0.9909).HFF and SVM with ploy order 3 learn and discriminate the tumor-specific patterns from MR images better than other classifiers.