Ensemble of 2D Residual Neural Networks Integrated with Atrous Spatial Pyramid Pooling Module for Myocardium Segmentation of Left Ventricle Cardiac MRI

Abstract

Cardiac disease diagnosis and identification is problematic mostly by inaccurate segmentation of the cardiac left ventricle (LV). Besides, LV segmentation is challenging since it involves complex and variable cardiac structures in terms of components and the intricacy of time-based crescendos. In addition, full segmentation and quantification of the LV myocardium border is even more challenging because of different shapes and sizes of the myocardium border zone. The foremost purpose of this research is to design a precise automatic segmentation technique employing deep learning models for the myocardium border using cardiac magnetic resonance imaging (MRI). The ASPP module (Atrous Spatial Pyramid Pooling) was integrated with a proposed 2D-residual neural network for segmentation of the myocardium border using a cardiac MRI dataset. Further, the ensemble technique based on a majority voting ensemble method was used to blend the results of recent deep learning models on different set of hyperparameters. The proposed model produced an 85.43% dice score on validation samples and 98.23% on training samples and provided excellent performance compared to recent deep learning models. The myocardium border was successfully segmented across diverse subject slices with different shapes, sizes and contrast using the proposed deep learning ensemble models. The proposed model can be employed for automatic detection and segmentation of the myocardium border for precise quantification of reflow, myocardial infarction, myocarditis, and h cardiomyopathy (HCM) for clinical applications.

1. Introduction

Cardiac diseases have profound effects on health and mortality. Prediction of the cardiac index from cardiac MR (magnetic resonance) images is essential to diagnose and identify cardiac disease. In particular, an accurate quantification and identification of cardiac disease from left ventricle (LV) cardiac imaging is an imperative and demanding task [1]. During clinical practice, LV segmentation algorithms [2] generate myocardium borders either automatically or by measuring myocardium contouring borders manually. This process needs reliable and accurate quantification of the myocardium. Besides, manual contouring of the myocardium border is a time constraint, subjective to high observer inconsistency, and typically insufficient for ED (end-diastolic) and ES (end-systolic) frames. These factors make the process inadequate for dynamic functional analysis. Due to shape variability and the lack of edge information, LV segmentation requires research that involves advanced techniques.

Various segmentation methods for cardia MR images [2,3] require strong user interactions and a priori information to achieve better and reliable results for effective clinical applications. Recently, direct methods without segmentation, based on machine learning (ML) techniques, have gained popularity and have exhibited reliable performance. However, a few limitations have been reported: (1) features obtained manually could not capture sufficient information from task-relevant cardiac structures; (2) some manual feature selection techniques and ML algorithms may not be optimally integrated; (3) only measuring the volume of cardiac features is insufficient for regional, inclusive universal and vibrant function valuation.

Recently, convolutional neural networks (CNN) have been used widely for cardiac image analysis. For example, CNNs have achieved success in medical oriented imaging analysis [4]. In addition, CNNs in combination with other deep learning models have been positively employed for cardiac image analysis, with deep learning models incorporated with deformable models to develop and evaluate a fully automatic LV segmentation tool from short-axis cardiac MRI datasets [5]. Further, CNNs have been used to automatically detect the LV chamber in MRI datasets and stacked autoencoders to determine the LV shape. The inferred shape is combined into deformable models to enhance the accuracy of the segmentation. In [6], the authors proposed a unique method based on deep learning and a level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance (MR) data. This combined method used small training sets and delivered accurate segmentation results. In another study [7], Tran presented a method to solve the problem of automated left and right ventricle segmentation using deep fully convolutional neural network architecture. He trained his proposed model end-to-end in a single learning stage from whole-image inputs and ground truths to analyze every pixel. Long et al. [8] applied AlexNet, VGG net, and GoogLeNet into CNNs and transferred their learned representations using fine-tuning to the segmentation task. They presented a model that integrates semantic information from a deep, coarse layer with image information from a shallow, fine layer to generate precise and elaborate segmentations. Wolterink et al. [9] proposed a fully automatic method for segmentation and disease classification using cardiac cine MR images. They used CNN segmentations of the left ventricle (LV), right ventricle (RV), myocardium in end-diastole (ED) and end-systole (ES) images. The extracted features from segmentations were used in Random Forest algorithms to classify different heart diseases. In [10], the authors presented a method based on multi-planar deep CNN with an adaptive fusion scheme in which they automatically used complementary information from different planes of the 3-D scans for improved delineations. They used CT and MRI images to train and test their model for cardiac substructures. In [11], the authors proposed a fully automatic MRI cardiac segmentation method based on CNN. This model used features of both levels, such as high-level features and low-level features. These features were learned with a grid-like CNN architecture. Further, they tested their model on the ACDC MICCAI’17 challenge dataset. This method can segment all three regions of a 3-D MRI cardiac image. Various 2-D and 3-D Unet-based deep learning models have been proposed in automatic cardiac diagnosis challenge (ACDC) for cardiac segmentation, including dilated CNN, encoder-decoder architecture. Moreover, Tan et al. [12] introduced a multiscale deep learning model for LV oriented segmentation in the polar typed spaced domain. Brahim et al. [13] presented 3-D CNN for segmenting volumetric oriented images. These models acquired sufficient success to handle LV segmentation based on MR cardiac images.

In this research, a deep learning model with a residual block and integrated ASPP module is proposed to extract multiscale features from the decoder side and to use these features in the encoder side of the proposed model for myocardium border segmentation based on MRI cardiac images. The main objectives and salient features of this paper are listed below:

• The research work proposes integration of the 2-D residual neural network with the ASPP module. The designed network has different numbers of layers at the encoder and the decoder components. In addition, the ASPP module is added at the lowermost of the encoder and the decoder. The integrated framework captures multiscale information, and the ASPP module can be detect small objects with different shapes, sizes and orientations from MR heart images.
• The proposed model is designed to be robust because its training involves various hyperparameters. In addition, different models are brought together using a majority voting scheme to further enhance segmentation accuracy.
• A contrast enhancement method is proposed for preprocessing the input cardiac MRI dataset. Various performance metrics are used to relate the performance of recent models and the proposed model in deep learning contexts.

The rest of the paper is organized as follows. Section 2 describes related work. Section 3 describes materials and methods for the dataset, preprocessing, proposed model, and evaluation criteria. The experimental results are discussed in Section 4. Section 5 concludes the work and provides direction for future work.

2. Related Work

LV quantification methods for cardiac MR images are grouped into three classifications: (1) physical quantification; (2) segmentation oriented quantification, and (3) straight regression-oriented quantification. Conventionally, experimental practice involves physically contouring the boundaries of myocardium and is considered reliable [14]. However, the manual contouring approach consumes more time than automatic methods and the results vary because of observer bias. In addition, the method typically limits the ED (end-diastolic) and ES (end-systolic) frames, which makes it inadequate for vibrant functional investigation during the complete cardia cycle. Segmentation-based quantification has been used to segment the cardiac myocardium towards multifaceted environments, and with visibly delineated corners of the epicardium and endocardium. Moreover, various methods [2,15], such as image driven methods, and training-based and deformable models, have been designed for cardiac LV classification. To obtain precise performance, these methods need user interaction and a priori information. Some methods are based on anatomical assumptions, such as circular geometry of the LV [16], intensity histogram distribution [17], or statistical shape modeling [18]. User interactions involves pointing out the center of the LV cavity and manually identifying the ventricle corner of the first frame [19]. This can create inaccuracies and may prevent LV separation methods being used for effective scientific applications. In [20], depth-wise dense network is proposed to detect infected area in lungs X-rays that improved feature representations by performing multilevel feature embedding.

Direct regression approaches have used recent ML approaches to estimate cardiac volumes. These methods can further be categorized into two groups: two phase classes only [21,22,23], and end-to-end deep learning models [24,25,26]. The two-phase method uses unsupervised ML such as the Bhattacharyya coefficient in image disseminations [27], appearance features [28], multiple low level image features [22], features from a multiscale convolution deep belief network (MCDBN) [23], and manually obtained features. Furthermore, these features are employed in regression models for cardiac volume estimation. In another study [29], a hybrid model based on 3D residual network (RN) with a squeeze-and-excitation (SE) block is proposed for volumetric segmentation of kidney, liver, and their associated tumors that improved performance in volumetric biomedical segmentation. End-to-end deep learning models have been effectively utilized in medical image investigation [30] and have the capability of extracting effective features in a complete fashion [31]. Deep learning models such as the deep belief network (DBN), auto stacked encoder and CNN, incorporated with traditional models [5,6,32] are used in cardiac image segmentation. The fully convolutional network (FCN) and recurrent FCN have been proposed in cardiac segmentation [6,7,33]. The author of [24] proposed 3-D deep learning models to segment and estimate volume. The end-to-end deep learning models could not obtain temporal information between each slice of cardiac MR images and were incapable of handling multiscale information or features from input cardiac images. Therefore, these end-to-end (E2E) deep learning models must incorporate multiscale features from input images to produced reliable performance. The designed method is capable of segmenting myocardium borders and extracting multiscale features using an ASPP module for all segments in the complete cardiac cycle.

Zheng et al. [34] proposed a model to address the overfitting problem in deep CNN based on two-stage training methods such as: 1—pretraining and 2—implicit regularization training. In the first method, image representation is extracted through training a model for anomaly detection. In the second method, the model is retrained based on anomaly detection results to standardize the feature boundary and converge it in a suitable position.

In another study, Zheng et al. [35] proposed a full stage data augmentation framework to enhance the accuracy of deep convolutional neural networks. This model acts as an implicit ensemble model without additional training costs. Coexisting data augmentation during the training and testing process can provide network optimization and improve its generalization capability.

Liu et al. [36] proposed a method to extract hierarchical neighborhood preserving features based on a stacked neighborhood preserving autoencoder (S-NPAE). A loss function feature was applied to reconstruct and preserve the neighborhood structure of the input data at the same time. NPAE extracted the features from its input data by minimizing the loss function. The deep S-NPAE network was developed hierarchically by stacking multiple NPAEs. The extracted features were provided to S-NPAE for prediction in soft sensor modeling.

Rucco et al. [37] proposed a model for radiomics features based on topological features for a personalized diagnostic system of Glioblastoma multiforme (GBM) analysis from fluid attenuated inversion recovery (FLAIR). They developed a method based on topological and textural features, and automatic interpretable machine learning for automatic GBM classification on FLAIR.

In [38], a semantic segmentation network was introduced to develop an indoor navigation system for a mobile robot based on a convolutional neural network (CNN).

To detect and remove outliers in high dimensional data, a multistage technique was proposed in [39]. This technique reduced the high dimensional features into two dimensional features based on t-distributed stochastic neighbour embedding (t-SNE). Further, a convolutional neural network model (ConvNet) was used for the image classification problem.

In [40], Hu et al. developed a TopoResNet that integrates topological information into the residual neural network architecture. They applied TopoResNet to a skin lesion classification problem. They determined that TopoResNet improves the accuracy and the stability of the training process.

Ensemble learning unites several specific models to obtain adequate generalization performance. Nowadays, deep learning models are demonstrating better performance than traditional classification models. Deep ensemble learning models combine the advantages of deep learning and ensemble learning to improve the generalization performance of the model. In this regard, several researchers have used ensemble learning in their studies [41,42,43,44].

3. Material and Methods

3.1. Dataset

A dataset from 56 subjects with SAX MR sequences based on a clinical environment was used for model training and validation. Each subject consisted of 20 frames and the pixel layout of the MR images was assorted between 0.6836 mm/pixel and 2.0833 mm/pixel, in addition to a mode with 1.5625 mm/pixel. The study obtained the dataset from three medical resources associated with health care hubs, namely, London and St. Joseph Healthcare Centers based on scanners of two merchants (GE & Siemens) [45,46]. The dataset is publicly available [45] with subjects having ages from 16 years to 97 years and averaging 58.9 years. The ground truth for each frame for all subjects was provided for endocardium and epicardium borders. The individual subject data comprised 20 frames all over the cardiac cycle.

3.2. Preprocessing of the Dataset

A dynamic histogram equalization technique was used for contrast enhancement of input cardiac MRI images. The raw and contrast images are shown in Figure 1 for different numbers of patients. The input slices with enhanced slices and ground truth mask are shown in Figure 1.

3.3. Proposed Model

The proposed model is a residual network comprising an encode-decode network. The ASPP module is integrated at the bottom of the proposed model. Figure 2 depicts the block diagram of the proposed model for myocardium segmentation using MR cardiac images. The input feature maps from the encoder side are passed to the ASPP module, and output feature maps after the ASPP module are provided to the decoder part of the proposed model. Details of each module are explained in the following sections.

3.3.1. ASPP Module

The ASPP network is a combination of various parallel atrous convolutions with different rates, global average pooling and a 1 × 1 convolutional layer. The proposed network can capture contextual information at multiple scales using atrous convolution with different filter kernels. In addition, it can extract multiscale features from the input feature maps for myocardium border segmentation. Atrous convolution is a promising tool for capturing multiscale feature information by controlling different resolutions by adjusting different receptive fields [47]. Atrous convolution was applied on the input x for each pixel i on the output y and filter w as shown in Equation (1):

$$y\left[i\right]={\displaystyle \sum }_{k}x\left[i+r.k\right]w\left[k\right]$$

(1)

Here, the atrous rate r controls the stride for sampling the input image. The subjected convolution inserts r − 1 zeros between two successive filter values and convolves the input x with those filters. The receptive field of filter can be modified by adjusting the filter rate r. We integrated an ASPP module was used in DeepLabv3 [48] in the proposed residual network to improve the segmentation results.

ASPP consisted of four atrous convolutions layers using different atrous rate r and one global average pooling layer. Three parallel atrous convolution layers were used with 3 × 3 size rates of 3, 5 and 7, and one 1 × 1 convolution layer. The resulting features from the four convolutions and one overall pooling average layer were up sampled bilinearly to the feed size and then concatenated and provided to another 1 × 1 type convolution. The ASPP was employed to the feature map produced by the bottom of the encoder segment and the outcome feature map from ASPP were provided to the decoder part of the network.

3.3.2. Proposed Hybrid Encoder-Decoder ASPP-RN Model

The pooling layers in convolutional neural network gathered contextual information and reduced the spatial information from input image. The main problem in semantic subdivision is that the feature maps have low resolution. AN encoder-decoder based networks such as U-net, V-net can be employed to solve the low-resolution problem based on the up-sampling layer, gathering the information from lower layers and reinstating the resolution of the estimate to that of the input image at the decoder side. The encoder-decoder based networks concatenate maps from the decoder side with a lower-level feature map to provide the spatial information in semantic segmentation using the up-sampling layer.

The proposed model is structured into a series of encoding and decoding operations constituting a residual network. The convolution task is performed by an encoder that carries a 2 × 2 residual block incorporating the maximum pooling operation that down-samples the data. The channels relating to feature maps are doubled once the encoder performs its residual activity in the network The ASPP network combines both the top and bottom of the encoder-decoder operations in the model. The ASPP network is provided with the features of input images that map the inputs to the up-sample component of the network. The residual part of the network typically consists of layers named the convolution layer, batch-normalization layer, and the ReLu layer containing an activation function. The process passes the inputs to the third later after convolving it by the convolutional layer. The products of the first two layers are combined and provided to the ReLu instigation function. The residual block is shown in Figure 3. A different set of channels is employed by the residual segment at various layer channels that perform encoding and decoding in the network.

The segments of the decoding part perform up-sampling employing a 2 × 2 deconvolution operation followed by a concatenation of features (inputs of encoding) fed into the residual block. The entire process divides the input data into halves with the consecutive operation of up-sampling and down-sampling. After a series of such operations, the process achieves a convolution 1 × 1 layer that is accompanied by a sigmoid activation method. The desired output in a binary format is thus predicted through the system.

3.3.3. Ensemble of Proposed Models

The ensemble technique based on majority voting was applied based on various machine and deep learning models for classification problem [49,50]. The majority voting technique was applied for creating an ensemble of the output of the proposed models based on predicted masks for myocardium segmentation. Figure 4 shows a block diagram of the ensemble method. Based on experimental results, seven best models with different hyperparameters space were chosen for this experiment. Majority voting based on maximum function scheme used for ensemble of the output of various models is shown in Figure 4. In majority voting, the predicted pixel value outputs of the best deep learning models are assigned votes and the most predicted values are assigned as the vote of the outputted model as an ensemble. The ensemble model output is expressed in Equation (2).

$$C\left(x\right)=max{{\displaystyle \sum }}^{ }{C}_i\left(x\right)=y$$

(2)

where y is the output pixel values of the ensemble model and C_i(x) is the prediction of each individual model. As a result, seven base segmentation models are selected to generate the output of ensemble model. The prediction of these best models is voted to obtain the final myocardium segmentation.

3.3.4. Network Parameters or Configuration

The proposed model was implemented using PyTorch with Adam optimizer. Different hyperparameters were used for training of the model. The learning degree, weight falloff, and batch size with number of epochs were also used in training the model.

3.4. Evaluation Criteria

The performance of the proposed model was evaluated based on evaluation metrics such as sensitivity, specificity, Jaccard coefficients, Dice coefficient, Volume Overlap Error (VOE), Relative Volume Difference (RVD), Surface Distance Metrics, and Hausdorff distance. These evaluation metrics are described below.

Sensitivity is used to measure the positive portion of the voxels between actual and predicted segmentation masks. Sensitivity is determined by Equation (3):

$$\mathrm{Sensitivity}=TPR=\frac{TP}{TP+FN}$$

(3)

Specificity is employed to measure the negative voxels between actual and predicted masks. Specificity is expressed in Equation (4):

$$\mathrm{Specificity}=TNR=\frac{TN}{TN+FP}$$

(4)

The Jaccard Coefficients index (JAC) [14] describes the overlap of two sets separated with union of sets as shown in Equation (5):

$$J\left(A,B\right)=\frac{\left|A{{\displaystyle \cap }}^{ }B\right|}{\left|A{{\displaystyle \cup }}^{ }B\right|}$$

(5)

where A is the actual volume, and B is the predicted volume.

The Dice coefficient (DC) is employed for validating the segmentation of the medical volume-oriented data. This is an overlapping index that calculates the magnitude of overlapping between actual result and achieved results in the context of task-related binary segmentation [40]. For the actual and achieved mask, the DC is expressed by following Equation (6):

$$\mathrm{Dice}\left(A,B\right)=\frac{2\left|A{{\displaystyle \cap }}^{ }B\right|}{\left|A{{\displaystyle \cup }}^{ }B\right|}$$

(6)

The volume overlap error [40] is almost the reciprocal of the Jaccard index. It is defined by Equation (7):

$$\mathrm{VOE}\left(A,B\right)=1-\frac{\left|A{{\displaystyle \cap }}^{ }B\right|}{\left|A{{\displaystyle \cup }}^{ }B\right|}$$

(7)

The relative volume difference (RVD) is calculated by Equation (8) [40]:

$$\mathrm{RVD}\left(A,B\right)=\frac{\left|B\right|-\left|A\right|}{\left|A\right|}$$

(8)

Surface distance metrics determine the measurements between actual and predicted segmentation [40] as defined in Equations (9) and (10):

$$d\left(v,S\left(A\right)\right)=mi{n}_{S_AϵS\left(A\right)}\Vert v-S_A\Vert$$

(9)

$$\mathrm{ASD}\left(A,B\right)=\frac{1}{\left|S\left(A\right)\right|+\left|S\left(B\right)\right|}\left({\displaystyle \sum }_{S_AϵS\left(A\right)}d\left(S_A,S\left(B\right)\right)+{\displaystyle \sum }_{S_BϵS\left(B\right)}d\left(S_B,S\left(A\right)\right)\right)$$

(10)

The Hausdorff Distance is employed for differentiating between the binary objects into two masks of segmentation. It is termed as the maximum surface distance (MSD) that exists in objects [51]. The MSD is expressed in Equation (11):

$$\mathrm{MSD}\left(A,B\right)=max\left\{ma{x}_{S_AϵS\left(A\right)}d\left(S_A,S\left(B\right)\right),ma{x}_{S_BϵS\left(B\right)}d\left(S_B,S\left(A\right)\right)\right\}$$

(11)

4. Simulation

The study employs eight metrics for the measurement of the accuracy of the proposed segmentation method by comparing results with those of recent models. Measurements are made of metrics such as relative volume difference (RVD), volumetric overlap error (VOE), maximum surface distance (MSD), average symmetric surface distance (ASD), Dice coefficients, Jaccard coefficients, specificity and sensitivity. It was noticed that better segmentation was achieved for smaller values of the first four measurements. Better segmentation was achieved for higher value of Dice and other three metrics (Jaccard, sensitivity, specificity).

4.1. Performance Analysis Based on Performance Metrics

Various recent models used for segmentation based on biomedical images were reimplemented and trained on various set of parameters (learning rate, optimizer, size of batch with number of epochs). The performance metrics for all existing and proposed segmentation models is shown in

Table 1. The Performance metrics based on prevailing and proposed models on test dataset.

Models	DC	JC	Hd95	HD	Specificity	Sensitivity	VOE	ASSD	RVD
SegNetM3	81.00	69.10	8.5227	13.6867	98.63	84.87	0.3089	2.5484	0.1201
FractalNetM4	81.95	71.71	9.0721	16.3544	98.72	89.72	0.2828	2.4974	0.3443
ProposedM4	85.43	75.93	6.1702	10.0142	99.14	85.51	0.2406	1.7823	0.0141
FractalNetM3	83.63	73.88	13.1938	20.2024	99.04	83.18	0.2611	2.9740	0.0169
ProposedM3	84.96	74.78	11.3915	24.4692	98.97	87.48	0.2521	2.3937	0.0795
FactalNetM1	81.91	71.38	9.2453	30.4901	98.60	85.92	0.2861	2.80443	0.1231
ProposedM1	82.51	71.80	11.4752	26.905	98.76	87.22	0.2819	2.7359	0.1746
Ensemble Model	84.99	75.17	5.6782	12.6781	97.99	90.78	0.2790	1.8932	0.0674

The last line in bold shows the best results.

. The models were trained using four different sets of hyperparameters (details of each hyperparameter set are depicted in

Table 2. Hyperparameters Employed in Proposed and Recent Deep Learning Models.

Hyperparameters	Proposed Model	SegNet [52]	ResNet [53]	Unet-Base [54]	Attention with Unet [55]	FractalNet [56]
Learning rate	3 × 10−4, 1 × 10−4, 2 × 10−4, 1 × 10−4	-	-	-	-	-
Optimizers	Adam	-	-	-	-	-
Batch size	8, 12, 16, 20	-	-	-	-	-
Number of epochs	100, 200, 300, 500	-	-	-	-	-

) and the best models chosen based on test samples.

Best training Dice coefficients and Jaccard coefficients based on proposed and existing models are shown in Figure 5.

Figure 6 shows the Dice and Jaccard coefficients for all models. The best four sets of hyperparameters were used for training the proposed and existing models.

The sets of hyperparameters used for recent existing deep learning models and proposed model are depicted in Table 2. The symbol ‘-’ represent the same value noted in the proposed model.

Percentage of equal pixels was employed to evaluate the binary segmentation between actual and predicted samples. The high value of percentage agreement between pixels of predicted and ground truth data shows that the proposed models produced optimal performance. A histogram of pixel agreement between predicted and ground truth test samples using the proposed model is shown in Figure 7. Most of the samples shows more than 96 percent agreement between predicted and ground truth values for all proposed models. Figure 7h shows that the ensemble model produced 99 percent histogram agreement for most of the samples. This pixel agreement analysis shows that the proposed model produced reliable and accurate segmentation results using cardiac MRI-validated dataset. Pixel agreement analysis revealed that our proposed model could be used for myocardium border segmentation.

Bland Altman analysis was conducted using ground truth and predicted segmentation values for all frames of the test data. Figure 8 shows the Bland Altman analysis for the chosen best models (proposed and existing). The analysis showed a 95% agreement between predicted and ground truth samples for all test data. Only a few samples produced error values, as shown in Figure 8a. Figure 8h shows densely grouped values for most of the samples indicating agreement between samples. FractalM1 and FractalM4 showed a high error rate for a certain number of samples and some samples were diverted from densely grouped samples. The Bland Altman analysis shows that the ensemble model provides a reliable solution and strong agreement between actual and predicted samples, except for a few samples. Statistical analysis was performed based on actual and predicted mask frames for one patient. The Pearson correlation coefficient was calculated for actual and predicted masks based on best proposed and existing models. The correlation values are shown in Figure 9. The ensemble produced highest correlation values between actual and predicted masks. Pixel agreement for 20 frames based on one patient’s data is shown in Figure 10. The best proposed and existing models were used to calculate pixel agreement percentage. The ensemble model produced the highest pixel agreement percentage between the actual and predicted masks.

4.2. Visualization Segmentation Results

The proposed model and existing deep learning models were used to evaluate segmentation of the myocardium border using heart MRI dataset. An 80% dataset was used for training and 20% for testing. The actual and segmented contours are shown in Figure 10.

Table 3. Comparison of proposed model with other publicly available datasets.

Datasets	DC	JC	Hd95	HD
Xue et al. [45]	84.99	75.17	5.6782	12.6781
Alain et al. [57]	79.67	72.09	8.33	10.33
Liu et al. [58]	78.33	70.01	8.34	14.22

shows comparison of the proposed ensemble model with datasets. Alain et al. [57] proposed a dataset based on left ventricle myocardium segmentation. The proposed model also evaluated a multi-sequence cardiac MRI dataset [58]. The dataset used in this paper with other publicly available datasets are shown in Table 3.

Training and validation loss curves for the proposed model are shown in Figure 11.

The training and validation loss curves are shown in Figure 11. The loss curves did not provide smooth learning when we used the Liu dataset [58] and resulted in overfitting; the reason being the small number of samples available for training. The dataset provided by Xue [45] produced better training and validation loss compared to the Liu dataset [58].

Table 4. Trainable parameters and number of flops for optimization of proposed model with different hyperparameters.

Models	Trainable Parameters	Number of Flops
Model1	27,889,221	334,516,681
Model2	27,414,339	333,277,610
Model3	27,228,939	337,723,411
Model4	27,907,333	337,612,235
Model5	27,221,797	332,236,130
Model6	27,698,773	332,112,244
Model7	27,896,532	338,333,759

shows the number of trainable parameters and number of flops used during optimization and training of proposed model with different number of hyperparameters. There was no big difference between trainable parameters across each model. The computational complexity was the same across each model when we used different number of parameters.

4.3. Discussion

The performance of the proposed framework was evaluated using various performance metrics for semantic segmentation. Performance metrics such as Dice coefficients, Jaccard coefficients, volume over error, Housdrouf distance, maximum average surface distance, accuracy, sensitivity, specificity were used to assess the accuracy between actual and predicted values based on each frame of the input dataset. Other performance metrics such as correlation coefficients, agreement between binary and Bland Altman plots between actual and predicted masks were assessed to determine the accuracy of the proposed model. Slice-by-slice popular segmentation deep learning models such as Unet, SegNet, FractelNet, ResNet, Attention U-net were reimplemented and results compared with our proposed Residual neural network including an atrous spatial pyramid pooling module deep learning model. Further, we trained state-of-the-art and proposed models using various hyperparameters and ensemble, or by fusing the four best chosen models using an ensemble majority voting method. The ensemble model produced the best results with the proposed models. The proposed model performed well for myocardium segmentation and handled the big challenges occurring in myocardium segmentation due to different sizes and shapes of the myocardium border. The texture between the myocardium borders and its surroundings is another challenge, and our proposed model handled this challenge well. Images of a few slices are shown in Figure 12, Figure 13, Figure 14 and Figure 15. Due to insufficient contrast between surroundings and the border of myocardium, some parts of the border could not be seen using fractalNet, as shown in Figure 15. The proposed model4 segmented the full myocardium border, as shown in Figure 13. The ensemble model produced excellent results and segmented the full border with different shapes and sizes of the myocardium border. The atrous spatial pyramid pooling module is used successfully for segmentation of computer vision and biomedical image semantic segmentation problems. The ASPP module integrated with the proposed encoder-decoder residual network has the ability to extract detailed features and achieved good performance compared with recent deep learning models.

Some boundary pixels of myocardium borders still require reconstruction using deep learning models. In the proposed ensemble solution we took maximum pixel values from the same location using different proposed models. Standard convolution usually uses a convolution kernel with a smaller receptive field, making it impossible to obtain contextual information. ASPP consisted of four atrous convolution layers using different atrous rates (r) and one global average pooling layer. The pooling layers in the convolutional neural network gather contextual information and reduce the spatial information from the input image. The main problem in semantic subdivision is that the feature maps have low resolution. Encoder-decoder-based networks such as U-net, and V-net can be employed to solve the low-resolution problem based on an up-sampling layer, gathering the information from lower layers and reinstating the resolution of the estimate to that of the input image at the decoder side.

The proposed model is structured into a series of encoding and decoding operations constituting a residual network. The majority voting technique was applied for ensemble of the output of the proposed models based on predicted masks for myocardium segmentation.

U-Net presented limited capacity in effectively learning the feature information of the images in complicated tasks such as myocardium border segmentation with the same texture surrounding the area and with different shapes. This limitation led to a need for optimizing the network architecture structure to enlarge the parameter space to allow the network to learn more representative features.

The proposed residual block with ASPP module has various advantages compared to traditional CNN-based models. The main advantages are (1) that the gradient can flow continuously, allowing the parameters to be updated in very deep networks, (2) the operations applied by a single layer are only a small modification to identity operation, and (3) ResNets modules are robust to layer permutations suggesting that neighboring layers perform similar operations. The main purpose of such residual blocks is the safeguarding of feature maps within convolutional layer blocks that are used before each encoder and decoder block required to bridge the semantic gap between the encoder and decoder by maintaining small increments in the computational overhead and providing an accurate segmentation map. The structural information for feature maps could be restored using ASPP with the addition of the residual blocks aimed to preserve the fine-grained structures that play an important role in medical image segmentation.

5. Conclusions

Quantification of the myocardium is an essential task for identification of myocardial infraction and other heart diseases. The main challenge is to segment the myocardium border that has different shapes and sizes of the myocardium border zone and the same texture as the surrounding tissues. Various deep learning models have been proposed to tackle the myocardium border segmentation task.

In this paper, a model based on a residual encoder-decoder integrated with an ASPP module was presented for myocardium segmentation using a cardiac MRI dataset. Structural information for feature maps could be restored using the proposed ASPP with the addition of residual blocks and different hyperparameters used in training of various models. We chose the best models based on the experimental results. Further the ensemble method based on a majority voting technique was applied to fuse the segmentation output of the best proposed and existing deep learning models. The ASPP module was used to extract relatively local features and multi-scale features to provide rich context information for the proposed network. Further, the ASPP and residual blocks from the encoder and decoder side of the proposed model were used to reduce the semantic gap between the high-level feature and the low-level feature maps to improve feature fusion capability and improve the performance of the myocardium border segmentation task. The results show that the proposed deep learning model could be used for cardiac myocardium segmentation. Accurate segmentation and detection of myocardium borders could be helpful in diagnosing myocardial infraction, no reflow and hypertrophic cardiomyopathy (HCM) disease.

In future work, different hyperparameters in deep learning models and postprocessing image processing steps could be used to reconstruct and segment the myocardium border accurately. The postprocessing steps will be helpful for further accurate detection of myocardium borders. The hybrid deep learning model approach could be developed for the myocardium border.