Next Article in Journal
Evaluation of Isolation Area, Myocardial Injury and Left Atrial Function Following High-Power Short-Duration Radiofrequency or Second-Generation Cryoballoon Ablation for Atrial Fibrillation
Next Article in Special Issue
Blood Pressure Prediction Using Ensemble Rules during Isometric Sustained Weight Test
Previous Article in Journal
Chronological Development of Cardiovascular Disease in Times of COVID-19: A Retrospective Analysis of Hospitalized Diseases of the Circulatory System and COVID-19 Patients of a German University Hospital
Previous Article in Special Issue
Pre-operative Machine Learning for Heart Transplant Patients Bridged with Temporary Mechanical Circulatory Support
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Attention-Based UNet Deep Learning Model for Plaque Segmentation in Carotid Ultrasound for Stroke Risk Stratification: An Artificial Intelligence Paradigm

School of Biomedical Engineering, Indian Institute of Technology (BHU), Varanasi 221005, India
Department of Electronics and Communication, Shree Mata Vaishno Devi University, Jammu 182301, India
Department of Radiology, Azienda Ospedaliero Universitaria (A.O.U.), 09100 Cagliari, Italy
Department of Cardiology, Indraprastha APOLLO Hospital, New Delhi 110076, India
Heart and Vascular Institute, Adventist Heath St. Helena, St. Helena, CA 94574, USA
Vascular Screening and Diagnostic Centre and University of Nicosia Medical School, Nicosia 2409, Cyprus
Department of Electrical and Computer Engineering, Idaho State University, Pocatello, ID 83209, USA
Stroke Diagnostic and Monitoring Division, AtheroPoint™, Roseville, CA 95661, USA
Authors to whom correspondence should be addressed.
J. Cardiovasc. Dev. Dis. 2022, 9(10), 326;
Submission received: 21 August 2022 / Revised: 6 September 2022 / Accepted: 14 September 2022 / Published: 27 September 2022


Stroke and cardiovascular diseases (CVD) significantly affect the world population. The early detection of such events may prevent the burden of death and costly surgery. Conventional methods are neither automated nor clinically accurate. Artificial Intelligence-based methods of automatically detecting and predicting the severity of CVD and stroke in their early stages are of prime importance. This study proposes an attention-channel-based UNet deep learning (DL) model that identifies the carotid plaques in the internal carotid artery (ICA) and common carotid artery (CCA) images. Our experiments consist of 970 ICA images from the UK, 379 CCA images from diabetic Japanese patients, and 300 CCA images from post-menopausal women from Hong Kong. We combined both CCA images to form an integrated database of 679 images. A rotation transformation technique was applied to 679 CCA images, doubling the database for the experiments. The cross-validation K5 (80% training: 20% testing) protocol was applied for accuracy determination. The results of the Attention-UNet model are benchmarked against UNet, UNet++, and UNet3P models. Visual plaque segmentation showed improvement in the Attention-UNet results compared to the other three models. The correlation coefficient (CC) value for Attention-UNet is 0.96, compared to 0.93, 0.96, and 0.92 for UNet, UNet++, and UNet3P models. Similarly, the AUC value for Attention-UNet is 0.97, compared to 0.964, 0.966, and 0.965 for other models. Conclusively, the Attention-UNet model is beneficial in segmenting very bright and fuzzy plaque images that are hard to diagnose using other methods. Further, we present a multi-ethnic, multi-center, racial bias-free study of stroke risk assessment.

1. Introduction

In recent decades, stroke and other cardiovascular diseases (CVD) have emerged as fatal diseases across the globe. For developing countries such as India, stroke has become a pandemic [1,2,3,4] in recent years with 105 to 152 cases per 0.1 million of the population [5]. In developed countries such as the USA, 795,000 people experience a stroke, and approximately 240,000 suffer a transient ischemic stroke yearly [6]. Various cardiovascular risk factors such as diabetes [7,8], smoking, hypertension, hypercholesterolemia, and irregular lifestyle have been discussed in many studies as leading to higher stroke and CVD events [6,9,10,11].
A major cause of stroke and CVD is atherosclerosis disease of the arteries [12,13], particularly in the intimal and media walls [14]. Different stages of plaque constitute soft plaque, such as lipid and macrophages, and hard plaque, such as calcium [15], or so-called plaque morphology, which is helpful in the diagnosis of atherosclerosis disease severity [16,17]. Three primary techniques available for carotid plaque identification are magnetic resonance imaging (MRI) [18], computed tomography (CT) [19], and ultrasound (US) imaging [20]. Based on its cost-effective, non-radiative, portable, and device-handling properties, ultrasound is more prevalent in carotid artery imaging [21,22]. Different grayscale levels that are generated by ultrasound echo segregate parts of the carotid artery, such as the lumen area, lumen intima layer, media-adventitia layer, and plaque, such as fibrin, fibrosis, and hard and soft plaque [23]. This also helps the sonographer or radiologist to mark the plaque area by delineating the lumen intima (LI) and media adventitia (MA) borders of the carotid arteries [24].
Delineating the carotid plaque using the manual technique is tedious and error-prone. Thus, researchers have proposed various automated methods of delineating the carotid plaques [25]. The computerized method for carotid plaque and intima-media thickness (IMT) delineation involves edge-based [26,27] methods including first order absolute moment (FOAM) [28] and local maxima [29,30]; anisotropic Gaussian derivative filtering [31,32] and non-maximum suppression [33]; snake-based [34,35,36] or dual-snake-based [36]; scale-space [37,38,39] (or AtheroEdge™ 1.0, 2.0, AtheroPoint LLC, CA, USA) [31]; shape-based [40,41]; and level-set [42,43,44,45,46,47,48]. A state-of-the-art automated segmentation system, AtheroEdge™ 2.0, was developed by AtheroPoint LLC, Roseville, CA, USA for cIMT measurement at bulb edge points [49]. Several methods have been published that benchmark computerized methods against manual methods [50,51], along with the variability of studies [28].
With the emergence of machine learning (ML), researchers used a single-layer feed-forward network (SLFN) for IMT boundary estimation [52,53]. While it uses knowledge of the database, it is also not fully automated as it involves some human interventions, particularly for grayscale feature extraction. This has been noticed well during the image-based tissue characterization in many fields of view [54,55,56,57,58]. Under the class of AI, deep learning (DL)-based methods have emerged over the past decade [59,60], offering the advantage of automated feature extraction. The DL-based methods use cascaded and sometimes parallel convolutional layers for feature extraction that can extract the fine details of the objects, which is sometimes not possible through ML-based feature extraction [26,52,61,62,63].
Our group has published nearly 100 machines learning journal publications and we strongly feel that while ML-based methods can be developed in a fully automated way, they, in general, require a more exhaustive method for optimization, unlike DL methods. Since the number of layers in ML-based methods are limited, the refining of features is not possible, unlike in the case of DL-based solutions. Our group has published over 50 deep learning journal publications and our observations prove that DL-based methods offer greater advantages compared to the ML-based methods.
Technically, DL-based methods fall into two categories: supervised and unsupervised. Supervised learning models learn from the labels (pixel or categorical labels or gold standard) provided along with the database, while unsupervised learning-based models do not use such labelled information. Cardiovascular imaging has widely used supervised learning-based methods. Cuadrado-Godia et al. [64] presented a two-stage model for cIMT and plaque measurement. Zhou et al. presented UNet and Unet++ models for plaque segmentation on multi-institutional databases [65,66]. Meshram et al. [67] applied the dilation factor on convolutional layers of the Unet model to study the effect on plaque segmentation.
Recently, a few studies have emerged regarding hybrid deep learning (HDL) models [68,69,70,71,72]. There is no specific definition or criteria for a hybrid deep learning model. Still, any modification in the basic encoder–decoder arms or skip connections of the fundamental UNet architecture may fall under the HDL category. In keeping the spirit of HDL, Jain et al. [73] were the first to propose a few hybrid deep learning (HDL) models for ICA plaque segmentation. The authors showed that HDL models are a fusion of two solo deep learning models with the feature extraction capability of two SDL models. Some faster, small, and low parameter HDL models were proposed by them recently [74]. Another study by Jain et al. [73] used the same HDL model on CCA plaque segmentation. Thus, HDL models are gaining more importance due to their better feature extraction capability.
Attention channel maps were recently introduced in UNet-based deep learning [75,76]; however, they have not been tested in the carotid ultrasound framework. Further, no study has yet benchmarked UNet against other models for a carotid ultrasound framework. Therefore, we hypothesize that attention channel maps, when added to the skip connection of UNet architectures, will improve the performance visually and quantitatively. Figure 1 shows the global system diagram of plaque segmentation using the attention-based UNet-based DL paradigm (named as AtheroEdge™ 3.0, AtheroPoint LLP, Roseville, CA, USA).
The layout of this study is as follows: Section 2 presents database selection, preparation, and baseline characteristics. Section 3 presents the architecture of All UNet models used in this study. Methodology and experiments are presented in Section 4. The results are presented in Section 5. Section 6 shows the performance evaluation, Section 7 presents the discussions, and the paper concludes in Section 8.

2. Database Selection, Preparation, and Baseline Characteristics

We considered multi-institution, multi-ethnic databases for our study. Hence, our experiments are free from data selection bias. We considered three databases in this research work, DB1: ICA database from the United Kingdom; DB2: CCA database from Japan; DB3: CCA database from Hong Kong. Detailed descriptions of the databases are provided here.

2.1. DB1: UK ICA Database

A video database of 99 B-mode ultrasound images was acquired from Imperial College London, UK. The database consisted of datasets from 47 male and 52 female patients with a mean age of 75.04 ± 9.96 years. Each US video was converted into still images. From each patient image pool, we selected 10 images at the interval of 10. After a retrospective analysis of US video scans, we found that two scans were fuzzy and noisy; therefore, we removed these two scans from the database. Finally, we achieved 970 images of moderate-to-high risk plaque images.

2.2. DB2: Japanese Diabetic CCA Database

DB2 was collected from Toho University, Japan. It consisted of 379 CCA images of 190 diabetic patients with a mean age of 68.78 ± 10.88 years. The database consisted of 147 male and 43 female patients. An experienced sonographer performed all the scans. This was a retrospective study, and the institutional review board approved the ethics. A detailed baseline characteristic was recorded at the time of diagnosis and presented in Table 1 based on the measured plaque area.

2.3. DB3: Hong Kong Post-Menopausal Women CCA Database

We collected the DB3 database from 50 Chinese women aged between 54 and 67 years. Further, we ordered six images from each patient; thus, the database constituted 300 images. The patients provided their written consent before the start of the experiment. All patients were post-menopausal women, and 28 such women were also identified and were suffering from other diseases. Of these 28 diseased women, one diabetic, three hypertensives, seven hypercholesteremic, fifteen hypertensive and hypercholesteremic, and two with all abnormalities were identified. The remaining patients had normal BP, cholesterol, and blood glucose levels in a fasting state.

2.4. Data Preparation and Augmentation Technique

Data preparation and augmentation play a vital role in DL-based systems. First, we removed the non-relevant information in ultrasound scans, such as patient ID (name), age, date, and model of the machine, etc., by cropping the grayscale area only. This is very popular in image processing and is a standardized method [45,77]. Secondly, we converted all images of DB1, DB2, and DB3 into equal-sized images of size 224 × 224, compatible with the first layer of the DL models. DB1 and DB2 are from different institutions and countries. Thus, we merged both databases to provide our experiments a multi-ethnic, multi-institutional database. Therefore, we combined the CCA images of DB2 and DB3 into a single folder, DB23, and achieved a new DB23 totaling 379 + 300 = 679 images. We also used a data augmentation technique on our combined CCA DB23 to enhance the number of images in the experiments. For this purpose, we used a rotation transform [–15° to +15°] on all images of combined CCA DB23. Finally, we accomplished a database DB2A, where A stands for augmentation of 679 × 2 = 1358 images.

2.5. Binary Mask Preparation for Supervised Learning

This work falls under the supervised learning-based semantic segmentation of the atherosclerotic plaque from carotid ultrasound images. Thus, the models require pixel-label information for the training phase. We have a team of experienced sonographers and cardiologists who successfully identified the LI and MA layers of the carotid artery and delineated the plaque. Further, another sonographer verified the same for any error. The delineated LI and MA closed borders (contours) in grayscale images are converted into a binary mask using a MATLAB-based program. Further, these binary masks are also converted into equal-sized images of 224 × 224, as with the greyscale image.
Inter- and intra-observer analysis always require a radiologist and sometimes become expensive. Further, while inter- and intra-observer studies were not an integral part of this pilot design, our observations has proven that such analysis leads to variations of between 1% and 5% [78,79,80,81,82,83,84,85,86,87,88]; such ranges are normal and meet the FDA 510 (K) regulations. We intend to integrate this practice in future studies

3. UNet Architectures

3.1. Basic UNet Model

Figure 2 below shows the basic UNet model with four encoder and decoder stages and one bottleneck layer between them. The left part of the architecture forms the encoder arm of the network. Each encoder stage follows the convolutional, ReLU, convolutional, ReLU, and maxpooling layers. The numbers of convolutional filters double in each stage. Thus, the numbers of convolutional filters in successive stages are 64, 128, 256, and 512. The number of trainable parameters (P) after each stage was calculated using the following formula P = m × n × d × b × k , where m is the height of the convolutional filter, n is the width of the convolutional filter, d is the number of filters in the previous layer, b is the bias, and k is the number of filters in the current layer. The convolutional filter size is 3 × 3, and the default bias value (b) is 1 for each stage. Thus, after each encoder stage the number of the trainable parameters was 38,720, 221,440, 885,248, and 3,539,968, respectively. A bottleneck layer is present after the encoder stage, which carries the most delicate features extracted from the encoder stage. This bottleneck layer holds 14,157,824 numbers of trainable parameters. The right-hand part of the UNet architecture is called the decoder arm. The decoder stage follows the up-convolutional layer, followed by concatenation, convolutional, and ReLU layers.
During top to down parameters extraction in the encoder stage, vital semantic features are left in the extraction process. Thus, these features are added to the corresponding decoder stage at the concatenation layer via a skip connection (shown in the Figure 2 by dotted lines). After adding features from the encoder stage, each decoder stage carries 9,176,576, 2,294,528, 573,824, and 143,552 parameters. Finally, fully connected and softmax layers comprise 1154 and six parameters. Thus, Unet architecture contains a total of 31,032,840 training parameters. The Unet model is trained with grayscale images of 224 × 224 images and their corresponding binary masks. The model uses a sparse categorical cross-entropy loss function given by Equation (2) and an ADAM optimizer to reduce the loss function.

3.2. Unet++ Architecture

The Unet++ network is a modified architecture of the basic Unet model. In the basic Unet model, the skip connection directly carries features from the encoder stage and concatenates to the decoder stage. However, the Unet++ model follows a dense skip connection path where features are passed through different intermediate blocks. As shown in the Unet++ architecture in Figure 3, the first, second, third, and fourth stage has 3, 2, 1, and 0 intermediate convolutional blocks. These intermediate blocks are connected to all previous blocks at the same level through the concatenation layer. The encoder arm has the same training parameters as the Unet model; however, in the decoder stage, training parameters change due to addition from the intermediate stages. Table 2 shows the number of training parameters in different parts of the Unet architectures.

3.3. Unet3P

Unet3P is another variant of basic Unet architecture. In the previous Unet and Unet++ models, the skip connections carry the features from the same encoder stage to the same decoder stage (same scale). However, multi-scale features exist at different encoder stages. Thus, these models lack in adding features from multi-scale feature connections. Therefore, the concept of adding multi-scale features to the Unet model arises as Unet+++ or Unet3P. This idiosyncratic Unet3P model merges features from the same-scale and lower-scale features from the encoder with the high-scale features from the decoder. As can be seen from Figure 4, the same-scale features from encoder stage 1, the large scale features from decoder stages 2, 3, and 4, and the bottleneck layers are added to decoder stage 1 via skip connections. Similarly, the lower-scale features from encoder stage 1, the same-scale features from encoder stage 2, and the large-scale features from decoder stages 3 and 4, as well as the bottleneck layers are concatenated to decoder stage 2 via skip connection. The lower scale features from encoder stage 1 and 2, the same-scale features from encoder stage 3, and the large-scale features from decoder stage 4 and the bottleneck layers are added to decoder stage 3. Finally, the lower-scale features from encoder stages 1, 2, and 3, the same-scale features from encoder stage 4, and the high-scale features from the bottleneck layer are concatenated with decoder stage 4. The complete architecture of Unet3P is shown in Figure 4.

3.4. Attention-based Unet Model

The concept of the attention mechanism was proposed earlier by Bahdanau et al. [89] and Luong et al. [90]. Further, the same concept was integrated with the UNet by Oktay et al. [75] for segmentation of the Pancreas. We utilized the attention-UNet for plaque segmentation as the atherosclerotic plaque has a very fuzzy nature, which is challenging to segment using other UNet models in many cases. Figure 5 below shows an attention block used in the place of the skip connection of the UNet model.
The attention gate has two inputs and one output. One of the inputs is “input features” ‘x’ from the same encoder level, and the second input is gating features ‘g’ from the lower decoder level. Both inputs have an inherent property to carry the features. The input feature ‘x’ from the same encoder level comes from a shallow network level; therefore, it contains spatial feature information. The gating signal derives from the decoder level, which is at a deeper level compared to the input signal ‘x’. Thus gating signal provides better feature representation. Further, the input signal is downsampled by a stride of ‘2′ to make it compatible with the gating signal. The downsampled signal ‘x’ is then combined with the gating signal ‘g’ and passed through the rectified linear unit (ReLU) activation function. The ReLU is a non-linear activation function which removes the negative values from the input, i.e., it provides the output if it has positive values.
Further, both ‘x’ and ‘g’ signals are passed through a 1 × 1 convolutional operation to acquire the weights from the combined weight signal. The combined weight signal passes through a Sigmoid activation function. Sigmoid is a ‘S-shaped’ non-linear curve defined by Sig(x) = 1/(1 + exp(–x)). Due to its large slope value, it is able to transform the falling input values to between ‘0′ and ‘1′. Finally, the combined weight signal is upsampled to the same scale as the input signal ‘x’ and multiplied by it element-wise.
We can understand the above attention mechanism with the example provided in Figure 6, which shows the first attention gate at the skip connection between the first encoder and decoder stages. The input features ‘x’ of size 224 × 224 × 64 originate from the first encoder stage, and the gating signal ‘g’ of size 112 × 112 × 128 comes from one lower decoder level. Input signal ‘x’ is downsampled by green block (convolutional 1 × 1 with #filter = gating signal filter i.e., 128) to the size 112 × 112 × 128. Also, the gating signal passes through the blue box (#filters = 128) with 1 × 1 convolution, so it acquires the same shape as ‘x’, i.e., 112 × 112 × 128. Both signals are added and applied to the ReLU activation function, where any nonlinearity is removed from the combined signal.
Further, the combined signal is passed through the psi (Ψ) block, which is a 1 × 1 convolutional block, to acquire the weights from the combined signal. These weights are the attention gate weights generated by resampling the shallow and deep features from the encoder and decoder stages. These weights are upsampled to size 224 × 224 by the upsampler and multiplied to the input signal ‘x’ of the same size. Figure 7 shows the complete attention-UNet model with four encoder and decoder stages. In this architecture, four attention blocks are used in place of the skip connection between the encoder and decoder stages.
Table 2 comprises the training parameters of different parts of all UNet architectures, such as the encoder arm, decoder arm, bottleneck layer, intermediate stages, fully connected layer (FC), and classification layers. We can also compare the total number of parameters of individual models.

4. Methodology and Experiments

All six models were trained using the raw images and binary masks of the ICA DB1 and CCA DB2A mentioned in database Section 2. Various experimental setup steps and key points are mentioned below.

4.1. Hyperparameter Selection and Optimization

The choice of hyperparameters is highly standardized and well-established. The main parameters were (1) # of layers, (2) CV protocol, (3) # of epochs, (4) learning rate, (5) batch size, (6) filter size. The number of layers is shown in a block diagram of all DL models. Additionally, Table 2 shows the total number of parameters in major parts of the DL architecture. The hyperparameters of the experiments include 100 epochs, which we optimized after many experiments with the database. At this number of epochs, no significant change in loss function is observed, and the loss value converges. We selected batch sizes of 8 for UNet, UNet3P, Squeeze-UNet, and attention-UNet, and 4 for the UNet++ and Fractal-UNet models. Further, we used a standard learning rate of 10−4, convolutional filter size of 3 × 3, and bias value of 1 with the same padding.

4.2. Sparse Categorical Cross-Entropy Loss Function

We used a sparse-categorical cross-entropy loss function to minimize the training loss. The loss function is defined as the following Equation (1):
L w = 1 N   i = 1 N y i log y i ^ + 1 y i log 1 y i ^
where, w refers to the model parameters, e.g., weights of the neural network; N is the total number of pixels in an image, y i is the true (actual) label; and y i ^ is the predicted label.

4.3. K5 Cross-Validation

We implemented a commonly used K5 cross-validation [91] method for training and testing. The complete database is divided into 80% training and 20% test datasets in this cross-validation system. In the first step, the training set is used to train the system and offline weight generation, and the test set is used to validate the system and segmentation parameters generation. Again, we transferred the 20% test set back into the main dataset and used another 20% of the images for testing and 80% for training. We repeated the previous step. We repeated this training and testing experiment five times; thus, five offline models were generated corresponding to each test dataset. Now, each test dataset is used with a corresponding offline weight to generate the test results. Thus, all images are tested at least once.
Since the datasets were relatively small and our past experiences in AI protocols showed strong validation results [92], this pilot study did not conduct the validation component. This method can potentially be applied in subsequent experiments.
Experiment 1 is conducted with 970 ICA images. All images are divided into 776 training and 194 test images. The test set of 194 images is replaced with a unique set of 194 images from the training set. Finally, we test our system with five unique sets of the 194 test images. Similarly, experiment 2 is performed with 2 × 679 = 1358 images (DB2A) of Japanese and Hong Kong patients. The complete dataset is divided into five unique test datasets of 271, 271, 272, 272, and 272 images and their corresponding training sets. Each test set is used with the offline model generated after training on the corresponding dataset. Once the test result of each image is generated, we obtain the arithmetic mean of all images using the following Equation (2).
X = 1 N i = 1 N x i
where xi is the extracted feature of image ‘i’, and ‘X’ is the arithmetic mean of ‘N’ images.

5. Results

The segmentation performance of the six models with the ICA DB1 and CCA DB2A databases are shown in Table 3 and Table 4, respectively. We have acquired accuracy, sensitivity, specificity, precision, Mathew’s correlation coefficient (MCC), dice-similarity coefficient (DSC) and Jaccard Index (JI). These indices are calculated by comparing estimated binary masks generated from all UNet models and ground truth masks. The attention-based UNet show mean ± SD values of all parameters as 98.58 ± 0.59, 86.86 ± 5.73, 99.52 ± 0.38, 93.54 ± 4.62, 89.31 ± 3.76, 89.90 ± 3.69, and 81.86 ± 5.90 (all in %) respectively for ICA DB1 database. The same parameters acquire mean ± SD values of 99.01 ± 0.42, 81.75 ± 7.94, 99.74 ± 0.26, 92.71 ± 7.20, 86.38 ± 5.70, 86.50 ± 5.94, and 76.65 ± 8.36 (all in %), respectively, for the CCA DB2A database.
The visual results of UNet, UNet++, UNet3P, Fractal-UNet, Squeeze-UNet, and attention-UNet models are shown in Figure 8. The top row represents binary masks of all databases. The second-row shows overlays of the GT masks over the raw grayscale images in green colour. The third row shows the overlay of the difference between estimated and GT masks on raw grayscale images. Similarly, the fourth, fifth, sixth, seventh, and eighth rows also show the difference between the estimated and GT mask on raw grayscale images for UNet++, UNet3P, Fractal-UNet, Squeeze-UNet, and attention-UNet models, respectively. The red color indicates the estimated mask, and the green color represents the difference of the two masks.
The attention mechanism states that the attention blocks modify the deep features by applying the attention weights. The same can be seen visually in some critical images shown in Figure 9, which are not successfully segmented using other models.

6. Performance Evaluation

The results of all models show almost equal segmentation indices. Hence we perform some more performance evaluation tests to validate our experiments. We conducted a series of performance tests on the ICA DB1 and DB2A databases, such as regression analysis, receiver operating characteristics (ROC) analysis, paired-t-test, and Bland–Altman’s plot analysis.

6.1. Regression Analysis

A regression analysis is a powerful statistical tool to analyze the relation between two quantities. It generates a correlation coefficient (CC) between the two variables, which occupies values between 0 and 1. A CC value close to ‘1’ refers to a very high correlation, whereas a CC close to ‘0’ refers to very little correlation between the two quantities. We used ground truth plaque area (GTPA) and predicted plaque area from the model for the regression analysis. Using ICA DB1, we obtained CC values between GTPA and PA from UNet, UNet++, UNet3P, Fractal-UNet, Squeeze-UNet and Attention-UNet of 0.99, 0.98, 0.98, 0.96, 0.96 and 0.99. Similarly, using CCA DB2A, the CC values for the same models are 0.93, 0.96, 0.92, 0.94, 0.89 and 0.96. All values are summarized in Table 5. The regression analysis curve is shown in Figure 10 and Figure 11. From the analysis of all CC numbers, it is clear that the attention-based UNet outperforms the other models.

6.2. Receiver Operating Characteristics

A receiver operating characteristics (ROC) analysis is another performance evaluation tool to assess the classification performance. We cross-examined the literature and found a plaque area threshold value of 40 mm2 used by researchers to classify the low and high risk plaque. Using this threshold value, we generated GT labels ‘1’ and ‘0’ for GTPA > 40 mm2 and GTPA < 40 mm2, respectively. Using the predicted area as the variable and GT labels as the classification variable, we plotted the ROC curve for UNet, UNet++, UNet3P, Fractal-UNet, Squeeze-UNet, and attention-based UNet models for ICA and CCA databases. These ROC curves are shown in Figure 12 and Figure 13 along with the area under the ROC curve (AUC) and p-values. The AUC values are compared in Table 5. Again, it is clear from the AUC numbers and the ROC curves that the attention-based UNet model outperforms other UNet models.

6.3. Paired-t-Test Analysis

A paired-t-test is mostly used in biostatistics to analyze the mean difference between the two measurements. The requirement for this test is the paired quantity of measurement for the same subject. In our case, we have GTPA and the predicted PA for the same arteries (CCA and ICA both). Thus, this method analyses whether the mean difference between the PA pair is zero or not. The distribution of GTPA and predicted PA is shown using the box and whiskers plot in Figure 14 and Figure 15 for ICA and CCA databases, respectively. The paired t-test results such as mean ± SD, the standard error of the mean, mean difference, Student’s t-value, and p-values are shown in Table 6 and Table 7 for the ICA and CCA databases, respectively.

6.4. Bland-Altman’s Plot

A biostatistical analysis frequently uses a Bland–Altman plot or simply a difference plot when two different methods or instruments measure a parameter. The BA plot is used to show the bias between the mean differences of the two methods. It also offers an agreement interval of 95% (confidence interval), in which the difference between the second and first method fall. In our experiments, plaque area is the quantity which is measured by the expert sonographer by a manual method, i.e., GTPA and the predicted area measured by all UNet models. Thus, the difference between GTPA and the predicted PA is plotted on the Y-axis, and the mean of the both quantity is plotted on the X-axis. The BA plots between GTPA and predicted PA for all the UNet models using the ICA and CCA databases are shown in Figure 16 and Figure 17, respectively.

7. Discussion

The current research work is a novel application of attention-based UNet models for carotid plaque segmentation. We hypothesized that many of the images would be very fuzzy and bright to identify the plaque constituents. Thus, the attention-based UNet model [75,76] enhances the deep features and provides better segmentation. The model also shows excellent segmentation performance for low-to-moderate CCA and moderate-to-high ICA plaque images. Further, we compared the attention-based model against UNet, UNet++, and UNet3P models for a multicenter, multi-ethnic database. We made an effort to present a bias-free stroke risk assessment system.

7.1. Bias in Medical Imaging Models

Deep learning-based clinical models have recently been gaining significant attention. The models use clinical data available from hospitals and medical research centers. Further, these clinical data are processed using established algorithms or new algorithms. These AI-based clinical decision support systems have shown promising results in diagnosis. However, the biases in these systems are not reported in many studies [92,93]. These models may suffer from data selection bias (from single source); observer bias; data-labelling bias; data source bias (data device selection bias); validation bias; racial bias (multi-ethnic data selection); measurement bias; bias due to variabilities in the data sets [94], and many other types of bias which may affect the clinical results in some way. These biases must be discussed in detail and considered while designing the clinical support systems. Recently, methods have been developed to compute biases, which can be extended for UNet-based systems [95,96,97].

7.2. Supervised and Unsupervised Learning Based DL Models

The current algorithm uses binary masks of ICA and CCA images for model training. To successfully train the model’s error-free mask, preparation is a must; failure to prepare may result in the false recognition of the model. Binary mask preparation is tedious and time-consuming, and expert sonographers are required to accomplish this task. The current study involves the hundreds of images due to the unavailability of the datasets. However, considering the case where multiple thousands of images are available for analysis, generating such masks is nearly impossible. Therefore, supervised learning loses its significance in big data analysis. Thus, unsupervised models may replace such scenarios where no binary mask or labelled information is required for training. However, such unsupervised DL models have not gained much attention in medical image processing until now, and significant scope is available in this area. We may see many Unsup-DL models in the near future.

7.3. Benchmarking

We have presented an attention-channel-based UNet model for atherosclerotic plaque segmentation. Many efforts have been made in ICA and CCA plaque area segmentation; still, these methods are not perfect in one way or another. Previous methods suffer from some of the biases discussed in the above section, resulting in poor image segmentation from other databases. Zhou et al. [66] presented the UNet++ model for ICA and CCA plaque segmentation from multi-ethnic databases. However, they trained their model with only 33, 33, and 34 images and tested on 44 images. Using such a low number of images for the test does not infer proper justification to the clinical setting. Further, they did not benchmark their system with any other established system. In another study, Jain et al. presented hybrid deep learning models for ICA plaque segmentation [68]. They proposed SegNet-UNet and SegNet-UNet+ HDL models for plaque area segmentation. Although their model used only one segment of the artery, i.e., ICA, they enhanced the image database by applying the rotation transform augmentation technique. Thus, their models also suffer some biases such as data selection bias, source bias, and racial bias. However, their models are a milestone in HDL model studies.
In another study, the same team of researchers, Jain et al., used the above HDL models for plaque segmentation from multi-ethnic CCA databases [78]. They used two kinds of databases for the experiments, one from Japan and another from Hong Kong, and performed some unseen experiments. Thus, they attempted to avoid data selection and racial bias using these HDL models for unseen experiments. However, they failed to validate their experiments against any established system; therefore, their models suffer from validation bias. Further, they attempted to avoid the validation bias by comparing their SDL and HDL models against a commercially available state-of-the-art plaque segmentation system, AtheroEdge 2.0, developed by AtheroPoint LLC, CA, USA [73]. Their HDL model shows a plaque area error of 8 mm2 compared to 9.9 mm2 for SDL and 9.6 mm2 for AtheroEdge 2.0 models for 90% of the image database. The proposed method attempted to overcome previous bias by intensive exercises. By using databases from ICA and CCA sections, we attempted to avoid data selection bias. We also managed racial biases by incorporating multi-ethnic, multi-center ICA and CCA databases. DB1, DB2, and DB3 are from the UK, Japan and Hong Kong, respectively. Further, we validated our Attention-UNet model against previous UNet, UNet++, and UNet3P models to avoid model selection and validation bias. Table 8 summarizes the comparisons of the present study with some benchmark studies.

7.4. Strength, Weakness, and Extension

The present work shows a bias-free study of the plaque segmentation from ICA and CCA images from multi-ethnic, multicenter databases. We presented a powerful attention mechanism to modify the shallow and deep features of the carotid plaque images, which can capture those plaque areas which other models do not detect. We compared attention-based UNet segmentation results with other models used in previous studies, such as UNet, UNet++, and UNet3P, and the results are comparable or superior to such models. Moreover, the visual results show a promising improvement in many images. Further, we validated our experiments using a series of performance evaluation tests such as regression analysis, ROC, Bland–Altman plots, and paired-t-tests. The results of such performance tests are also comparable or superior to other models.
Further, we attempted to fill the gap in data selection and racial biases from previous studies using multicenter, multi-ethnic, and augmented databases. We believe that these biases are not sufficient to overcome, and there is still scope for their improvement, which can be attempted in future studies, as attempted here [98]. Further, the attention mechanism can be employed in other variants of the UNet to view its effect on other HDLs and such integration of advanced image processing [99] methods with UNet.
Since training models are large, one can adapt weight pruning techniques using algorithms such as genetic algorithms and whale optimization [100,101]. Moreover, the UNet-based segmentation method can be used for plaque tissue characterization due to its vital feature extraction and modification capability [71,102,103]. Carotid segmented lesions and plaque need to be correlated to a coronary SYNTAX score as part of clinical validation [104]. Segmented plaque can be attempted with different clinical groups such as rheumatology patients to understand cardiovascular risk [105]. Finally, since coronary plaque has been observed in COVID-19 patients [106], one can extend the UNet-based solution for plaque segmentation and measurement in carotid scans on COVID-19 patients.

8. Conclusions

This work presents a novel concept of the attention mechanism incorporated with UNet as an attention-based UNet model. The attention-based UNet model successfully demonstrated plaque segmentation in complex images with fuzzy and bright plaque. The results of the attention-UNet models were benchmarked against UNet, UNet++, and UNet3P models. The CC value of the attention-based UNet model for the CCA database was 0.96, compared to 0.93, 0.96, and 0.92 for UNet, UNet++, and UNet3P. The AUC value for attention-based UNet was 0.97, compared to 0.964, 0.966, and 0.965 for the other models. The attention gate weight modifies the shallow and deep features to identify the complex plaque images; therefore, the attention mechanism is vital in plaque feature extraction and tissue characterization. The system can be adopted in clinical settings for cardiovascular disease risk stratification.

Author Contributions

Conceptualization, J.S.S.; Data curation, P.K.J., L.S., N.N.K., J.R.L. Jain, L.S., N.N.K., J.R.L. and A.N.; Formal analysis, L.S., N.N.K. and N.S.; Funding acquisition, M.M.F.; Investigation, N.N.K., J.R.L., A.N., J.S.S. and N.S.; Methodology, P.K.J.; Project administration, J.S.S. and N.S.; Resources, N.S.; Software, P.K.J. and A.D.; Supervision, J.S.S. and N.S.; Validation, A.N.; Visualization, P.K.J. and A.N.; Writing—original draft, P.K.J.; Writing—review & editing, L.S., N.N.K., J.R.L., A.N., M.M.F., J.S.S. and N.S. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by Ohashi Ethics Committee, Approval Number: Hashijo 13–56, Approval Date: 31 October 2013 for Japanese database; The Chinese University of Hong Kong, Hong Kong for Hong Kong database; Institutional ethics committee approval by London-Harrow NRES Committee 9 (11/LO/0299) for ICA database.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The Institutional Review Board has been issued to AtheroPoint, Roseville, CA, USA and therefore this database cannot be shared publically.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Chauhan, S.; Aeri, B.T. The rising incidence of cardiovascular diseases in India: Assessing its economic impact. J. Prev. Cardiol. 2015, 4, 735–740. [Google Scholar]
  2. Jones, S.P.; Baqai, K.; Clegg, A.; Georgiou, R.; Harris, C.; Holland, E.-J.; Kalkonde, Y.; Lightbody, C.E.; Maulik, P.K.; Srivastava, P.M.; et al. Stroke in India: A systematic review of the incidence, prevalence, and case fatality. Int. J. Stroke. 2022, 17, 132–140. [Google Scholar] [CrossRef] [PubMed]
  3. Dorairaj, P.; Jeemon, P.; Roy, A. Cardiovascular Diseases in India: Current Epidemiology and Future Directions. Circulation 2016, 133, 1605–1620. [Google Scholar] [CrossRef]
  4. Molinari, F.; Meiburger, K.M.; Saba, L.; Acharya, U.R.; Famiglietti, L.; Georgiou, N.; Nicolaides, A.; Mamidi, R.S.; Kuper, H.; Suri, J.S. Automated carotid IMT measurement and its validation in low contrast ultrasound database of 885 patient Indian population epidemiological study: Results of AtheroEdge® software. In Multi-Modality Atherosclerosis Imaging and Diagnosis; Springer: Berlin/Heidelberg, Germany, 2014; pp. 209–219. [Google Scholar]
  5. Kamalakannan, S.; Gudlavalleti, A.S.V.; Gudlavalleti, V.S.M.; Goenka, S.; Kuper, H. Incidence & prevalence of stroke in India: A systematic review. Ind. J. Med. Res. 2017, 146, 175–185. [Google Scholar] [CrossRef] [PubMed]
  6. Kleindorfer, D.O.; Towfighi, A.; Chaturvedi, S.; Cockroft, K.M.; Gutierrez, J.; Lombardi-Hill, D.; Kamel, H.; Kernan, W.N.; Kittner, S.J.; Leira, E.C.; et al. 2021 Guideline for the Prevention of Stroke in Patients With Stroke and Transient Ischemic Attack: A Guideline From the American Heart Association/American Stroke Association. Stroke 2021, 52. [Google Scholar] [CrossRef]
  7. Viswanathan, V.; Puvvula, A.; Jamthikar, A.D.; Saba, L.; Johri, A.M.; Kotsis, V.; Khanna, N.N.; Dhanjil, S.K.; Majhail, M.; Misra, D.P.; et al. Bidirectional link between diabetes mellitus and coronavirus disease 2019 leading to cardiovascular disease: A narrative review. World J. Diabetes 2021, 12, 215–237. [Google Scholar] [CrossRef]
  8. Puvvula, A.; Jamthikar, A.D.; Gupta, D.; Khanna, N.N.; Porcu, M.; Saba, L.; Viskovic, K.; Ajuluchukwu, J.N.A.; Gupta, A.; Mavrogeni, S.; et al. Morphological Carotid Plaque Area Is Associated With Glomerular Filtration Rate: A Study of South Asian Indian Patients With Diabetes and Chronic Kidney Disease. Angiology 2020, 71, 520–535. [Google Scholar] [CrossRef] [PubMed]
  9. Virani, S.S.; Alonso, A.; Aparicio, H.J.; Benjamin, E.J.; Bittencourt, M.S.; Callaway, C.W.; Carson, A.P.; Chamberlain, A.M.; Cheng, S.; Delling, F.N.; et al. Heart Disease and Stroke Statistics-2021 Update. Circulation 2021, 143, e254–e743. [Google Scholar] [CrossRef] [PubMed]
  10. Benjamin, E.J.; Virani, S.S.; Callaway, C.W.; Chamberlain, A.M.; Chang, A.R.; Cheng, S.; Chiuve, S.E.; Cushman, M.; Delling, F.N.; Deo, R. Heart disease and stroke statistics—2018 update: A report from the American Heart Association. Circulation 2018, 137, e67–e492. [Google Scholar] [CrossRef] [PubMed]
  11. Benjamin, E.J.; Muntner, P.; Alonso, A.; Bittencourt, M.S.; Callaway, C.W.; Carson, A.P.; Chamberlain, A.M.; Chang, A.R.; Cheng, S.; Das, S.R.; et al. Heart disease and stroke statistics—2019 update: A report from the American heart association. Circulation 2019, 139, e56–e528. [Google Scholar] [CrossRef]
  12. Suri, J.S.; Kathuria, C.; Molinari, F. Atherosclerosis Disease Management; Springer: New York, NY, USA, 2010. [Google Scholar]
  13. Saba, L.; Sanches, J.M.; Pedro, L.M.; Suri, J.S. Multi-Modality Atherosclerosis Imaging and Diagnosis; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
  14. Saba, L.; Antignani, P.L.; Gupta, A.; Cau, R.; Paraskevas, K.I.; Poredos, P.; Wasserman, B.A.; Kamel, H.; Avgerinos, E.D.; Salgado, R.; et al. International Union of Angiology (IUA) consensus paper on imaging strategies in atherosclerotic carotid artery imaging: From basic strategies to advanced approaches. Atherosclerosis 2022, 354, 23–40. [Google Scholar] [CrossRef] [PubMed]
  15. Patel, A.K.; Suri, H.S.; Singh, J.; Kumar, D.; Shafique, S.; Nicolaides, A.; Jain, S.K.; Saba, L.; Gupta, A.; Laird, J.R.; et al. A Review on Atherosclerotic Biology, Wall Stiffness, Physics of Elasticity, and Its Ultrasound-Based Measurement. Curr. Atheroscler. Rep. 2016, 18, 83. [Google Scholar] [CrossRef] [PubMed]
  16. Saba, L.; Nardi, V.; Cau, R.; Gupta, A.; Kamel, H.; Suri, J.S.; Balestrieri, A.; Congiu, T.; Butler, A.P.; Gieseg, S.; et al. Carotid Artery Plaque Calcifications: Lessons From Histopathology to Diagnostic Imaging. Stroke 2022, 53, 290–297. [Google Scholar] [CrossRef] [PubMed]
  17. Kotsis, V.; Jamthikar, A.; Araki, T.; Gupta, D.; Laird, J.R.; Giannopoulos, A.A.; Saba, L.; Suri, H.S.; Mavrogeni, S.; Kitas, G.D.; et al. Echolucency-based phenotype in carotid atherosclerosis disease for risk stratification of diabetes patients. Diabetes Res. Clin. Pract. 2018, 143, 322–331. [Google Scholar] [CrossRef]
  18. Placidi, G. MRI: Essentials for Innovative Technologies; CRC Press: Boca Raton, FL, USA, 2012. [Google Scholar]
  19. Saba, L.; Suri, J.S. Multi-Detector CT Imaging: Principles, Head, Neck, and Vascular Systems; CRC Press: Florida, FL, USA, 2013; Volume 1. [Google Scholar]
  20. Trivedi, R.; Saba, L.; Suri, J.S. 3D Imaging Technologies in Atherosclerosis; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
  21. Suri, J.S. Advances in Diagnostic and Therapeutic Ultrasound Imaging; Artech House: London, UK, 2008. [Google Scholar]
  22. Laine, A.; Sanches, J.M.; Suri, J.S. Ultrasound Imaging: Advances and Applications; Springer: New York, NY, USA, 2012. [Google Scholar]
  23. Sharma, A.M.; Gupta, A.; Kumar, P.K.; Rajan, J.; Saba, L.; Nobutaka, I.; Laird, J.R.; Nicolades, A.; Suri, J.S. A Review on Carotid Ultrasound Atherosclerotic Tissue Characterization and Stroke Risk Stratification in Machine Learning Framework. Curr. Atheroscler. Rep. 2015, 17. [Google Scholar] [CrossRef]
  24. Kumar, P.K.; Araki, T.; Rajan, J.; Laird, J.R.; Nicolaides, A.; Suri, J.S. State-of-the-art review on automated lumen and adventitial border delineation and its measurements in carotid ultrasound. Comput. Methods Programs Biomed. 2018, 163, 155–168. [Google Scholar] [CrossRef]
  25. Saba, L.; Jamthikar, A.; Gupta, D.; Khanna, N.N.; Viskovic, K.; Suri, H.S.; Gupta, A.; Mavrogeni, S.; Turk, M.; Laird, J.R. Global perspective on carotid intima-media thickness and plaque: Should the current measurement guidelines be revisited? Int. Angiol. 2019, 38, 451–465. [Google Scholar] [CrossRef]
  26. Molinari, F.; Meiburger, K.M.; Acharya, U.R.; Zeng, G.; Rodrigues, P.S.; Saba, L.; Nicolaides, A.; Suri, J.S. CARES 3.0: A two stage system combining feature-based recognition and edge-based segmentation for CIMT measurement on a multi-institutional ultrasound database of 300 images. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011; pp. 5149–5152. [Google Scholar] [CrossRef]
  27. Molinari, F.; Meiburger, K.M.; Zeng, G.; Nicolaides, A.; Suri, J.S. CAUDLES-EF: Carotid Automated Ultrasound Double Line Extraction System Using Edge Flow. In Ultrasound Imaging; Springer: Boston, MA, USA, 2012; pp. 129–162. [Google Scholar] [CrossRef]
  28. Molinari, F.; Liboni, W.; Pantziaris, M.; Suri, J. CALSFOAM-completed automated local statistics based first order absolute moment for carotid wall recognition, segmentation and IMT measurement: Validation and benchmarking on a 300 patient database. Int. J. Angiol. 2011, 30, 227–241. [Google Scholar]
  29. Molinari, F.; Meiburger, K.M.; Saba, L.; Acharya, U.R.; Ledda, G.; Zeng, G.; Ho, S.Y.S.; Ahuja, A.T.; Ho, S.C.; Nicolaides, A.; et al. Ultrasound IMT measurement on a multi-ethnic and multi-institutional database: Our review and experience using four fully automated and one semi-automated methods. Comput. Methods Programs Biomed. 2012, 108, 946–960. [Google Scholar] [CrossRef]
  30. Faita, F.; Gemignani, V.; Bianchini, E.; Giannarelli, C.; Ghiadoni, L.; Demi, M. Real-time Measurement System for Evaluation of the Carotid Intima-Media Thickness With a Robust Edge Operator. J. Ultrasound Med. 2008, 27, 1353–1361. [Google Scholar] [CrossRef] [Green Version]
  31. Molinari, F.; Pattichis, C.S.; Zeng, G.; Saba, L.; Acharya, U.R.; Sanfilippo, R.; Nicolaides, A.; Suri, J.S. Completely Automated Multiresolution Edge Snapper—A New Technique for an Accurate Carotid Ultrasound IMT Measurement: Clinical Validation and Benchmarking on a Multi-Institutional Database. IEEE Trans. Image Process. 2012, 21, 1211–1222. [Google Scholar] [CrossRef] [PubMed]
  32. Zahnd, G.; Kapellas, K.; van Hattem, M.; van Dijk, A.; Sérusclat, A.; Moulin, P.; van der Lugt, A.; Skilton, M.; Orkisz, M. A Fully-Automatic Method to Segment the Carotid Artery Layers in Ultrasound Imaging: Application to Quantify the Compression-Decompression Pattern of the Intima-Media Complex During the Cardiac Cycle. Ultrasound Med. Biol. 2016, 43, 239–257. [Google Scholar] [CrossRef] [PubMed]
  33. Rouco, J.; Carvalho, C.; Domingues, A.; Azevedo, E.; Campilho, A. A robust anisotropic edge detection method for carotid ultrasound image processing. Procedia Comput. Sci. 2018, 126, 723–732. [Google Scholar] [CrossRef]
  34. Loizou, C.P.; Pattichis, C.; Pantziaris, M.; Tyllis, T.; Nicolaides, A. Snakes based segmentation of the common carotid artery intima media. Med. Biol. Eng. Comput. 2007, 45, 35–49. [Google Scholar] [CrossRef] [PubMed]
  35. Molinari, F.; Meiburger, K.M.; Saba, L.; Rajendra Acharya, U.; Ledda, M.; Nicolaides, A.; Suri, J.S. Constrained snake vs. conventional snake for carotid ultrasound automated IMT measurements on multi-center data sets. Ultrasonics 2012, 52, 949–961. [Google Scholar] [CrossRef]
  36. Molinari, F.; Meiburger, K.M.; Saba, L.; Zeng, G.; Acharya, U.R.; Ledda, M.; Nicolaides, A.; Suri, J.S. Fully Automated Dual-Snake Formulation for Carotid Intima-Media Thickness Measurement. J. Ultrasound Med. 2012, 31, 1123–1136. [Google Scholar] [CrossRef]
  37. Kumar, P.K.; Araki, T.; Rajan, J.; Saba, L.; Lavra, F.; Ikeda, N.; Sharma, A.M.; Shafique, S.; Nicolaides, A.; Laird, J.R.; et al. Accurate lumen diameter measurement in curved vessels in carotid ultrasound: An iterative scale-space and spatial transformation approach. Med. Biol. Eng. Comput. 2016, 55, 1415–1434. [Google Scholar] [CrossRef]
  38. Suri, J.S.; Liu, K.; Reden, L.; Laxminarayan, S. A review on MR vascular image processing algorithms: Acquisition and prefiltering: Part I. IEEE Trans. Inf. Technol. Biomed. 2002, 6, 324–337. [Google Scholar] [CrossRef]
  39. Molinari, F.; Zeng, G.; Suri, J.S. Intima-media thickness: Setting a standard for a completely automated method of ultrasound measurement. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 2010, 57, 1112–1124. [Google Scholar] [CrossRef]
  40. Araki, T.; Ikeda, N.; Dey, N.; Acharjee, S.; Molinari, F.; Saba, L.; Godia, E.C.; Nicolaides, A.; Suri, J.S. Shape-Based Approach for Coronary Calcium Lesion Volume Measurement on Intravascular Ultrasound Imaging and Its Association With Carotid Intima-Media Thickness. J. Ultrasound Med. 2015, 34, 469–482. [Google Scholar] [CrossRef]
  41. Yang, X.; Jin, J.; Xu, M.; Wu, H.; He, W.; Yuchi, M.; Ding, M. Ultrasound Common Carotid Artery Segmentation Based on Active Shape Model. Comput. Math. Methods Med. 2013, 2013, 1–11. [Google Scholar] [CrossRef] [PubMed]
  42. Molinari, F.; Krishnamurthi, G.; Acharya, U.R.; Subbhuraam, V.S.; Saba, L.; Nicolaides, A.; Suri, J. Hypothesis Validation of Far-Wall Brightness in Carotid-Artery Ultrasound for Feature-Based IMT Measurement Using a Combination of Level-Set Segmentation and Registration. IEEE Trans. Instrum. Meas. 2012, 61, 1054–1063. [Google Scholar] [CrossRef]
  43. El-Baz, A.; Suri, J.S. Level Set Method in Medical Imaging Segmentation; CRC Press: Boca Raton, FL, USA, 2019. [Google Scholar] [CrossRef]
  44. Li, C.; Xu, C.; Gui, C.; Fox, M.D. Level set evolution without re-initialization: A new variational formulation. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 20–25 June 2005; pp. 430–436. [Google Scholar]
  45. Araki, T.; Kumar, P.K.; Suri, H.S.; Ikeda, N.; Gupta, A.; Saba, L.; Rajan, J.; Lavra, F.; Sharma, A.M.; Shafique, S. Two automated techniques for carotid lumen diameter measurement: Regional versus boundary approaches. J. Med. Syst. 2016, 40, 1–19. [Google Scholar] [CrossRef]
  46. Suri, J.; Liu, K.; Singh, S.; Laxminarayan, S.; Zeng, X.; Reden, L. Shape recovery algorithms using level sets in 2-D/3-D medical imagery: A state-of-the-art review. IEEE Trans. Inf. Technol. Biomed. 2002, 6, 8–28. [Google Scholar] [CrossRef]
  47. Giraldi, G.A.; Rodrigues, P.S.; Suri, J.S. Implicit dual snakes for medical imaging. In Proceedings of the 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, USA, 30 August–3 September 2006; pp. 3025–3028. [Google Scholar]
  48. Roy, S.; Bhattacharyya, D.; Bandyopadhyay, S.K.; Kim, T.-H. An iterative implementation of level set for precise segmentation of brain tissues and abnormality detection from MR images. IETE J. Res. 2017, 63, 769–783. [Google Scholar] [CrossRef]
  49. Ikeda, N.; Dey, N.; Sharma, A.; Gupta, A.; Bose, S.; Acharjee, S.; Shafique, S.; Cuadrado-Godia, E.; Araki, T.; Saba, L.; et al. Automated segmental-IMT measurement in thin/thick plaque with bulb presence in carotid ultrasound from multiple scanners: Stroke risk assessment. Comput. Methods Programs Biomed. 2017, 141, 73–81. [Google Scholar] [CrossRef] [PubMed]
  50. Saba, L.; Banchhor, S.K.; Suri, H.S.; Londhe, N.D.; Araki, T.; Ikeda, N.; Viskovic, K.; Shafique, S.; Laird, J.R.; Gupta, A. Accurate cloud-based smart IMT measurement, its validation and stroke risk stratification in carotid ultrasound: A web-based point-of-care tool for multicenter clinical trial. Comput. Biol. Med. 2016, 75, 217–234. [Google Scholar] [CrossRef]
  51. Meiburger, K.M.; Marzola, F.; Zahnd, G.; Faita, F.; Loizou, C.P.; Lainé, N.; Carvalho, C.; Steinman, D.A.; Gibello, L.; Bruno, R.M. Carotid Ultrasound Boundary Study (CUBS): Technical considerations on an open multi-center analysis of computerized measurement systems for intima-media thickness measurement on common carotid artery longitudinal B-mode ultrasound scans. Comput. Biol. Med. 2022, 144, 105333. [Google Scholar] [CrossRef]
  52. Menchón-Lara, R.-M.; Bastida-Jumilla, M.-C.; Larrey-Ruiz, J.; Verdú-Monedero, R.; Morales-Sánchez, J.; Sancho-Gómez, J.-L. Measurement of Carotid Intima-Media Thickness in ultrasound images by means of an automatic segmentation process based on machine learning. Proceedings of Eurocon 2013, Zagreb, Croatia, 1–4 July 2013; pp. 2086–2093. [Google Scholar]
  53. Biswas, M.; Saba, L.; Omerzu, T.; Johri, A.M.; Khanna, N.N.; Viskovic, K.; Mavrogeni, S.; Laird, J.R.; Pareek, G.; Miner, M. A review on joint carotid intima-media thickness and plaque area measurement in ultrasound for cardiovascular/stroke risk monitoring: Artificial intelligence framework. J. Digit. Imaging. 2021, 34, 581–604. [Google Scholar] [CrossRef]
  54. Kuppili, V.; Biswas, M.; Sreekumar, A.; Suri, H.S.; Saba, L.; Edla, D.R.; Marinhoe, R.T.; Sanches, J.; Suri, J.S. Extreme Learning Machine Framework for Risk Stratification of Fatty Liver Disease Using Ultrasound Tissue Characterization. J. Med Syst. 2017, 41, 152. [Google Scholar] [CrossRef]
  55. Shrivastava, V.K.; Londhe, N.D.; Sonawane, R.S.; Suri, J.S. Reliable and accurate psoriasis disease classification in dermatology images using comprehensive feature space in machine learning paradigm. Expert Syst. Appl. 2015, 42, 6184–6195. [Google Scholar] [CrossRef]
  56. Araki, T.; Ikeda, N.; Shukla, D.; Jain, P.K.; Londhe, N.D.; Shrivastava, V.K.; Banchhor, S.K.; Saba, L.; Nicolaides, A.; Shafique, S.; et al. PCA-based polling strategy in machine learning framework for coronary artery disease risk assessment in intravascular ultrasound: A link between carotid and coronary grayscale plaque morphology. Comput. Methods Programs Biomed. 2016, 128, 137–158. [Google Scholar] [CrossRef] [PubMed]
  57. Acharya, U.R.; Faust, O.S.; Faust, O.S.V.S.; Krishnamurthi, A.; Faust, G.; Seabra, J.; Sanches, J.; Suri, J.S. Understanding symptomatology of atherosclerotic plaque by image-based tissue characterization. Comput. Methods Programs Biomed. 2013, 110, 66–75. [Google Scholar] [CrossRef] [PubMed]
  58. Boi, A.; Jamthikar, A.D.; Saba, L.; Gupta, D.; Sharma, A.; Loi, B.; Laird, J.R.; Khanna, N.N.; Suri, J.S. A survey on coronary atherosclerotic plaque tissue characterization in intravascular optical coherence tomography. Curr. Atheroscler. Rep. 2018, 20, 1–17. [Google Scholar] [CrossRef]
  59. Saba, L.; Biswas, M.; Kuppili, V.; Godia, E.C.; Suri, H.S.; Edla, D.R.; Omerzu, T.; Laird, J.R.; Khanna, N.N.; Mavrogeni, S.; et al. The present and future of deep learning in radiology. Eur. J. Radiol. 2019, 114, 14–24. [Google Scholar] [CrossRef]
  60. Mainak, B.; Venkatanareshbabu, K.; Luca, S.; Damodar, R.E.; Elisa, C.-G.; Tato, M.R.; Andrew, N. State-of-the-art review on deep learning in medical imaging. Front. Bioscience-Landmark 2019, 24, 380–406. [Google Scholar]
  61. Saba, L.; Jain, P.K.; Suri, H.S.; Ikeda, N.; Araki, T.; Singh, B.K.; Nicolaides, A.; Shafique, S.; Gupta, A.; Laird, J.R.; et al. Plaque Tissue Morphology-Based Stroke Risk Stratification Using Carotid Ultrasound: A Polling-Based PCA Learning Paradigm. J. Med Syst. 2017, 41, 98. [Google Scholar] [CrossRef]
  62. Araki, T.; Jain, P.K.; Suri, H.S.; Londhe, N.D.; Ikeda, N.; El-Baz, A.; Shrivastava, V.K.; Saba, L.; Nicolaides, A.; Shafique, S.; et al. Stroke Risk Stratification and its Validation using Ultrasonic Echolucent Carotid Wall Plaque Morphology: A Machine Learning Paradigm. Comput. Biol. Med. 2017, 80, 77–96. [Google Scholar] [CrossRef]
  63. Menchón-Lara, R.-M.; Sancho-Gómez, J.-L. Fully automatic segmentation of ultrasound common carotid artery images based on machine learning. Neurocomputing 2015, 151, 161–167. [Google Scholar] [CrossRef]
  64. Cuadrado-Godia, E.; Srivastava, S.K.; Saba, L.; Araki, T.; Suri, H.S.; Giannopolulos, A.; Omerzu, T.; Laird, J.; Khanna, N.N.; Mavrogeni, S. Geometric total plaque area is an equally powerful phenotype compared with carotid intima-media thickness for stroke risk assessment: A deep learning approach. J. Vasc. Ultrasound 2018, 42, 162–188. [Google Scholar] [CrossRef]
  65. Zhou, R.; Azarpazhooh, M.R.; Spence, J.D.; Hashemi, S.; Ma, W.; Cheng, X.; Gan, H.; Ding, M.; Fenster, A. Deep Learning-Based Carotid Plaque Segmentation from B-Mode Ultrasound Images. Ultrasound Med. Biol. 2021, 47, 2723–2733. [Google Scholar] [CrossRef] [PubMed]
  66. Zhou, R.; Guo, F.; Azarpazhooh, M.R.; Hashemi, S.; Cheng, X.; Spence, J.D.; Ding, M.; Fenster, A. Deep Learning-Based Measurement of Total Plaque Area in B-Mode Ultrasound Images. IEEE J. Biomed. Heal. Informatics 2021, 25, 2967–2977. [Google Scholar] [CrossRef] [PubMed]
  67. Meshram, N.H.; Mitchell, C.C.; Wilbrand, S.; Dempsey, R.J.; Varghese, T. Deep Learning for Carotid Plaque Segmentation using a Dilated U-Net Architecture. Ultrason. Imaging 2020, 42, 221–230. [Google Scholar] [CrossRef] [PubMed]
  68. Jain, P.K.; Sharma, N.; Giannopoulos, A.A.; Saba, L.; Nicolaides, A.; Suri, J.S. Hybrid deep learning segmentation models for atherosclerotic plaque in internal carotid artery B-mode ultrasound. Comput. Biol. Med. 2021, 136, 104721. [Google Scholar] [CrossRef]
  69. Jena, B.; Saxena, S.; Nayak, G.K.; Saba, L.; Sharma, N.; Suri, J.S. Artificial intelligence-based hybrid deep learning models for image classification: The first narrative review. Comput. Biol. Med. 2021, 137, 104803. [Google Scholar] [CrossRef]
  70. Suri, J.S.; Agarwal, S.; Elavarthi, P.; Pathak, R.; Ketireddy, V.; Columbu, M.; Saba, L.; Gupta, S.K.; Faa, G.; Singh, I.M.; et al. Inter-Variability Study of COVLIAS 1.0: Hybrid Deep Learning Models for COVID-19 Lung Segmentation in Computed Tomography. Diagnostics 2021, 11, 2025. [Google Scholar] [CrossRef]
  71. Skandha, S.S.; Nicolaides, A.; Gupta, S.K.; Koppula, V.K.; Saba, L.; Johri, A.M.; Kalra, M.S.; Suri, J.S. A hybrid deep learning paradigm for carotid plaque tissue characterization and its validation in multicenter cohorts using a supercomputer framework. Comput. Biol. Med. 2022, 141, 105131. [Google Scholar] [CrossRef]
  72. Suri, J.S.; Agarwal, S.; Chabert, G.L.; Carriero, A.; Paschè, A.; Danna, P.S.C.; Saba, L.; Mehmedović, A.; Faa, G.; Singh, I.M.; et al. COVLIAS 2.0-cXAI: Cloud-Based Explainable Deep Learning System for COVID-19 Lesion Localization in Computed Tomography Scans. Diagnostics 2022, 12, 1482. [Google Scholar] [CrossRef]
  73. Jain, P.K.; Sharma, N.; Saba, L.; Paraskevas, K.I.; Kalra, M.K.; Johri, A.; Nicolaides, A.N.; Suri, J.S. Automated deep learning-based paradigm for high-risk plaque detection in B-mode common carotid ultrasound scans: An asymptomatic Japanese cohort study. Int. Angiol. 2021, 41, 9–23. [Google Scholar] [CrossRef]
  74. Jain, P.K.; Sharma, N.; Kalra, M.K.; Johri, A.; Saba, L.; Suri, J.S. Far wall plaque segmentation and area measurement in common and internal carotid artery ultrasound using U-series architectures: An unseen Artificial Intelligence paradigm for stroke risk assessment. Comput. Biol. Med. 2022, 149, 106017. [Google Scholar] [CrossRef]
  75. Oktay, O.; Schlemper, J.; Folgoc, L.; Lee, M.; Heinrich, M.; Misawa, K.; Mori, K.; McDonagh, S.; Hammerla, N.; Kainz, B.; et al. Attention U-Net: Learning Where to Look for the Pancreas. arXiv 2018, arXiv:1804.03999v3. [Google Scholar]
  76. Pal, D.; Reddy, P.B.; Roy, S. Attention UW-Net: A fully connected model for automatic segmentation and annotation of chest X-ray. Comput. Biol. Med. 2022, 106083. [Google Scholar] [CrossRef] [PubMed]
  77. Molinari, F.; Liboni, W.; Giustetto, P.; Badalamenti, S.; Suri, J.S. AUTOMATIC COMPUTER-BASED TRACINGS (ACT) IN LONGITUDINAL 2-D ULTRASOUND IMAGES USING DIFFERENT SCANNERS. J. Mech. Med. Biol. 2009, 9, 481–505. [Google Scholar] [CrossRef]
  78. Jain, P.K.; Sharma, N.; Saba, L.; Paraskevas, K.I.; Kalra, M.K.; Johri, A.; Laird, J.R.; Nicolaides, A.N.; Suri, J.S. Unseen Artificial Intelligence—Deep Learning Paradigm for Segmentation of Low Atherosclerotic Plaque in Carotid Ultrasound: A Multicenter Cardiovascular Study. Diagnostics 2021, 11, 2257. [Google Scholar] [CrossRef]
  79. Suri, J.S.; Agarwal, S.; Chabert, G.L.; Carriero, A.; Paschè, A.; Danna, P.S.C.; Saba, L.; Mehmedović, A.; Faa, G.; Singh, I.M.; et al. COVLIAS 1.0Lesion vs. MedSeg: An Artificial Intelligence Framework for Automated Lesion Segmentation in COVID-19 Lung Computed Tomography Scans. Diagnostics 2022, 12, 1283. [Google Scholar] [CrossRef]
  80. Suri, J.; Agarwal, S.; Pathak, R.; Ketireddy, V.; Columbu, M.; Saba, L.; Gupta, S.; Faa, G.; Singh, I.; Turk, M.; et al. COVLIAS 1.0: Lung Segmentation in COVID-19 Computed Tomography Scans Using Hybrid Deep Learning Artificial Intelligence Models. Diagnostics 2021, 11, 1405. [Google Scholar] [CrossRef]
  81. Saba, L.; Banchhor, S.K.; Araki, T.; Viskovic, K.; Londhe, N.D.; Laird, J.R.; Suri, H.S.; Suri, J.S. Intra- and inter-operator reproducibility of automated cloud-based carotid lumen diameter ultrasound measurement. Ind. Heart J. 2018, 70, 649–664. [Google Scholar] [CrossRef]
  82. Than, J.C.; Saba, L.; Noor, N.M.; Rijal, O.M.; Kassim, R.M.; Yunus, A.; Suri, H.S.; Porcu, M.; Suri, J.S. Lung disease stratification using amalgamation of Riesz and Gabor transforms in machine learning framework. Comput. Biol. Med. 2017, 89, 197–211. [Google Scholar] [CrossRef]
  83. Saba, L.; Than, J.C.M.; Noor, N.M.; Rijal, O.M.; Kassim, R.M.; Yunus, A.; Ng, C.R.; Suri, J.S. Inter-observer Variability Analysis of Automatic Lung Delineation in Normal and Disease Patients. J. Med. Syst. 2016, 40, 1–18. [Google Scholar] [CrossRef]
  84. Lucatelli, P.; Raz, E.; Saba, L.; Argiolas, G.M.; Montisci, R.; Wintermark, M.; King, K.S.; Molinari, F.; Ikeda, N.; Siotto, P.; et al. Relationship between leukoaraiosis, carotid intima-media thickness and intima-media thickness variability: Preliminary results. Eur. Radiol. 2016, 26, 4423–4431. [Google Scholar] [CrossRef]
  85. Ikeda, N.; Araki, T.; Sugi, K.; Nakamura, M.; Deidda, M.; Molinari, F.; Meiburger, K.M.; Acharya, U.R.; Saba, L.; Bassareo, P.P.; et al. Ankle–Brachial Index and Its Link to Automated Carotid Ultrasound Measurement of Intima–Media Thickness Variability in 500 Japanese Coronary Artery Disease Patients. Curr. Atheroscler. Rep. 2014, 16, 1–8. [Google Scholar] [CrossRef] [PubMed]
  86. Saba, L.; Molinari, F.; Meiburger, K.M.; Acharya, U.R.; Nicolaides, A.; Suri, J.S. Inter- and intra-observer variability analysis of completely automated cIMT measurement software (AtheroEdge™) and its benchmarking against commercial ultrasound scanner and expert Readers. Comput. Biol. Med. 2013, 43, 1261–1272. [Google Scholar] [CrossRef] [PubMed]
  87. Rajendra Acharya, U.; Paul Joseph, K.; Kannathal, N.; Lim, C.M.; Suri, J.S. Heart rate variability: A review. Med. Biol. Eng. Comput. 2006, 44, 1031–1051. [Google Scholar] [CrossRef] [PubMed]
  88. Zhang, S.; Suri, J.S.; Salvado, O.; Chen, Y.; Wacker, F.K.; Wilson, D.L.; Duerk, J.L.; Lewin, J.S. Inter- and Intra-Observer Variability Assessment of in Vivo Carotid Plaque Burden Quantification Using Multi-Contrast Dark Blood MR Images. Stud. Heal. Technol. Inform. 2005, 113, 384–393. [Google Scholar]
  89. Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint 2014, arXiv:1409.0473. [Google Scholar]
  90. Luong, M.-T.; Pham, H.; Manning, C.D. Effective approaches to attention-based neural machine translation. arXiv Preprint 2015, arXiv:1508.04025. [Google Scholar]
  91. Jain, P.K.; Gupta, S.; Bhavsar, A.; Nigam, A.; Sharma, N. Localization of common carotid artery transverse section in B-mode ultrasound images using faster RCNN: A deep learning approach. Med. Biol. Eng. Comput. 2020, 58, 471–482. [Google Scholar] [CrossRef]
  92. Suri, J.S.; Agarwal, S.; Gupta, S.K.; Puvvula, A.; Viskovic, K.; Suri, N.; Alizad, A.; El-Baz, A.; Saba, L.; Fatemi, M.; et al. Systematic Review of Artificial Intelligence in Acute Respiratory Distress Syndrome for COVID-19 Lung Patients: A Biomedical Imaging Perspective. IEEE J. Biomed. Health Inform. 2021, 25, 4128–4139. [Google Scholar] [CrossRef]
  93. Suri, J.S.; Agarwal, S.; Jena, B.; Saxena, S.; El-Baz, A.; Agarwal, V.; Kalra, M.K.; Saba, L.; Viskovic, K.; Fatemi, M.; et al. Five Strategies for Bias Estimation in Artificial Intelligence-based Hybrid Deep Learning for Acute Respiratory Distress Syndrome COVID-19 Lung Infected Patients using AP(ai)Bias 2.0: A Systematic Review. IEEE Trans. Instrum. Meas 2022, 1. [Google Scholar] [CrossRef]
  94. Acharya, U.R.; Joseph, K.P.; Kannathal, N.; Min, L.C.; Suri, J.S. Heart rate variability. In Advances in Cardiac Signal Processing; Springer: Berlin/Heidelberg, Germany, 2007; pp. 121–165. [Google Scholar]
  95. Suri, J.S.; Bhagawati, M.; Paul, S.; Protogerou, A.D.; Sfikakis, P.P.; Kitas, G.D.; Khanna, N.N.; Ruzsa, Z.; Sharma, A.M.; Saxena, S.; et al. A Powerful Paradigm for Cardiovascular Risk Stratification Using Multiclass, Multi-Label, and Ensemble-Based Machine Learning Paradigms: A Narrative Review. Diagnostics 2022, 12, 722. [Google Scholar] [CrossRef]
  96. Paul, S.; Maindarkar, M.; Saxena, S.; Saba, L.; Turk, M.; Kalra, M.; Krishnan, P.R.; Suri, J.S. Bias Investigation in Artificial Intelligence Systems for Early Detection of Parkinson’s Disease: A Narrative Review. Diagnostics 2022, 12, 166. [Google Scholar] [CrossRef] [PubMed]
  97. Suri, J.S.; Bhagawati, M.; Paul, S.; Protogeron, A.; Sfikakis, P.P.; Kitas, G.D.; Khanna, N.N.; Ruzsa, Z.; Sharma, A.M.; Saxena, S.; et al. Understanding the bias in machine learning systems for cardiovascular disease risk assessment: The first of its kind review. Comput. Biol. Med. 2022, 142, 105204. [Google Scholar] [CrossRef] [PubMed]
  98. Das, S.; Nayak, G.; Saba, L.; Kalra, M.; Suri, J.S.; Saxena, S. An artificial intelligence framework and its bias for brain tumor segmentation: A narrative review. Comput. Biol. Med. 2022, 143, 105273. [Google Scholar] [CrossRef] [PubMed]
  99. El-Baz, A.; Gimel’Farb, G.; Suri, J.S. Stochastic Modeling for Medical Image Analysis; CRC Press: Boca Raton, FL, USA, 2015. [Google Scholar] [CrossRef]
  100. Acharya, U.R.; Mookiah, M.R.K.; Subbhuraam, V.S.; Yanti, R.; Martis, R.; Saba, L.; Molinari, F.; Guerriero, S.; Suri, J. Evolutionary Algorithm-Based Classifier Parameter Tuning for Automatic Ovarian Cancer Tissue Characterization and Classification. Ultraschall Der Med. Eur. J. Ultrasound 2012, 35, 237–245. [Google Scholar] [CrossRef]
  101. Agarwal, M.; Agarwal, S.; Saba, L.; Chabert, G.L.; Gupta, S.; Carriero, A.; Pasche, A.; Danna, P.; Mehmedovic, A.; Faa, G.; et al. Eight pruning deep learning models for low storage and high-speed COVID-19 computed tomography lung segmentation and heatmap-based lesion localization: A multicenter study using COVLIAS 2. Comput. Biol. Med. 2022, 146, 105571. [Google Scholar] [CrossRef]
  102. Saba, L.; Sanagala, S.S.; Gupta, S.K.; Koppula, V.K.; Johri, A.M.; Khanna, N.N.; Mavrogeni, S.; Laird, J.R.; Pareek, G.; Miner, M.; et al. Multimodality carotid plaque tissue characterization and classification in the artificial intelligence paradigm: A narrative review for stroke application. Ann. Transl. Med. 2021, 9, 1206. [Google Scholar] [CrossRef]
  103. Sanagala, S.S.; Nicolaides, A.; Gupta, S.K.; Koppula, V.K.; Saba, L.; Agarwal, S.; Johri, A.M.; Kalra, M.S.; Suri, J.S. Ten Fast Transfer Learning Models for Carotid Ultrasound Plaque Tissue Characterization in Augmentation Framework Embedded with Heatmaps for Stroke Risk Stratification. Diagnostics 2021, 11, 2109. [Google Scholar] [CrossRef]
  104. Ikeda, N.; Gupta, A.; Dey, N.; Bose, S.; Shafique, S.; Arak, T.; Godia, E.C.; Saba, L.; Laird, J.R.; Nicolaides, A.N.; et al. Improved correlation between carotid and coronary atherosclerosis SYNTAX score using automated ultrasound carotid bulb plaque IMT measurement. Ultrasound Med. Biol. 2015, 41, 1247–1262. [Google Scholar] [CrossRef]
  105. Khanna, N.N.; Jamthikar, A.D.; Gupta, D.; Piga, M.; Saba, L.; Carcassi, C.; Giannopoulos, A.A.; Nicolaides, A.; Laird, J.R.; Suri, H.S.; et al. Rheumatoid Arthritis: Atherosclerosis Imaging and Cardiovascular Risk Assessment Using Machine and Deep Learning–Based Tissue Characterization. Curr. Atheroscler. Rep. 2019, 21, 7. [Google Scholar] [CrossRef]
  106. Suri, J.S.; Puvvula, A.; Biswas, M.; Majhail, M.; Saba, L.; Faa, G.; Singh, I.M.; Oberleitner, R.; Turk, M.; Chadha, P.S.; et al. COVID-19 pathways for brain and heart injury in comorbidity patients: A role of medical imaging and artificial intelligence-based COVID severity classification: A review. Comput. Biol. Med. 2020, 124, 103960. [Google Scholar] [CrossRef]
Figure 1. Global system (under the class of AtheroEdge™ 3.0) of carotid plaque segmentation.
Figure 1. Global system (under the class of AtheroEdge™ 3.0) of carotid plaque segmentation.
Jcdd 09 00326 g001
Figure 2. Basic Unet model with four encoder and decoder stages.
Figure 2. Basic Unet model with four encoder and decoder stages.
Jcdd 09 00326 g002
Figure 3. Unet++ model with four encoder and decoder stages.
Figure 3. Unet++ model with four encoder and decoder stages.
Jcdd 09 00326 g003
Figure 4. Unet3P model with four encoder and decoder stages.
Figure 4. Unet3P model with four encoder and decoder stages.
Jcdd 09 00326 g004
Figure 5. Attention block with input features, gating signals and weighted output.
Figure 5. Attention block with input features, gating signals and weighted output.
Jcdd 09 00326 g005
Figure 6. The attention gate mechanism between the first encoder and decoder layer.
Figure 6. The attention gate mechanism between the first encoder and decoder layer.
Jcdd 09 00326 g006
Figure 7. Four-stage Attention-UNet model.
Figure 7. Four-stage Attention-UNet model.
Jcdd 09 00326 g007
Figure 8. Visual results of the segmentation of Japanese, Hong Kong and UK (ICA) databases, performed by UNet, UNet++, UNet3P, and Attention-UNet models.
Figure 8. Visual results of the segmentation of Japanese, Hong Kong and UK (ICA) databases, performed by UNet, UNet++, UNet3P, and Attention-UNet models.
Jcdd 09 00326 g008
Figure 9. Attention channel effect on plaque segmentation on critical images of moderate and high plaque.
Figure 9. Attention channel effect on plaque segmentation on critical images of moderate and high plaque.
Jcdd 09 00326 g009
Figure 10. Regression plots for all 6 types of UNet models for ICA DB1 databases.
Figure 10. Regression plots for all 6 types of UNet models for ICA DB1 databases.
Jcdd 09 00326 g010
Figure 11. Regression Plot for all 6 types of UNet models for CCA DB2A database.
Figure 11. Regression Plot for all 6 types of UNet models for CCA DB2A database.
Jcdd 09 00326 g011
Figure 12. ROC curves for all 6 types of UNet model for ICA DB1 database.
Figure 12. ROC curves for all 6 types of UNet model for ICA DB1 database.
Jcdd 09 00326 g012
Figure 13. ROC Plot for all 6 types of UNet models for CCA DB2A database.
Figure 13. ROC Plot for all 6 types of UNet models for CCA DB2A database.
Jcdd 09 00326 g013
Figure 14. Paired samples t-test Plot for all 6 types of UNet models for ICA DB1 databases.
Figure 14. Paired samples t-test Plot for all 6 types of UNet models for ICA DB1 databases.
Jcdd 09 00326 g014
Figure 15. Paired samples t-test Plot for all 6 types of UNet models for CCA DB2A database.
Figure 15. Paired samples t-test Plot for all 6 types of UNet models for CCA DB2A database.
Jcdd 09 00326 g015
Figure 16. Bland–Altman Plot of all 6 types of models for ICA DB1 database.
Figure 16. Bland–Altman Plot of all 6 types of models for ICA DB1 database.
Jcdd 09 00326 g016
Figure 17. Bland–Altman Plot of all 6 types of models for CCA DB2A database.
Figure 17. Bland–Altman Plot of all 6 types of models for CCA DB2A database.
Jcdd 09 00326 g017
Table 1. Baseline characteristics of CCA cohort.
Table 1. Baseline characteristics of CCA cohort.
Risk Type-High-RiskLow-Risk-
PA Threshold-GTPA ≥ 40 mm2GTPA < 40 mm2-
NTotal (379)48331-
Gender 293(M)86(F)37(M)11(F)256(M)75(F)
Age68.7810.8869.6910.7168.6410.91 0.54
SBP134.628.88137.17.133134.39.061 0.04
DBP87.314.4488.543.56787.134.531 0.04
HbA1c6.251.066.4421.1326.2211.052 0.18
eGFR45.6720.4343.7723.3545.9519.99 0.49
LDL100.8431.48101.629.01100.731.87 0.87
HDL50.6014.7347.6312.6351.0314.98 0.14
TC174.2736.55174.936.01174.236.68 0.9
HT277-41-236- <0.0001
Smoking151-14-137- <0.0001
FH47-7-40- <0.0001
Table 2. Number of trainable parameters in different parts of the UNet architectures.
Table 2. Number of trainable parameters in different parts of the UNet architectures.
ModelEncoder ArmBottleneck LayerDecoder ArmIntermediate StagesFC Classification LayerTotal Parameters
FC = fully connected layer.
Table 3. Segmentation performance (in %) of all UNet models using ICA database DB1.
Table 3. Segmentation performance (in %) of all UNet models using ICA database DB1.
UNet98.58 ± 0.6187.43 ± 5.4599.48 ± 0.4093.11 ± 4.6189.39 ± 3.6390.02 ± 3.5382.03 ± 5.66
UNet++98.51 ± 0.6585.72 ± 6.5699.53 ± 0.3893.70 ± 4.5588.74 ± 4.1489.32 ± 4.1680.94 ± 6.45
UNet3P98.51 ± 0.6686.73 ± 6.3599.47 ± 0.4192.90 ± 4.8488.87 ± 3.9689.49 ± 3.9081.19 ± 6.16
Fractal-UNet98.41 ± 0.7485.68 ± 6.7199.45 ± 0.4392.61 ± 5.3588.13 ± 4.5888.78 ± 4.5080.10 ± 6.92
Squeeze-UNet98.53 ± 0.6386.29 ± 6.0199.51 ± 0.3993.40 ± 4.9588.91 ± 4.0989.52 ± 4.0181.25 ± 6.30
Attention-UNet98.58 ± 0.5986.86 ± 5.7399.52 ± 0.3893.54 ± 4.6289.31 ± 3.7689.90 ± 3.6981.86 ± 5.90
Table 4. Segmentation performance (in %) of all UNet models using CCA database DB2X.
Table 4. Segmentation performance (in %) of all UNet models using CCA database DB2X.
UNet98.97 ± 0.5381.58 ± 8.5899.72 ± 0.2792.20 ± 7.3985.97 ± 5.7786.07 ± 6.1776.01 ± 8.59
UNet++99.05 ± 0.3885.82 ± 7.2199.62 ± 0.3190.04 ± 8.0987.23 ± 5.4387.48 ± 5.5778.14 ± 8.01
UNet3P98.93 ± 0.5680.39 ± 9.0999.73 ± 0.2692.42 ± 7.2585.43 ± 6.2485.49 ± 6.6875.18 ± 9.00
Fractal-UNet98.91 ± 0.5480.08 ± 8.6799.72 ± 0.2785.17 ± 6.5691.96 ± 7.6185.06 ± 6.2974.68 ± 8.92
Squeeze-UNet98.90 ± 0.6379.67 ± 10.4599.73 ± 0.2784.94 ± 7.5692.33 ± 7.0984.94 ± 7.6074.48 ± 9.98
Attention-UNet99.01 ± 0.4281.75 ± 7.9499.74 ± 0.2692.71 ± 7.2086.38 ± 5.7086.50 ± 5.9476.65 ± 8.36
Table 5. Performance parameters of all UNet models for ICA and CCA database experiments.
Table 5. Performance parameters of all UNet models for ICA and CCA database experiments.
UNet0.990.99p < 0.0010.930.964p < 0.001
UNet++0.980.988p < 0.0010.960.966p < 0.001
UNet3P0.980.988p < 0.0010.920.965p < 0.001
Fractal-UNet0.960.962p < 0.0010.940.959p < 0.001
Squeeze-UNet0.960.969p < 0.0010.890.956p < 0.001
Attention-UNet0.990.988p < 0.0010.960.97p < 0.001
Table 6. Paired-t-test metrics for GTPA and estimated area by all UNet models for ICA database.
Table 6. Paired-t-test metrics for GTPA and estimated area by all UNet models for ICA database.
ModelsMean ± SDStd Error of MeanMean DifferenceSD of DifferencesStd Error of Mean Difference95% CITest Statistic-tp-Value
UNet44.5496 ± 24.21300.7774−2.94494.57160.1468−3.2330 to −2.6568−20.063<0.0001
UNet++43.5908 ± 24.23860.7783−3.90375.07390.1629−4.2234 to −3.5840−23.962<0.0001
UNet3P44.2960 ± 24.13130.7748−3.19855.30200.1702−3.5326 to −2.8644−18.789<0.0001
Fractal-UNet43.8306 ± 23.69770.7609−3.66395.43490.1745−4.0063 to −3.3314−20.996<0.0001
Squeeze-UNet43.8814 ± 23.98530.7701−361314.79150.1538−3.9150 to −3.3112−23.485<0.0001
Attention-UNet44.1855 ± 24.34270.7816−3.30904.55700.1463−3.5961 to −3.0219−22.615<0.0001
GTPA: Mean = 47.4945 mm2; SD = 25.8415 mm2.
Table 7. Paired-t-test metrics for GTPA and estimated area by all UNet models for CCA database.
Table 7. Paired-t-test metrics for GTPA and estimated area by all UNet models for CCA database.
Mean ± SDStd Error of MeanMean DifferenceSD of DifferencesStd Error of Mean Difference95% CITest Statistic-tp-Value <
UNet34.0097 ± 15.87750.6093−4.58587.51370.2883−5.1520 to −4.0197−15.904 0.0001
UNet++36.6595 ± 18.27810.7014−1.93615.41770.2079−2.3443 to −1.5278−9.3120.0001
UNet3P33.4363 ± 15.68830.6031−5.15927.92470.3041−5.7564 to −4.5621−16.9640.0001
Fractal-UNet33.5501 ± 16.10220.6179−5.04547.14210.2741−5.5836 to −4.5073−18.4080.0001
Squeeze-UNet33.1997 ± 15.55890.5971−5.39589.09530.349−6.0811 to −4.7105−15.4590.0001
Attention-UNet34.0731 ± 17.80050.6831−4.52245.30390.2035−4.9221 to −4.1228−22.2180.0001
GTPA: Mean = 38.5955 mm2 SD = 19.4775 mm2.
Table 8. Benchmarking current research against previous studies.
Table 8. Benchmarking current research against previous studies.
AuthorsArtery SegmentDL Model#Patients/#ImagesResultsBias Identified
Zhou et al. [66]ICA, CCAUNet++N1 = 144/510
N2 = 497/638
TPA error: 5.55 ± 4.34 mm2 Data selection, model selection, validation bias
Jain et al. [68]ICAUNet, UNet+, SegNet, SegNet-UNet, SegNet-UNet+,N = 97/970PA error 3.49 mm2 for SDL; 4.21 mm2 for HDL Data selection bias
Jain et al. [78]CCAUNetN1 = 379
N2 = 300
FoM of 70.96 and 91.14 (unseen) against 97.57, 88.89, and 99.14 (seen)Validation bias
Jain et al. [73]CCAUNet, SegNet-UNet, AtheroEdge 2.0N1 = 379PA error
HDL = 8 mm2
SDL = 9.9 mm2
AtheroEdge 2.0 = 9.6 mm2
Data selection bias, racial bias
Proposed methodICA, CCAUNet, UNet++, UNet3P, Attention-UNetN1 = 970;
N2 = 379;
N3 = 300
CC: 0.99 and 0.96 for ICA and CCA experimentsFree from data selection, racial, and validation biases
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Jain, P.K.; Dubey, A.; Saba, L.; Khanna, N.N.; Laird, J.R.; Nicolaides, A.; Fouda, M.M.; Suri, J.S.; Sharma, N. Attention-Based UNet Deep Learning Model for Plaque Segmentation in Carotid Ultrasound for Stroke Risk Stratification: An Artificial Intelligence Paradigm. J. Cardiovasc. Dev. Dis. 2022, 9, 326.

AMA Style

Jain PK, Dubey A, Saba L, Khanna NN, Laird JR, Nicolaides A, Fouda MM, Suri JS, Sharma N. Attention-Based UNet Deep Learning Model for Plaque Segmentation in Carotid Ultrasound for Stroke Risk Stratification: An Artificial Intelligence Paradigm. Journal of Cardiovascular Development and Disease. 2022; 9(10):326.

Chicago/Turabian Style

Jain, Pankaj K., Abhishek Dubey, Luca Saba, Narender N. Khanna, John R. Laird, Andrew Nicolaides, Mostafa M. Fouda, Jasjit S. Suri, and Neeraj Sharma. 2022. "Attention-Based UNet Deep Learning Model for Plaque Segmentation in Carotid Ultrasound for Stroke Risk Stratification: An Artificial Intelligence Paradigm" Journal of Cardiovascular Development and Disease 9, no. 10: 326.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop