Next Article in Journal
Diagnostic Role of 18F-PSMA-1007 PET/CT in Prostate Cancer Staging: A Systematic Review
Next Article in Special Issue
Minimally Invasive Intracerebral Hemorrhage Evacuation Techniques: A Review
Previous Article in Journal
Development of a Solid and Flexible Matching Medium for Microwave Medical Diagnostic Systems
Previous Article in Special Issue
Effect of Aging, Gender and Sensory Stimulation of TRPV1 Receptors with Capsaicin on Spontaneous Swallowing Frequency in Patients with Oropharyngeal Dysphagia: A Proof-of-Concept Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 11
Issue 3
10.3390/diagnostics11030551
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Machine Learning Quantitation of Cardiovascular and Cerebrovascular Disease: A Systematic Review of Clinical Applications

by
Chris Boyd
1,2,*,
Greg Brown
1,
Timothy Kleinig
3,4,
Joseph Dawson
5,6,
Mark D. McDonnell
7,
Mark Jenkinson
8 and
Eva Bezak
9,10
1
Allied Health and Human Performance, University of South Australia, SA 5000, Australia
2
South Australia Medical Imaging, Adelaide, SA 5000, Australia
3
Department of Neurology, Royal Adelaide Hospital, Adelaide, SA 5000, Australia
4
Adelaide Medical School, The University of Adelaide, Adelaide, SA 5000, Australia
5
Discipline of Surgery, University of Adelaide, Adelaide, SA 5000, Australia
6
Department of Vascular & Endovascular Surgery, Royal Adelaide Hospital, Adelaide, SA 5000, Australia
7
Computational Learning Systems Laboratory, UniSA STEM, University of South Australia, Mawson Lakes, SA 5095, Australia
8
Oxford Centre for Functional MRI of the Brain (FMRIB), Wellcome Centre for Integrative Neuroimaging, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX3 9DU, UK
9
Cancer Research Institute, University of South Australia, Adelaide, SA 5001, Australia
10
Department of Physics, University of Adelaide, Adelaide, SA 5000, Australia
*
Author to whom correspondence should be addressed.
Diagnostics 2021, 11(3), 551; https://doi.org/10.3390/diagnostics11030551
Submission received: 30 January 2021 / Revised: 12 March 2021 / Accepted: 16 March 2021 / Published: 19 March 2021
(This article belongs to the Special Issue Diagnosis and Management of Ischemic and Hemorrhagic Stroke)
Browse Figure
Versions Notes

Abstract

:
Research into machine learning (ML) for clinical vascular analysis, such as those useful for stroke and coronary artery disease, varies greatly between imaging modalities and vascular regions. Limited accessibility to large diverse patient imaging datasets, as well as a lack of transparency in specific methods, are obstacles to further development. This paper reviews the current status of quantitative vascular ML, identifying advantages and disadvantages common to all imaging modalities. Literature from the past 8 years was systematically collected from MEDLINE® and Scopus database searches in January 2021. Papers satisfying all search criteria, including a minimum of 50 patients, were further analysed and extracted of relevant data, for a total of 47 publications. Current ML image segmentation, disease risk prediction, and pathology quantitation methods have shown sensitivities and specificities over 70%, compared to expert manual analysis or invasive quantitation. Despite this, inconsistencies in methodology and the reporting of results have prevented inter-model comparison, impeding the identification of approaches with the greatest potential. The clinical potential of this technology has been well demonstrated in Computed Tomography of coronary artery disease, but remains practically limited in other modalities and body regions, particularly due to a lack of routine invasive reference measurements and patient datasets.

1. Introduction

As the first and second leading causes of global mortality, ischemic heart disease and stroke demonstrate the need for improved tools in the management of occlusive vascular disease [1]. In spite of the global incidence of both being on the decline, regional trends vary, and the total number of persons affected continues to rise due to population growth [2,3]. Patients with cardiovascular disease leading to stroke and myocardial infarction often require significant medical imaging in the acute, sub-acute, and chronic settings, using a range of imaging modalities. Vascular imaging is then used as a key source of information in the determination of appropriate clinical management, from a range of potential pharmacological and surgical approaches [4,5,6]. The large datasets obtained from this imaging are traditionally interpreted qualitatively by clinicians and are highly heterogeneous, varying due to differences in patient, imaging technology, and site scanning protocols.
Artificial Intelligence (AI) is a broad field usually characterised by two key commonalities; the design of machines to mimic human cognition, and the design of machines to complete a task whilst optimising the outcome [7]. The potential applications of machine learning (ML) to medical imaging is an emergent field, attracting growing research investment [8]. Patients suspected of cardiovascular and neurovascular diseases undergo multiple medical imaging procedures, which generates large amounts of data used in conjunction with conventional medical datasets such as patient records. Whilst these records are meticulously maintained for each individual patient, pooling this data into multi-site collaborative databases will bolster the development of ML tools and automated ML analysis. Within medicine, ML has emerged as a prominent approach for automated diagnosis and image segmentation; for detailed background of ML theory and associated medical applications, readers are directed to [9,10].
Selecting the appropriate algorithm from a great many possibilities, with each having inherent strengths and weaknesses, is a core component in development of any ML technology. Discussion of algorithm selection, technical explanation of algorithm function, or explanation of supporting mathematics are beyond the scope of this work. ML, as the currently preferred approach for analysing medical imaging datasets, refers to algorithms used to build a model capable of identifying correlations between data features. The correlations in data features are identified by “seen” input data, before then being applied to previously unseen data to perform predictions.
This seen input data can be provided in two forms, supervised and unsupervised, with each using different classes of algorithms for model development. Supervised machine learning uses input data labelled by a relevant domain expert, such as a medical specialist. These labels may be in the form of sematic segmentation, labelling at a voxel level by contouring structures from an image, or classify at an image level with labels singularly classifying the image as a whole. These labels could identify the presence of pathology in an image, or sort images based on disease stage or sub-type [10]. Common supervised ML algorithms include Support Vector Machines (SVM), k-nearest neighbour, deep neural networks, and random forest [11]. In unsupervised machine learning algorithms such as fuzzy C-means, experts do not provide data labels, instead allowing the algorithm to determine classifications independently [9,10]. Although medical imaging data can be acquired using a variety of modalities or protocols and is notoriously heterogeneous, datasets can be considered as decimal arrays with two or more dimensions. Each pixel or voxel then represents levels of grey between 0 and the maximum bit-depth of the image. These matrices of grey level values are the input data which ML algorithms use for their model development.
The appearance, volume, and variability of data made available to researchers, as well as the clinical condition being investigated, all determine which machine learning algorithm and imaging modality are employed. Invasive Coronary Angiography (ICA), Computed Tomography Angiography (CTA), two-dimensional ltrasound (US), Intravascular ultraSound (IVUS), Magnetic Resonance Imaging (MRI), Optical Coherence Tomography (OCT), Invasive Coronary Angiography (ICA), and Nuclear Medicine (NM) each play a role in diagnostic and therapeutic vascular imaging, in the process generating datasets suitable for ML analysis.
Previous reviews of medical imaging ML applications have provided a broad inspection of literature across all diseases [12,13,14] and covered narrow topics in great detail, such as a single imaging modality [15,16], task [17], vascular location (overwhelmingly cardiac vessels [18,19,20,21,22]), or have covered a specific combination of all three [23,24,25].
Imaging of vasculature following myocardial infarction or stroke is routinely performed using a range of imaging techniques, as each provides unique information. The currently published research in clinically useful vascular ML demonstrates different stages of development, varying with both vascular region and imaging modality. For example, US and IVUS ML research has a wealth of carotid publications, but a comparative dearth of coronary papers. Conversely, coronary CTA products have been commercialised for routine clinical use, whilst preliminary studies into non-coronary CTA products are lacking or absent. Both cardiovascular and neurovascular diseases have large disease burdens, especially in developed countries [26], but current ML research is not proportionate, and coronary applications are far more advanced. This is due to many factors, including coronary disease being the global leading cause of death and recent advances in the hardware and software used for coronary CTA, improving both image quality and acquisition speed. These techniques have the potential to assist in clinical management and reduce disease burdens, but the highly heterogeneous state of current literature makes the identification of future areas difficult. Review, synthesis, and critical analysis of ML approaches from a wide range of modalities and vascular locations is needed to identify commonalities and gaps in research. With this, researchers are better placed to ensure future work focuses on relevant clinical needs and more effectively translate into clinical practice, improving patient care.
This work reviews the current status of combined knowledge for ML based quantitation of both coronary and neurological vascular disease. The current effectiveness of ML to provide quantitative descriptors of vessel disease will be examined, as well as the impact of this quantitation on clinical decision making and patient outcomes, for a wide range of medical imaging modalities.

2. Materials and Methods

A systematic review was conducted in accordance with guidance included in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [27].

2.1. Data Sources

A computerised literature search of Ovid MEDLINE and Elsevier SCOPUS was performed to find full text, original articles published in the eight years prior to 29 January 2021 in medicine, engineering, or computing, which investigated ML analysis of human vascular imaging. The search terms were combined as follows: (ALL (“Machine Learning” OR “Neural Network” OR “Support Vector” OR “Random Forest” OR “Bayesian Network” OR “Nearest Neighbor”)) AND (ALL (plaque OR calcification OR ulceration OR stenosis))) AND (ALL (quantif* OR quantitati*))) AND NOT (ALL (spectroscop* OR alzheimer OR “tau” OR “amyloid” OR “lung” OR “multiple sclerosis”)).

2.2. Data Extraction and Quality Assessment

Studies satisfying these criteria underwent assessment of title and abstract by two authors (C.B and E.B), with equivocal papers additionally reviewed by a third (G.B). This included analysis of abstract and titles which indicated quantitative machine learning analysis of human vascular imaging, with pertinent studies continuing to full text review. Using a standardised data extraction form, relevant information was collected, including (a) participant characteristics (patient numbers, patient age, age range, and sex), (b) data characteristics (number of images, imaging modality, and disease site), (c) AI characteristics (algorithm, algorithm parameters, quantitation results, and model validation/cross-validation), and (d) per patient outcomes (gold standard, sensitivity/specificity/accuracy, and Area Under the Curve (AUC)).

3. Results

3.1. Literature Search

The detailed literature selection process is shown in Figure 1. The systematic search identified 1098 articles, with most records deemed outside the area of interest based on title and abstract review. Although several studies quoted large datasets, some were referencing total number of pixels [28], Region-Of-Interest (ROI) subsets extracted from small image numbers [29], or number of images extracted from a small number of patients [30,31,32,33]. To manage any potential bias and avoid inclusion of overfit or low variability datasets, articles where total unique patient numbers were less than or equal to 50 were also excluded. Seven articles failed to identify the number of patient datasets used and were also excluded. The reference lists of the remaining 42 papers were then pearled, and an additional 5 papers were included, for a total of 47.
As previously mentioned, the appropriate ML techniques depend on both the type and quantity of available data, as well as the ML researchers’ experience, skill, and currency of knowledge. Experienced ML researchers consider these and other practical factors in determining the most suitable algorithm, although algorithm choice is not unique, and several algorithms may be suitable for one data type, or multiple data types may be analysed successfully with one algorithm. The literature identified was sorted by imaging modality and algorithm selection, before extracting and tabulating data relevant for the development of an integrative overview of automated ML vascular quantitation. Vascular pathology quantification was most often performed using CTA (27) and ultrasound (6) imaging, as well as several publications using other modalities (9). All but one IVUS study [34] was excluded due to insufficient patient numbers [28,35,36,37,38,39,40,41], highlighting the need for more large-scale research in this modality. A few studies attempting vascular quantitation from Nuclear Magnetic Resonance (NMR) [42] and Nuclear Medicine (NM) [43,44,45] imaging were also included, as well as one publication using ICA [46].

3.2. Modality Specific Vascular Imaging

3.2.1. Computed Tomography Angiography

CT for vascular imaging has increased in utility due to consistent improvement in CT technology and is now the most prevalent imaging modality for quantitative ML analysis, spurred on by recent research into the low diagnostic yield of ICA [47]. Technological advancements allowing increased tube rotation speeds, reducing motion blur, smaller detector element sizes increasing spatial resolution, and the clinical implementation of advanced filtration and reconstruction techniques, have all improved visualisation of vasculature. A summary of CTA quantitative analysis is shown in Table 1.
ML vascular analysis from CTA imaging is a well-established field [48,49], with assessment of ML and CTA-based calculation of Fractional Flow Reserve (FFR) published by multiple sites [50,51,52,53,54,55,56,57] and now explored by several comprehensive reviews [58,59,60]. FFR is defined as the ratio of blood flow, proximally and distally to a coronary vascular lesion, measured under pharmacologically maximised coronary blood flow [61]. Han et al. [62] performed CT-based FFR quantification using resting perfusion CT, instead of the more widely used CTA, obtaining sensitivity, specificity, and accuracy values shown in Table 2.
Siemens® Syngo cFFR (Erlangen, Germany) software was widely used across the literature, in various development stages, for the quantitation of FFR, and results are summarised in Table 3. This technology, based on coronary deep neural network research published previously [63], was used by both [53] and Yu et al. [57] to quantify FFR as part of a wider evaluation of lesion specific ischemia and functional significance. Both authors found a limited but measurable benefit of including CT-based FFR in plaque analysis. The resulting Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) measured by Coenen et al. [64], Hu et al. [51], and Yu et al. [54] for CT-based FFR were all similar to that quoted above, with all using invasive FFR measurements as a benchmark. FFR clinical significance was typically based on a threshold of 0.8 and equivocality around this value (termed the ‘grey zone’ [57]) was observed for all measurement techniques, although to varying degrees [54].
Various other aspects of vascular health were quantitatively analysed using machine learning, based on CT imaging datasets. In these instances, hold-out [65,66,67,68] and k-fold cross validation [50,65,67], two ML validation techniques, were employed to improve algorithm performance, with both used in conjunction where possible to minimise bias. Quantification of individual plaque risk and Major Adverse Cardiovascular Events (MACE) risk stratification was examined in several papers, both with [53,69] and without [65,67,70] CTA-derived FFR.
Specific data properties used in ML analysis, often termed ‘features’, contribute unequally to the overall model performance and the inclusion of features should be done judiciously [71]. In vascular analysis, features such as remodelling index, plaque length, mean lumen diameter, mean luminal area, and napkin ring sign showed some utility as discriminating features, whilst others such as calcified plaque burden and spotty calcification had near zero information gain, meaning they were unlikely to provide any predictive power to model performance [50,53,69,70,72]. Several equivocal image features were identified as both strongly and weakly predictive by different publications, and these included total plaque volume, non-calcified plaque volume, and Agatston score (a score based on the maximal Hounsfield Unit (HU) value observed in CT imaging of coronary artery calcifications) [50,53,69,72]. This equivocality suggests feature selection should be performed systematically for each research work individually, due to variabilities in available data and project outcomes. Generally, ranking of features by information gain was consistent for CTA imaging, with quantitative image parameters contributing greater to model function than qualitative imaging parameters or clinical/laboratory values [71]. The automated and semi-automated determination of Agatston score has also been investigated separately [68,73,74,75,76,77], although some publications utilise thresholding and individual/direct image analysis approaches, rather than ML.
Although useful, plaque quantitation is not the only informative quantitative metric in cardiovascular risk management. Motwani et al. [78], with access to one of the largest CTA imaging datasets in current vascular ML research, compared existing coronary risk stratification quantifiers such as Agatston score and Framingham risk score with a ML LogitBoost model for quantification of five-year All-Cause Mortality (ACM). Analysis of 44 imaging features and 25 clinical features found a statistically significant increase in the AUC of all-cause mortality risk, compared to Framingham risk score alone (0.79 v 0.61), although no validation cohort was investigated. Similar ML performance was observed in the ACM risk quantification of an 86,155 strong cohort of Korean patients [79], with the AUC values of ML and Agatston score mortality risk of 0.82 and 0.70, respectively. Despite the large dataset, performance in a validation cohort was not found to be statistically significant (AUC: 0.78 v 0.62), possibly suggestive of model overfitting in spite of large sample sizes.
Zreik et al. [80] analysed clinical plaque significance using a convolutional neural network developed in-house, investigating only left ventricular myocardium CT images with clusters of pixels belonging to myocardium identified by fast K-means clustering. This method contrasted to the direct vessel imaging approached employed widely throughout the literature across all modalities. The model development utilised 50% random dropout and 10-fold cross validation to minimise overfitting, and plaque significance was benchmarked against invasive FFR. Multiple FFR thresholds were investigated for their impact on the determination of individual lesion clinical significance, ranging from 0.72 to 0.8. Technical parameters were similarly varied, using multiple fast K-means cluster values (1–1000) for division of the myocardium and several convolutional auto-encoders. The mean AUC across the 50 cross-validations was 0.74 ± 0.02, with the majority of algorithm and clinical setting combinations returning AUC values between 0.65 and 0.75. In a continuation of this work, van Hamersvelt et al. [81] applied the analysis method developed previously to intermediary stenosis (defined as 25–69% stenosis as assessed by invasive coronary angiography), noting improved sensitivity and AUC with a slight decrease in specificity.
ML-based coronary vessel FFR quantitation using CTA datasets is rapidly becoming established as a clinical methodology, although as with all clinical practices, critical evaluation remains ongoing. Two elements of FFR quantitation subject to this critical evaluation were the significance of X-ray tube peak kilovoltage (kVp) dependencies undertaken by De Geer et al. [82], and the role of partial volume effects by Freiman et al. [83]. De Geer et al. [82] concluded no impact in FFR quantitation between 100 and 120 kVp and slightly better agreement between ML and invasive FFR quantitation at 100 kVp. The large pixels of CT imaging, in comparison to ultrasound and fluoroscopy, can result in partial volume effects in which assigned pixel values represent a mean of values from multiple structures contained within. Consideration of the role these partial volume effects play in FFR were found to improve both the specificity (0.51 to 0.73) and AUC (0.76 to 0.8) of FFR ML quantitation as compared with angiographically determined values.
Work in quantitative machine learning cerebrovascular analysis was extremely limited. No clear practical or technical reason could be found to sufficiently explain why research in this area is so sparse, particularly given the comparative examination throughput and vessel lumen diameter of coronary vessels and many cerebral vessels of clinical significance. The absence of a routinely performed invasive quantitation comparison methods and no collaborative multi-centre databases appear to be two possible reasons, with just two groups attempting to investigate this area [66,84]. The work of Park et al. [66] investigated aneurysm detection by expert operator from CT imaging, with and without the assistance of a segmentation deep neural network. An incremental but statistically significant increase in reader sensitivity, accuracy, and agreement was found. Deep learning approaches to neurological CTA were noted by the authors as absent from current literature prior to their investigation, suggesting future development may provide clinically useful results warranting further research. A preliminary pilot study by Acharya et al. [84] into carotid lumen segmentation and pathology quantification from CT data produced sensitivities, specificities, and accuracies of 0.88, 0.865, and 0.902, respectively, using an SVM classifier with radial basis functions. Statistically significant high-level features (p < 0.01) were identified for the differentiation of symptomatic and asymptomatic plaques, in particular higher energy and lower entropy in symptomatic images due to increased texture complexity. The generalisation of these results was limited, however, as only 20 consecutively sampled patients were used.

3.2.2. Ultrasound

The excellent soft tissue resolution and sub-millimetre spatial resolution of ultrasound is well suited to imaging of vascular detail, readily applicable to the large, superficial, and accessible common carotid artery bifurcation. As shown in Table 4 in contrast with CTA investigations, cardiac studies were comparatively few, with external ultrasound ill-suited to coronary vasculature investigation. Study sample sizes often included less than 50 patients and were too small for generalisation, particularly for highly invasive IVUS studies [28,36,39,40], although recently published works have attempted to address this [34].
Research examining segmentation of vascular anatomy and pathology using ultrasound is extensive, with publications reaching as back as far as 2000 [85,86], although advancements enabling automated quantification are comparatively few. The development of ML tools capable of identifying at-risk asymptomatic carotid disease and providing decision support is the primary focus of current ultrasound research. Bae et al. [34] developed and compared the performance of multiple ML methods for identification of vulnerable coronary plaques from IVUS investigations, as defined by the presence of a fibrous thin cap atheroma. Results were compared with OCT, the highest resolution vascular imaging currently clinically available. The implementation of tools such as these would support the management of neurovascular and cardiovascular diseases, although to date ultrasound is not widely employed for initial diagnosis in acute stroke [87,88,89].
As expected from the acquisition technology employed, US examinations offered an alternative set of image parameters compared to CTA, with some having utility as machine learning features. The most consistent of these features was the measurement of carotid Intima Media Thickness (IMT).
IMT is below the spatial resolution of CTA and cannot be visualized, but has been correlated to increased vascular risk [90]. This makes US a uniquely practical tool for measurement of IMT thickness. Segmentation of IMT has been performed using several US imaging parameters such as the Hough transform [91], frequency domain analysis [92], and pixel intensity analysis, the latter using a deep learning single-layer feed-forward neural network. This method achieved a sensitivity and specificity greater than 97% (shown in Table 5) when compared to expert manual segmentation [33].
Another analysis utilising image features unique to ultrasound was evaluation of the discrete Fréchet distances of greyscale cumulative distribution functions, as compared to idealised functions, from Huang et al. [93]. This greyscale distribution analysis for individual plaques in combination with a k-nearest neighbour classification system sorted plaques into echo-rich, intermediate, and echo-lucent, with association between echo type and plaque vulnerability established elsewhere [94,95].
The correlation of echogenicity and carotid plaque vulnerability was also utilised by Pedro et al. [96], Roy-Cardinal et al. [97] and Golemati et al. [98], although the former used a simplistic ROC cut-off analysis to produce a semi-quantitative plaque vulnerability indicator. Echogenicity was used in conjunction with both general greyscale image features such as Rayleigh parameters or grey level co-occurrence matrix (GLCM) decomposition and features more specific to ultrasound imaging such as homodyned-K parametric mapping, wavelet energy decomposition, or elastography. The two other papers, however, provided comprehensive examination of both symptomatic and asymptomatic carotid stenosis, detailing machine learning techniques, patient descriptors, and clinical outcomes. Using a range of extracted image parameters, US elastography, and plaque motion synchronisation, the composition and clinical significance of symptomatic and asymptomatic plaques were analysed with a random forest classifier. Roy-Cardinal et al. [97] also compared US plaque to composition determined by MRI, as well as comparing patient symptomology to experimental predictions, keeping results focused on improvement of clinical patient outcomes, the ultimate target of any ML model.

3.2.3. Other Imaging Modalities

Whilst US and CTA are the dominant imaging modalities, several other methods exist which are useful in the diagnosis and management of vascular disease, including MRI, NM, OCT, and ICA. Application of these modalities showed the greatest variability with ML approaches used on both direct image analysis and routinely obtained clinical descriptors. Image analysis research from these modalities used markedly smaller sample sizes, especially when compared to US and CT patient databases described above. Some reasons may include increased invasiveness and fewer centres to facilitate multi-institutional research or cost, although the exact reason remains unclear. ICA showed extensive use as a gold standard for all other imaging modalities, but ML image analysis of ICA was limited. Table 6 summarises ML vascular quantitation results performed using modalities other than CT and ultrasound.

Magnetic Resonance Imaging

Only two identified MRI publications satisfied all selection criteria. Waddle et al. [99], applied an SVM model with a fitcsvm function and radial bias function kernel to MRI, Magnetic Resonance Angiography (MRA), and functional MRI (fMRI) data of moyamoya patients. The hemispheric blood flow of patients was compared with healthy controls, and ML classification of hemispheres was performed with a resulting sensitivity, specificity, and AUC of 0.7, 0.83, and 0.71, respectively. The authors commented that ML techniques are notably underutilised in vascular imaging, despite finding their results “collectively offer increased support that both anatomical and functional hemodynamic imaging can serve as important machine learning inputs” [99].
Wu et al. [100] developed a deep convolutional neural network to investigate MRA of carotid vessels, applied to blood signal suppressed, or “black blood” images. Patient imaging was supplied from two previously collected research datasets [101,102] with carotid segmentation performed and quantitatively compared to expert manual segmentation. In addition to analysis of individual patient slices, data from the images immediately before and after each slice were considered, producing a “2.5-dimension” dataset. Carotid lesion types, as defined by the American Heart Association [103], were determined from segmented vessels and identified as atherosclerotic or non-atherosclerotic, with maximum ML accuracy and AUC of 0.89 and 0.95, respectively, in relation to expert decisions.
Four other publications utilising magnetic resonance techniques for quantitative vasculature analysis were identified, which although not satisfying inclusion criteria, warrant further discussion. These publications show future potential of MRI both for direct analysis of vessels, and the possible application of ML to information unique to MRI such as relaxation times or spectroscopic analysis.
The relationship between carotid vessel image parameters and stroke risk was investigated by Van Den Bouwhuijsen et al. [104] using logistic regression, from a large, pre-existing patient database [105]. Despite the simplicity of the approach, this method supports the importance of large datasets, associating stroke risk with intraplaque haemorrhage, carotid wall thickness, and calcification. Automated segmentation of Carotid artery plaque directly from contrast enhanced MRI has also shown promise, with a recent study [106] using 35 patients to obtain automated segmentations with a Dice score and true-positive of 0.89 and 0.93, respectively, as compared to manual analysis.
Van Hespen et al. [107] used sub-millimetre isotropic voxels to image ex vivo circle of Willis specimens and train a convolutional neural net to measure wall thicknesses for intracranial aneurysm. Although the results were promising, with less than 0.1 mm error in intracranial vessel wall estimation, a three patient validation sample size, and long acquisitions in a high field MRI (7T) places limits on any immediate clinical translation.
Forssen et al. [42] demonstrated the breadth of MRI to obtain clinically useful information when combined with machine learning, utilising supervised ML to quantify 256 metabolites associated with coronary artery disease, through Nuclear Magnetic Resonance spectroscopy.

Nuclear Medicine

The large difference in spatial resolution between CTA, US, and IVUS discussed previously is similar when comparing CTA and many common NM procedures. Instead of imaging vessels directly, Nakajima et al. [43,44,45] used approximately 2000 Technetium-99m Myocardial Perfusion Imaging (MPI) studies from Swedish and Japanese datasets to train an artificial neural network in the detection of perfusion defects and ischemia. Each model was tested on an in-house cohort of 106 patients and compared to both expert readers and a >50% stenosis gold standards, with version 1.1 of the model further validated on 364 patients. As shown in Table 7, improvement was observed in the upgraded 1.1 version ML tool, with sensitivities and specificities in excess of 87% for all patients, peaking at 88 and 100%, respectively, in patients with no history of prior infarction or coronary revascularization.
Quantitative and semi-quantitative descriptors routinely obtained during cardiac Positron Emission Tomography (PET) were also analysed using six ML algorithms and linear regression. Obstructive coronary disease status was determined from a sample of 88 patients, with performance statistics approaching 0.9 and SVM performing best.

4. Discussion

Across many imaging modalities and organ vasculatures, clinically useful information can be gained by quantitation of atherosclerotic disease and associated infarction. The task of segmenting and quantifying vascular pathology is currently both repetitive and laborious, as well as requiring specialist expertise. This combination makes the task well suited to automation by machine learning.
The ultimate focus of any research into ML vascular quantitation across all modalities and vascular territories must remain the meaningful and positive impact to patient outcomes. Although ML analysis of coronary CTA imaging has progressed furthest towards broad clinical use, accuracies in the range of 70–80% show that research remains to be done. Researchers should be buoyed by these results however, as they do demonstrate the clinical potential of quantitative vascular ML steadily becoming actualised.

4.1. Limitations and Future Work

The works reviewed in this paper achieved performance comparable to current methods and in some cases demonstrated commercialised ML vascular analysis in clinic. Despite the success of these methods, several hurdles remain before ML vascular image quantitation is ready to be applied to patient care in some contexts.

4.1.1. Common Machine Learning Limitations

Limited data availability and a lack of code accessibility (the black box reputation of ML) are limitations seen in many machine learning applications, within medicine and beyond. Although not unique to quantitative vascular machine learning, both are nonetheless important considerations in any future research.
Data management is the foundation from which all ML research is undertaken, and the comparative infancy of most current vascular ML quantitation research provides the ideal opportunity to establish standardised data sharing and result reporting approaches, which will support the development of useful clinical technology into the future. Construction of vascular imaging benchmark datasets has begun to address both the problem of transparency and data availability simultaneously. Unfortunately, current datasets are not truly publicly available (instead limited to ethically approved research trials with data available to a select group of researchers) and participant numbers insufficient in most cases for generalisation to the diversity of patients seen in many hospitals. These small datasets offer developers a starting point for useful evaluation and comparison of privately developed models, on the understanding that this database is not also used for model training. Some identified datasets include the MACHINE consortium [64], CONFIRM registry [108], and PARADIGM [109] CTA datasets, as well as the annual challenge databases of the Medical Image Computing and Computer Assisted Intervention (MICCAI) society from all modalities, all of which have been used multiple times throughout the literature [67,73,79,83,110,111]. The handling of patient imaging for use in medical machine learning is a complex issue being contended with around the world [112]. Only once a broad framework of requisite protections is in place to allow ethical data handling can large databases be constructed to allow robust model development.
Many algorithms produced by current research have been developed and evaluated using patient data that is not independently accessible and which cannot be externally validated for reproducibility. Once these models are demonstrated by researchers to provide benefit, some are then packaged into commercial offerings with little or no public details on further changes or advances [63]. Without an external or open access reference database, further research benchmarking of the performance of comparative models in a transparent and useful fashion is impossible. This issue of data availability was discussed in both Coenen et al. [64] and Cho et al. [46], in which the authors stated patient data will not be provided for the purpose of independent result validation, with no explicit justification given. Failure by ML developers to make data or model design specifics available perpetuates the black box stereotype of these tools and adversely impacts clinician confidence when considering whether to use potentially beneficial tools. Gao et al. [113] attempted to address this with the provision of detailed algorithm processes in a summarised step-by-step fashion that, although not equivalent to open access, is an interesting intermediary step to improving reproducibility. Waddle et al. [99] was the only publication identified in this review which actively supported reproducibility, explicitly stating a provision for de-identified data to be made available on request.
For CTA studies, and in particular for the assessment of FFR, the commercial availability of deep learning platforms, such as cFFR v1.0–3.0 (Siemens Healthineers, Erlangen, Germany), CAAS vFFR (Pie-Medical, Maastricht, The Netherlands), and HeartFlow® FFR-CT (HeartFlow, Redwood City, CA, USA) limited the provision of algorithm details due to commercial and intellectual property interests.

4.1.2. Reference Standards

The standardisation of cardiac analysis from CT imaging by the American Heart Foundation [114] has provided consistent segmentation nomenclature in this region. Similarly, the definition of FFR methodology [61] allowed results of works investigating FFR using CTA imaging to be compared with this standard metric [70,115]. Routine quantification of FFR for coronary CTA provides the ideal reference standard for ML based quantification from CTA imaging. Well defined coronary vascular segments, a defined methodology for FFR determination and the inclusion of quantitative invasive vascular quantitation for every patient allows method performances to be compared and approaches to be benchmarked. The routine quantitation of vascular disease in carotid vessels is currently limited to manually measured criteria such as that outlined by NASCET [116]. Although such measurements are clinically useful [117], the risk of inter and intra observer variability cannot be discounted, particularly when incorporated into large multi-nation, multi-centre databases, the likes of which will be required for robust machine learning model development.
Although not a singular reference measurement, the development and adoption of MICCAI and AAPM publications providing detailed methodologies for standardised evaluation of algorithms examining stenosis and lumen segmentation or coronary vascular quantitation supports model evaluation and comparison in a similar way [76,118,119]. Research into model benchmarking has begun, but is still in the early stages, with small patient sample sizes (n =10) and limited algorithms investigated (n = 4) [120].

4.1.3. Image Standards

Beyond the absence of invasive quantitation as a reference standard, the variation in acquisition methodology between sites also presents a challenge for generalisation from vascular machine learning. CT image reconstruction methods (filtered back projection, iterative reconstruction, and AI enhanced reconstruction), MRI settings (bandwidth, TR and TE times, matrix size), training of sonographers, and combinations of administered dose and acquisition time in nuclear medicine, all vary with institution. Image appearance preferences also vary between institutions and even reporting clinicians within one institution, with acquisition parameters changed accordingly. Such wide variability necessitates greater robustness in any clinical models to account for application of developed technologies to image appearances not previously encountered.

4.1.4. Reporting Standards

Reporting of machine learning results throughout the literature was vague and inconsistent for all modalities. Statistical metrics varied widely with combinations of sensitivity, specificity, accuracy, and AUC reported inconsistently, and compared to different gold standards. The nature of ML development and analysis further complicated inter-publication comparison with the result section of each paper quoting sensitivity, specificity, and AUC values obtained using different image feature combinations, algorithm parameter settings (such as SVM kernel), and repeating this process for multiple algorithms, resulting in large tables of statistics encompassing a wide range of values. Finally, the reporting of studies in vascular quantitation allowed results to be reported on a per-patient, per vessel, per vessel segment or per lesion basis, with some papers reporting results for several of these.

5. Conclusions

The review highlights that open access data, or a systematic and independent validation solution, are essential to the continued research growth of vascular ML quantitation. This issue was identified and best explained by Zreik et al. [121], who despite having access to a large imaging research facility, observed that “[with] a sufficiently large and diverse data set, a deeper CNN-only…analyzing a large single volume along the artery, could be employed to perform the presented analyses. However, obtaining such a large data set remains highly challenging…”. Neural networks, and convolutional neural networks especially, have been widely recognised across multiple modalities as the most promising candidates for vascular imaging ML analysis. Despite this, other articles identified by this review [99] concluded, similarly to Zreik et al. [121], that current data is insufficient for clinical CNN implementation. Although many techniques exist for maximising model performance from limited data, large quantities of unique datasets are essential to clinical model performance [122,123]. The imaging data accessible to vascular ML researchers at present is forcing the selection of algorithms to be heavily influenced by available datasets, not necessarily those that are the best performing or most informative. Once standardised datasets are made readily available, transparent and standardised reporting of results will promote collaboration and improve both the development of ML techniques and the clinical confidence in the use of the technology.

Author Contributions

Conceptualization, J.D., T.K. and E.B.; methodology, C.B. and E.B.; validation, G.B. and E.B.; investigation, C.B., G.B. and E.B.; data curation, C.B., G.B. and E.B.; writing—original draft preparation, C.B.; writing—review and editing, C.B., G.B., J.D., T.K., M.D.M., M.J. and E.B.; visualization, C.B.; supervision, G.B. and E.B.; project administration, E.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. GBD 2015 Mortality and Causes of Death Collaborators. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: A systematic analysis for the Global Burden of Disease Study 2015. Lancet 2016, 388, 1459–1544. [Google Scholar] [CrossRef] [Green Version]
  2. Nowbar, A.N.; Gitto, M.; Howard, J.P.; Francis, D.P.; Al-Lamee, R. Mortality From Ischemic Heart Disease. Circ. Cardiovasc. Qual. Outcomes 2019, 12, e005375. [Google Scholar] [CrossRef] [PubMed]
  3. Feigin, V.L.; Norrving, B.; Mensah, G.A. Global Burden of Stroke. Circ. Res. 2017, 120, 439–448. [Google Scholar] [CrossRef] [PubMed]
  4. American Stroke Association. Types of Stroke. Available online: www.strokeassociation.org/en/about-stroke/types-of-stroke (accessed on 4 December 2019).
  5. Birenbaum, D.; Bancroft, L.W.; Felsberg, G.J. Imaging in acute stroke. West. J. Emerg. Med. 2011, 12, 67–76. [Google Scholar]
  6. Flachskampf, F.A.; Schmid, M.; Rost, C.; Achenbach, S.; DeMaria, A.N. Cardiac imaging after myocardial infarction. Eur. Heart J. 2011, 32. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Russell, S.; Norvig, P. Artificial Intelligence: A Modern Approach, 3 ed.; Prentice Hall: Upper Saddle River, NJ, USA, 2010; p. 1132. [Google Scholar]
  8. Buch, V.H.; Ahmed, I.; Maruthappu, M. Artificial intelligence in medicine: Current trends and future possibilities. Br. J. Gen. Pract. 2018, 68, 143–144. [Google Scholar] [CrossRef] [Green Version]
  9. Pesapane, F.; Codari, M.; Sardanelli, F. Artificial intelligence in medical imaging: Threat or opportunity? Radiologists again at the forefront of innovation in medicine. Eur. Radiol. Exp. 2018, 2, 35. [Google Scholar] [CrossRef] [Green Version]
  10. Erickson, B.J.; Korfiatis, P.; Akkus, Z.; Kline, T.L. Machine Learning for Medical Imaging. Radiographics 2017, 37, 505–515. [Google Scholar] [CrossRef]
  11. Bishop, C.M. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2006. [Google Scholar]
  12. Shen, D.; Wu, G.; Suk, H.I. Deep Learning in Medical Image Analysis. Annu. Rev. Biomed. Eng. 2017, 19, 221–248. [Google Scholar] [CrossRef] [Green Version]
  13. Hesamian, M.H.; Jia, W.; He, X.; Kennedy, P. Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. J. Digit. Imaging 2019, 32, 582–596. [Google Scholar] [CrossRef] [Green Version]
  14. Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.W.M.; van Ginneken, B.; Sánchez, C.I. A survey on deep learning in medical image analysis. Med. Image Anal. 2017, 42, 60–88. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Faust, O.; Acharya, U.R.; Sudarshan, V.K.; Tan, R.S.; Yeong, C.H.; Molinari, F.; Ng, K.H. Computer aided diagnosis of Coronary Artery Disease, Myocardial Infarction and carotid atherosclerosis using ultrasound images: A review. Phys. Med. 2017, 33, 1–15. [Google Scholar] [CrossRef] [Green Version]
  16. Liu, S.; Wang, Y.; Yang, X.; Lei, B.; Liu, L.; Li, S.X.; Ni, D.; Wang, T. Deep Learning in Medical Ultrasound Analysis: A Review. Engineering 2019, 5, 261–275. [Google Scholar] [CrossRef]
  17. Jodas, D.S.; Pereira, A.S.; Tavares, J.M.R.S. A review of computational methods applied for identification and quantification of atherosclerotic plaques in images. Expert Syst. Appl. 2016, 46, 1–14. [Google Scholar] [CrossRef] [Green Version]
  18. Commandeur, F.; Goeller, M.; Dey, D. Cardiac CT: Technological Advances in Hardware, Software, and Machine Learning Applications. Curr. Cardiovasc. Imaging Rep. 2018, 11. [Google Scholar] [CrossRef] [PubMed]
  19. Gandhi, S.; Mosleh, W.; Shen, J.; Chow, C.M. Automation, machine learning, and artificial intelligence in echocardiography: A brave new world. Echocardiography 2018, 35, 1402–1418. [Google Scholar] [CrossRef]
  20. Krittanawong, C.; Zhang, H.; Wang, Z.; Aydar, M.; Kitai, T. Artificial Intelligence in Precision Cardiovascular Medicine. J. Am. Coll. Cardiol. 2017, 69, 2657–2664. [Google Scholar] [CrossRef]
  21. Henglin, M.; Stein, G.; Hushcha, P.V.; Snoek, J.; Wiltschko, A.B.; Cheng, S. Machine Learning Approaches in Cardiovascular Imaging. Circ. Cardiovasc. Imaging 2017, 10. [Google Scholar] [CrossRef]
  22. Opincariu, D.; Benedek, T.; Chitu, M.; Rat, N.; Benedek, I. From CT to artificial intelligence for complex assessment of plaque-associated risk. Int. J. Card. Imaging 2020, 36, 2403–2427. [Google Scholar] [CrossRef]
  23. Sharma, A.M.; Gupta, A.; Kumar, P.K.; Rajan, J.; Saba, L.; Nobutaka, I.; Laird, J.R.; Nicolades, A.; Suri, J.S. A Review on Carotid Ultrasound Atherosclerotic Tissue Characterization and Stroke Risk Stratification in Machine Learning Framework. Curr. Atheroscler. Rep. 2015, 17, 55. [Google Scholar] [CrossRef]
  24. Soun, J.E.; Chow, D.S.; Nagamine, M.; Takhtawala, R.S.; Filippi, C.G.; Yu, W.; Chang, P.D. Artificial Intelligence and Acute Stroke Imaging. AJNR Am. J. Neuroradiol. 2021, 42, 2–11. [Google Scholar] [CrossRef] [PubMed]
  25. Muscogiuri, G.; Van Assen, M.; Tesche, C.; De Cecco, C.N.; Chiesa, M.; Scafuri, S.; Guglielmo, M.; Baggiano, A.; Fusini, L.; Guaricci, A.I.; et al. Artificial Intelligence in Coronary Computed Tomography Angiography: From Anatomy to Prognosis. BioMed Res. Int. 2020, 2020, 6649410. [Google Scholar] [CrossRef]
  26. Australian Institute of Health and Welfare. Australia’s Health 2018; Australia’s Health Series no. 16; AIHW: Canberra, Australia, 2018. [Google Scholar]
  27. Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gotzsche, P.C.; Ioannidis, J.P.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med. 2009, 6, e1000100. [Google Scholar] [CrossRef] [PubMed]
  28. Uchino, E.; Kubota, R.; Koga, T.; Misawa, H.; Suetake, N. Multiple k-nearest neighbor classifier and its application to tissue characterization of coronary plaque. IEICE Trans. Inf. Syst. 2016, E99D, 1920–1927. [Google Scholar] [CrossRef] [Green Version]
  29. Menchón-Lara, R.M.; Sancho-Gómez, J.L.; Bueno-Crespo, A. Early-stage atherosclerosis detection using deep learning over carotid ultrasound images. Appl. Soft Comput. J. 2016, 49, 616–628. [Google Scholar] [CrossRef]
  30. Khowaja, S.A.; Unar, M.A.; Ismaili, I.A.; Khuwaja, P. Supervised method for blood vessel segmentation from coronary angiogram images using 7-D feature vector. Imaging Sci. J. 2016, 64, 196–203. [Google Scholar] [CrossRef]
  31. Rosati, S.; Meiburger, K.M.; Balestra, G.; Acharya, U.R.; Molinari, F. Carotid wall measurement and assessment based on pixel-based and local texture descriptors. J. Mech. Med. Biol. 2016, 16. [Google Scholar] [CrossRef]
  32. Jawaid, M.M.; Narejo, S.; Pirzada, N.; Baloch, J.; Reyes-Aldasoro, C.C.; Slabaugh, G. Automated Quantification of Non-Calcified Coronary Plaques in Cardiac CT Angiographic Imagery. Int. J. Adv. Comput. Sc. 2018, 9, 216–222. [Google Scholar] [CrossRef]
  33. Hassan, M.; Chaudhry, A.; Khan, A.; Iftikhar, M.A. Robust information gain based fuzzy c-means clustering and classification of carotid artery ultrasound images. Comput. Methods Programs Biomed. 2014, 113, 593–609. [Google Scholar] [CrossRef]
  34. Bae, Y.; Kang, S.J.; Kim, G.; Lee, J.G.; Min, H.S.; Cho, H.; Kang, D.Y.; Lee, P.H.; Ahn, J.M.; Park, D.W.; et al. Prediction of coronary thin-cap fibroatheroma by intravascular ultrasound-based machine learning. Atherosclerosis 2019, 288, 168–174. [Google Scholar] [CrossRef] [PubMed]
  35. Araki, T.; Ikeda, N.; Shukla, D.; Londhe, N.D.; Shrivastava, V.K.; Banchhor, S.K.; Saba, L.; Nicolaides, A.; Shafique, S.; Laird, J.R.; et al. A new method for IVUS-based coronary artery disease risk stratification: A link between coronary & carotid ultrasound plaque burdens. Comput. Methods Programs Biomed. 2016, 124, 161–179. [Google Scholar] [CrossRef]
  36. Araki, T.; Ikeda, N.; Shukla, D.; Jain, P.K.; Londhe, N.D.; Shrivastava, V.K.; Banchhor, S.K.; Saba, L.; Nicolaides, A.; Shafique, S.; et al. PCA-based polling strategy in machine learning framework for coronary artery disease risk assessment in intravascular ultrasound: A link between carotid and coronary grayscale plaque morphology. Comput. Methods Programs Biomed. 2016, 128, 137–158. [Google Scholar] [CrossRef] [PubMed]
  37. Banchhor, S.K.; Londhe, N.D.; Araki, T.; Saba, L.; Radeva, P.; Laird, J.R.; Suri, J.S. Wall-based measurement features provides an improved IVUS coronary artery risk assessment when fused with plaque texture-based features during machine learning paradigm. Comput. Biol. Med. 2017, 91, 198–212. [Google Scholar] [CrossRef]
  38. Giannoglou, V.G.; Stavrakoudis, D.G.; Theocharis, J.B.; Petridis, V. Genetic fuzzy rule based classification systems for coronary plaque characterization based on intravascular ultrasound images. Eng. Appl. Artif. Intell. 2015, 38, 203–220. [Google Scholar] [CrossRef]
  39. Rezaei, Z.; Selamat, A.; Taki, A.; Mohd Rahim, M.S.; Abdul Kadir, M.R. Automatic plaque segmentation based on hybrid fuzzy clustering and k nearest neighborhood using virtual histology intravascular ultrasound images. Appl. Soft Comput. J. 2017, 53, 380–395. [Google Scholar] [CrossRef]
  40. Su, S.; Hu, Z.; Lin, Q.; Hau, W.K.; Gao, Z.; Zhang, H. An artificial neural network method for lumen and media-adventitia border detection in IVUS. Comput. Med. Imaging Graph. 2017, 57, 29–39. [Google Scholar] [CrossRef] [PubMed]
  41. Sudarshan, V.; Acharya, U.R.; Ng, E.Y.K.; Meng, C.S.; Tan, R.S.; Ghista, D.N. Automated Identification of Infarcted Myocardium Tissue Characterization Using Ultrasound Images: A Review. IEEE Rev. Biomed. Eng. 2015, 8, 86–97. [Google Scholar] [CrossRef]
  42. Forssen, H.; Patel, R.; Fitzpatrick, N.; Hingorani, A.; Timmis, A.; Hemingway, H.; Denaxas, S. Evaluation of Machine Learning Methods to Predict Coronary Artery Disease Using Metabolomic Data. In Stud Health Technol Inform; Randell, R., Cornet, R., Scott, P.J., Cornet, R., Peek, N., McCowan, C., Eds.; IOS Press: Amsterdam, The Netherlands, 2017; Volume 235, pp. 111–115. [Google Scholar]
  43. Nakajima, K.; Okuda, K.; Watanabe, S.; Matsuo, S.; Kinuya, S.; Toth, K.; Edenbrandt, L. Artificial neural network retrained to detect myocardial ischemia using a Japanese multicenter database. Ann. Nucl. Med. 2018, 32, 303–310. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Nakajima, K.; Matsuo, S.; Wakabayashi, H.; Yokoyama, K.; Bunko, H.; Okuda, K.; Kinuya, S.; Nystrom, K.; Edenbrandt, L. Diagnostic Performance of Artificial Neural Network for Detecting Ischemia in Myocardial Perfusion Imaging. Circ. J. 2015, 79, 1549–1556. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Nakajima, K.; Kudo, T.; Nakata, T.; Kiso, K.; Kasai, T.; Taniguchi, Y.; Matsuo, S.; Momose, M.; Nakagawa, M.; Sarai, M.; et al. Diagnostic accuracy of an artificial neural network compared with statistical quantitation of myocardial perfusion images: A Japanese multicenter study. Eur. J. Nucl. Med. Mol. Imaging 2017, 44, 2280–2289. [Google Scholar] [CrossRef] [Green Version]
  46. Cho, H.; Lee, J.G.; Kang, S.J.; Kim, W.J.; Choi, S.Y.; Ko, J.; Min, H.S.; Choi, G.H.; Kang, D.Y.; Lee, P.H.; et al. Angiography-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions. Am. Heart J. 2019, 8. [Google Scholar] [CrossRef] [Green Version]
  47. Patel, M.R.; Peterson, E.D.; Dai, D.; Brennan, J.M.; Redberg, R.F.; Anderson, H.V.; Brindis, R.G.; Douglas, P.S. Low diagnostic yield of elective coronary angiography. N. Engl. J. Med. 2010, 362, 886–895. [Google Scholar] [CrossRef] [Green Version]
  48. Kolossváry, M.; De Cecco, C.N.; Feuchtner, G.; Maurovich-Horvat, P. Advanced atherosclerosis imaging by CT: Radiomics, machine learning and deep learning. J. Cardiovasc. Comput. Tomogr. 2019, 13, 274–280. [Google Scholar] [CrossRef]
  49. Oikonomou, E.K.; West, H.W.; Antoniades, C. Cardiac Computed Tomography: Assessment of Coronary Inflammation and Other Plaque Features. Arterioscler. Thromb. Vasc. Biol. 2019, 39, 2207–2219. [Google Scholar] [CrossRef]
  50. Dey, D.; Gaur, S.; Ovrehus, K.A.; Slomka, P.J.; Betancur, J.; Goeller, M.; Hell, M.M.; Gransar, H.; Berman, D.S.; Achenbach, S.; et al. Integrated prediction of lesion-specific ischaemia from quantitative coronary CT angiography using machine learning: A multicentre study. Eur. Radiol. 2018, 28, 2655–2664. [Google Scholar] [CrossRef] [PubMed]
  51. Hu, X.; Yang, M.; Han, L.; Du, Y. Diagnostic performance of machine-learning-based computed fractional flow reserve (FFR) derived from coronary computed tomography angiography for the assessment of myocardial ischemia verified by invasive FFR. Int. J. Cardiovasc. Imaging 2018, 34, 1987–1996. [Google Scholar] [CrossRef] [PubMed]
  52. Tesche, C.; De Cecco, C.N.; Baumann, S.; Renker, M.; McLaurin, T.W.; Duguay, T.M.; Bayer, R.R.; Steinberg, D.H.; Grant, K.L.; Canstein, C.; et al. Coronary CT angiography–derived fractional flow reserve: Machine learning algorithm versus computational fluid dynamics modeling. Radiology 2018, 288, 64–72. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. von Knebel Doeberitz, P.L.; De Cecco, C.N.; Schoepf, U.J.; Duguay, T.M.; Albrecht, M.H.; van Assen, M.; Bauer, M.J.; Savage, R.H.; Pannell, J.T.; De Santis, D.; et al. Coronary CT angiography-derived plaque quantification with artificial intelligence CT fractional flow reserve for the identification of lesion-specific ischemia. Eur. Radiol. 2018, 6. [Google Scholar] [CrossRef]
  54. Yu, M.; Lu, Z.; Li, W.; Wei, M.; Yan, J.; Zhang, J. CT morphological index provides incremental value to machine learning based CT-FFR for predicting hemodynamically significant coronary stenosis. Int. J. Cardiol. 2018, 265, 256–261. [Google Scholar] [CrossRef]
  55. Kurata, A.; Fukuyama, N.; Hirai, K.; Kawaguchi, N.; Tanabe, Y.; Okayama, H.; Shigemi, S.; Watanabe, K.; Uetani, T.; Ikeda, S.; et al. On-site computed tomography-derived fractional flow reserve using a machine-learning algorithm: Clinical effectiveness in a retrospective multicenter cohort. Circ. J. 2019, 83, 1563–1571. [Google Scholar] [CrossRef] [Green Version]
  56. Wardziak, Ł.; Kruk, M.; Pleban, W.; Demkow, M.; Rużyłło, W.; Dzielińska, Z.; Kępka, C. Coronary CTA enhanced with CTA based FFR analysis provides higher diagnostic value than invasive coronary angiography in patients with intermediate coronary stenosis. J. Cardiovasc. Comput. Tomogr. 2019, 13, 62–67. [Google Scholar] [CrossRef]
  57. Yu, M.; Lu, Z.; Shen, C.; Yan, J.; Wang, Y.; Lu, B.; Zhang, J. The best predictor of ischemic coronary stenosis: Subtended myocardial volume, machine learning–based FFR CT, or high-risk plaque features? Eur. Radiol. 2019. [Google Scholar] [CrossRef]
  58. Nicol, E.D.; Norgaard, B.L.; Blanke, P.; Ahmadi, A.; Weir-McCall, J.; Horvat, P.M.; Han, K.; Bax, J.J.; Leipsic, J. The Future of Cardiovascular Computed Tomography: Advanced Analytics and Clinical Insights. JACC Cardiovasc. Imaging 2019, 12, 1058–1072. [Google Scholar] [CrossRef]
  59. Zhuang, B.; Wang, S.; Zhao, S.; Lu, M. Computed tomography angiography-derived fractional flow reserve (CT-FFR) for the detection of myocardial ischemia with invasive fractional flow reserve as reference: Systematic review and meta-analysis. Eur. Radiol. 2019. [Google Scholar] [CrossRef]
  60. Bizopoulos, P.; Koutsouris, D. Deep Learning in Cardiology. IEEE Rev. Biomed. Eng. 2019, 12, 168–193. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  61. Pijls, N.H.; Van Gelder, B.; Van der Voort, P.; Peels, K.; Bracke, F.A.; Bonnier, H.J.; el Gamal, M.I. Fractional flow reserve. A useful index to evaluate the influence of an epicardial coronary stenosis on myocardial blood flow. Circulation 1995, 92, 3183–3193. [Google Scholar] [CrossRef]
  62. Han, D.; Lee, J.H.; Rizvi, A.; Gransar, H.; Baskaran, L.; Schulman-Marcus, J.; ó Hartaigh, B.; Lin, F.Y.; Min, J.K. Incremental role of resting myocardial computed tomography perfusion for predicting physiologically significant coronary artery disease: A machine learning approach. J. Nucl. Cardiol. 2018, 25, 223–233. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  63. Itu, L.; Rapaka, S.; Passerini, T.; Georgescu, B.; Schwemmer, C.; Schoebinger, M.; Flohr, T.; Sharma, P.; Comaniciu, D. A machine-learning approach for computation of fractional flow reserve from coronary computed tomography. J. Appl. Physiol. 2016, 121, 42–52. [Google Scholar] [CrossRef] [Green Version]
  64. Coenen, A.; Kim, Y.H.; Kruk, M.; Tesche, C.; De Geer, J.; Kurata, A.; Lubbers, M.L.; Daemen, J.; Itu, L.; Rapaka, S.; et al. Diagnostic accuracy of a machine-learning approach to coronary computed tomographic angiography–Based fractional flow reserve result from the MACHINE Consortium. Circ. Cardiovasc. Imaging 2018, 11. [Google Scholar] [CrossRef] [Green Version]
  65. Hae, H.; Kang, S.J.; Kim, W.J.; Choi, S.Y.; Lee, J.G.; Bae, Y.; Cho, H.; Yang, D.H.; Kang, J.W.; Lim, T.H.; et al. Machine learning assessment of myocardial ischemia using angiography: Development and retrospective validation. PLoS Med. 2018, 15, e1002693. [Google Scholar] [CrossRef] [Green Version]
  66. Park, A.; Chute, C.; Rajpurkar, P.; Lou, J.; Ball, R.L.; Shpanskaya, K.; Jabarkheel, R.; Kim, L.H.; McKenna, E.; Tseng, J.; et al. Deep Learning-Assisted Diagnosis of Cerebral Aneurysms Using the HeadXNet Model. JAMA Netw. Open 2019, 2, e195600. [Google Scholar] [CrossRef] [Green Version]
  67. van Rosendael, A.R.; Maliakal, G.; Kolli, K.K.; Beecy, A.; Al’Aref, S.J.; Dwivedi, A.; Singh, G.; Panday, M.; Kumar, A.; Ma, X.; et al. Maximization of the usage of coronary CTA derived plaque information using a machine learning based algorithm to improve risk stratification; insights from the CONFIRM registry. J. Cardiovasc. Comput. Tomogr. 2018, 12, 204–209. [Google Scholar] [CrossRef] [PubMed]
  68. Wang, W.; Wang, H.; Chen, Q.; Zhou, Z.; Wang, R.; Wang, H.; Zhang, N.; Chen, Y.; Sun, Z.; Xu, L. Coronary artery calcium score quantification using a deep-learning algorithm. Clin. Radiol. 2020, 75, 237.e11–237.e16. [Google Scholar] [CrossRef] [PubMed]
  69. von Knebel Doeberitz, P.L.; De Cecco, C.N.; Schoepf, U.J.; Albrecht, M.H.; van Assen, M.; De Santis, D.; Gaskins, J.; Martin, S.; Bauer, M.J.; Ebersberger, U.; et al. Impact of Coronary Computerized Tomography Angiography-Derived Plaque Quantification and Machine-Learning Computerized Tomography Fractional Flow Reserve on Adverse Cardiac Outcome. Am. J. Cardiol. 2019, 124, 1340–1348. [Google Scholar] [CrossRef]
  70. Priyatharshini, R.; Chitrakala, S. An efficient coronary disease diagnosis system using dual-phase multi-objective optimization and embedded feature selection. Int. J. Intell. Inf. Technol. 2017, 13, 15–36. [Google Scholar] [CrossRef] [Green Version]
  71. Han, D.; Kolli, K.K.; Al’Aref, S.J.; Baskaran, L.; van Rosendael, A.R.; Gransar, H.; Andreini, D.; Budoff, M.J.; Cademartiri, F.; Chinnaiyan, K.; et al. Machine Learning Framework to Identify Individuals at Risk of Rapid Progression of Coronary Atherosclerosis: From the PARADIGM Registry. Am. Heart J. 2020, 9, e013958. [Google Scholar] [CrossRef]
  72. Tesche, C.; Plank, F.; De Cecco, C.N.; Duguay, T.M.; Albrecht, M.H.; Varga-Szemes, A.; Bayer, R.R., 2nd; Yang, J.; Jacks, I.L.; Gramer, B.M.; et al. Prognostic implications of coronary CT angiography-derived quantitative markers for the prediction of major adverse cardiac events. J. Cardiovasc. Comput. Tomogr. 2016, 10, 458–465. [Google Scholar] [CrossRef]
  73. Zhang, W.; Zhang, J.; Du, X.; Zhang, Y.; Li, S. An end-to-end joint learning framework of artery-specific coronary calcium scoring in non-contrast cardiac CT. Computing 2019, 101, 667–678. [Google Scholar] [CrossRef]
  74. Wolterink, J.M.; Leiner, T.; Viergever, M.A.; Išgum, I. Automatic coronary calcium scoring in cardiac CT angiography using convolutional neural networks. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Cham, Switzerland, 2015; Volume 9349, pp. 589–596. [Google Scholar]
  75. Wolterink, J.M.; Leiner, T.; Takx, R.A.P.; Viergever, M.A.; Išgum, I. Automatic Coronary Calcium Scoring in Non-Contrast-Enhanced ECG-Triggered Cardiac CT with Ambiguity Detection. IEEE Trans. Med. Imaging 2015, 34, 1867–1878. [Google Scholar] [CrossRef]
  76. Wolterink, J.M.; Leiner, T.; De Vos, B.D.; Coatrieux, J.L.; Kelm, B.M.; Kondo, S.; Salgado, R.A.; Shahzad, R.; Shu, H.; Snoeren, M.; et al. An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework. Med. Phys. 2016, 43, 2361–2373. [Google Scholar] [CrossRef] [PubMed]
  77. Shahzad, R.; van Walsum, T.; Schaap, M.; Rossi, A.; Klein, S.; Weustink, A.C.; de Feyter, P.J.; van Vliet, L.J.; Niessen, W.J. Vessel specific coronary artery calcium scoring: An automatic system. Acad. Radiol. 2013, 20, 1–9. [Google Scholar] [CrossRef]
  78. Motwani, M.; Dey, D.; Berman, D.S.; Germano, G.; Achenbach, S.; Al-Mallah, M.H.; Andreini, D.; Budoff, M.J.; Cademartiri, F.; Callister, T.Q.; et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: A 5-year multicentre prospective registry analysis. Eur. Heart J. 2017, 38, 500–507. [Google Scholar] [CrossRef]
  79. Han, D.; Kolli, K.K.; Gransar, H.; Lee, J.H.; Choi, S.Y.; Chun, E.J.; Han, H.W.; Park, S.H.; Sung, J.; Jung, H.O.; et al. Machine learning based risk prediction model for asymptomatic individuals who underwent coronary artery calcium score: Comparison with traditional risk prediction approaches. J. Cardiovasc. Comput. Tomogr. 2019. [Google Scholar] [CrossRef] [PubMed]
  80. Zreik, M.; Lessmann, N.; van Hamersvelt, R.W.; Wolterink, J.M.; Voskuil, M.; Viergever, M.A.; Leiner, T.; Išgum, I. Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis. Med. Image Anal. 2018, 44, 72–85. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  81. van Hamersvelt, R.W.; Zreik, M.; Voskuil, M.; Viergever, M.A.; Išgum, I.; Leiner, T. Deep learning analysis of left ventricular myocardium in CT angiographic intermediate-degree coronary stenosis improves the diagnostic accuracy for identification of functionally significant stenosis. Eur. Radiol. 2019, 29, 2350–2359. [Google Scholar] [CrossRef] [Green Version]
  82. De Geer, J.; Coenen, A.; Kim, Y.H.; Kruk, M.; Tesche, C.; Schoepf, U.J.; Kepka, C.; Yang, D.H.; Nieman, K.; Persson, A. Effect of tube voltage on diagnostic performance of fractional flow reserve derived from coronary CT angiography with machine learning: Results from the machine registry. Am. J. Roentgenol. 2019, 213, 325–331. [Google Scholar] [CrossRef] [PubMed]
  83. Freiman, M.; Nickisch, H.; Prevrhal, S.; Schmitt, H.; Vembar, M.; Maurovich-Horvat, P.; Donnelly, P.; Goshen, L. Improving CCTA-based lesions’ hemodynamic significance assessment by accounting for partial volume modeling in automatic coronary lumen segmentation. Med. Phys. 2017, 44, 1040–1049. [Google Scholar] [CrossRef]
  84. Acharya, U.R.; Sree, S.V.; Mookiah, M.R.; Saba, L.; Gao, H.; Mallarini, G.; Suri, J.S. Computed tomography carotid wall plaque characterization using a combination of discrete wavelet transform and texture features: A pilot study. Proc. Inst. Mech. Eng. H 2013, 227, 643–654. [Google Scholar] [CrossRef]
  85. Mao, F.; Gill, J.; Downey, D.; Fenster, A. Segmentation of carotid artery in ultrasound images. In Proceedings of the Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143), Chicago, IL, USA, 23–28 July 2000; pp. 1734–1737. [Google Scholar]
  86. Sabetai, M.M.; Tegos, T.J.; Nicolaides, A.N.; Dhanjil, S.; Pare, G.J.; Stevens, J.M. Reproducibility of computer-quantified carotid plaque echogenicity: Can we overcome the subjectivity? Stroke 2000, 31, 2189–2196. [Google Scholar] [CrossRef] [Green Version]
  87. WA Health. Diagnostic Imaging Pathways—Stroke. Available online: http://www.imagingpathways.health.wa.gov.au/index.php/imaging-pathways/neurological/stroke#pathway (accessed on 26 April 2019).
  88. WA Health. Diagnostic Imaging Pathways—Transient Ischemic Attack. Available online: http://www.imagingpathways.health.wa.gov.au/index.php/imaging-pathways/neurological?id=63#pathway (accessed on 26 April 2019).
  89. National Institute for Health and Care Excellence. Stroke Overview; National Institute for Health and Care Excellence: London, UK, 2020. [Google Scholar]
  90. Touboul, P.J.; Hennerici, M.G.; Meairs, S.; Adams, H.; Amarenco, P.; Bornstein, N.; Csiba, L.; Desvarieux, M.; Ebrahim, S.; Hernandez Hernandez, R.; et al. Mannheim carotid intima-media thickness and plaque consensus (2004–2006–2011). An update on behalf of the advisory board of the 3rd, 4th and 5th watching the risk symposia, at the 13th, 15th and 20th European Stroke Conferences, Mannheim, Germany, 2004, Brussels, Belgium, 2006, and Hamburg, Germany, 2011. Cerebrovasc. Dis. 2012, 34, 290–296. [Google Scholar] [CrossRef] [Green Version]
  91. Golemati, S.; Stoitsis, J.; Sifakis, E.G.; Balkizas, T.; Nikita, K.S. Using the Hough transform to segment ultrasound images of longitudinal and transverse sections of the carotid artery. Ultrasound Med. Biol. 2007, 33, 1918–1932. [Google Scholar] [CrossRef] [PubMed]
  92. Bastida-Jumilla, M.C.; Menchón-Lara, R.M.; Morales-Sánchez, J.; Verdú-Monedero, R.; Larrey-Ruiz, J.; Sancho-Gómez, J.L. Frequency-domain active contours solution to evaluate intima–media thickness of the common carotid artery. Biomed. Signal. Process. Contr 2015, 16, 68–79. [Google Scholar] [CrossRef]
  93. Huang, X.; Zhang, Y.; Meng, L.; Qian, M.; Wong, K.K.L.; Abbott, D.; Zheng, R.; Zheng, H.; Niu, L.; Huang, X.; et al. Identification of Ultrasonic Echolucent Carotid Plaques Using Discrete Frechet Distance Between Bimodal Gamma Distributions. IEEE Trans. Biomed. Eng. 2018, 65, 949–955. [Google Scholar] [CrossRef] [PubMed]
  94. Grønholdt, M.L.M.; Wiebe, B.M.; Laursen, H.; Nielsen, T.G.; Schroeder, T.V.; Sillesen, H. Lipid-rich carotid artery plaques appear echolucent on ultrasound B-mode images and may be associated with intraplaque haemorrhage. Eur. J. Vasc. Endovasc. Surg. 1997, 14, 439–445. [Google Scholar] [CrossRef] [Green Version]
  95. Mathiesen, E.B.; Bonaa, K.H.; Joakimsen, O. Echolucent plaques are associated with high risk of ischemic cerebrovascular events in carotid stenosis: The tromso study. Circulation 2001, 103, 2171–2175. [Google Scholar] [CrossRef] [Green Version]
  96. Pedro, L.M.; Sanches, J.M.; Seabra, J.; Suri, J.S.; Fernandes, E.; Fernandes, J. Asymptomatic carotid disease—A new tool for assessing neurological risk. Echocardiography 2014, 31, 353–361. [Google Scholar] [CrossRef]
  97. Roy-Cardinal, M.H.; Destrempes, F.; Soulez, G.; Cloutier, G. Assessment of carotid artery plaque components with machine learning classification using homodyned-K parametric maps and elastograms. IEEE Trans. Ultrason Ferroelectr. Freq. Control. 2018, 29, 29. [Google Scholar] [CrossRef] [PubMed]
  98. Golemati, S.; Patelaki, E.; Gastounioti, A.; Andreadis, I.; Liapis, C.D.; Nikita, K.S. Motion synchronisation patterns of the carotid atheromatous plaque from B-mode ultrasound. Sci. Rep. 2020, 10, 11221. [Google Scholar] [CrossRef]
  99. Waddle, S.L.; Juttukonda, M.R.; Lants, S.K.; Davis, L.T.; Chitale, R.; Fusco, M.R.; Jordan, L.C.; Donahue, M.J. Classifying intracranial stenosis disease severity from functional MRI data using machine learning. J. Cereb. Blood Flow Metab. 2020, 40, 705–719. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  100. Wu, J.; Xin, J.; Yang, X.; Sun, J.; Xu, D.; Zheng, N.; Yuan, C. Deep morphology aided diagnosis network for segmentation of carotid artery vessel wall and diagnosis of carotid atherosclerosis on black-blood vessel wall MRI. Med. Phys. 2019, 46, 5544–5561. [Google Scholar] [CrossRef]
  101. Zhao, X.; Li, R.; Hippe, D.S.; Hatsukami, T.S.; Yuan, C. Chinese Atherosclerosis Risk Evaluation (CARE II) study: A novel cross-sectional, multicentre study of the prevalence of high-risk atherosclerotic carotid plaque in Chinese patients with ischaemic cerebrovascular events-design and rationale. Stroke Vasc. Neurol. 2017, 2, 15–20. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  102. Brown, B.G.; Boden, W.E.; Probstfield, J.R.; McBride, R.; Anderson, T.; Chaitman, B.; Marcovina, S.; Weintraub, W.; Zhao, X.Q.; Teo, K.; et al. The role of niacin in raising high-density lipoprotein cholesterol to reduce cardiovascular events in patients with atherosclerotic cardiovascular disease and optimally treated low-density lipoprotein cholesterol Rationale and study design. The Atherothrombosis Intervention in Metabolic syndrome with low HDL/high triglycerides: Impact on Global Health outcomes (AIM-HIGH). Am. Heart J. 2011, 161, 471–477. [Google Scholar] [CrossRef] [Green Version]
  103. Cai, J.M.; Hatsukami, T.S.; Ferguson, M.S.; Small, R.; Polissar, N.L.; Yuan, C. Classification of human carotid atherosclerotic lesions with in vivo multicontrast magnetic resonance imaging. Circulation 2002, 106, 1368–1373. [Google Scholar] [CrossRef] [Green Version]
  104. Van Den Bouwhuijsen, Q.J.A.; Vernooij, M.W.; Verhaaren, B.F.J.; Vrooman, H.A.; Niessen, W.J.; Krestin, G.P.; Ikram, M.A.; Franco, O.H.; Van Der Lugt, A. Carotid plaque morphology and ischemic vascular brain disease on MRI. Am. J. Neuroradiol. 2017, 38, 1776–1782. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  105. Vermeer, S.E.; Hollander, M.; van Dijk, E.J.; Hofman, A.; Koudstaal, P.J.; Breteler, M.M.; Rotterdam Scan, S. Silent brain infarcts and white matter lesions increase stroke risk in the general population: The Rotterdam Scan Study. Stroke 2003, 34, 1126–1129. [Google Scholar] [CrossRef] [Green Version]
  106. Ziegler, M.; Good, E.; Engvall, J.; Warntjes, M.; de Muinck, E.; Dyverfeldt, P. Towards Automated Quantification of Vessel Wall Composition Using MRI. J. Magn. Reson. Imaging 2020, 52, 710–719. [Google Scholar] [CrossRef] [Green Version]
  107. van Hespen, K.M.; Zwanenburg, J.J.M.; Hendrikse, J.; Kuijf, H.J. Subvoxel vessel wall thickness measurements of the intracranial arteries using a convolutional neural network. Med. Image Anal. 2021, 67, 101818. [Google Scholar] [CrossRef]
  108. Hadamitzky, M.; Achenbach, S.; Al-Mallah, M.; Berman, D.; Budoff, M.; Cademartiri, F.; Callister, T.; Chang, H.J.; Cheng, V.; Chinnaiyan, K.; et al. Optimized prognostic score for coronary computed tomographic angiography: Results from the CONFIRM registry (COronary CT Angiography EvaluatioN For Clinical Outcomes: An InteRnational Multicenter Registry). J. Am. Coll. Cardiol. 2013, 62, 468–476. [Google Scholar] [CrossRef] [Green Version]
  109. Lee, S.E.; Chang, H.J.; Rizvi, A.; Hadamitzky, M.; Kim, Y.J.; Conte, E.; Andreini, D.; Pontone, G.; Volpato, V.; Budoff, M.J.; et al. Rationale and design of the Progression of AtheRosclerotic PlAque DetermIned by Computed TomoGraphic Angiography IMaging (PARADIGM) registry: A comprehensive exploration of plaque progression and its impact on clinical outcomes from a multicenter serial coronary computed tomographic angiography study. Am. Heart J. 2016, 182, 72–79. [Google Scholar] [CrossRef] [Green Version]
  110. Nous, F.M.A.; Coenen, A.; Boersma, E.; Kim, Y.H.; Kruk, M.B.P.; Tesche, C.; de Geer, J.; Yang, D.H.; Kepka, C.; Schoepf, U.J.; et al. Comparison of the Diagnostic Performance of Coronary Computed Tomography Angiography-Derived Fractional Flow Reserve in Patients With Versus Without Diabetes Mellitus (from the MACHINE Consortium). Am. J. Cardiol. 2019, 123, 537–543. [Google Scholar] [CrossRef]
  111. Sofian, H.; Ming, J.T.C.; Muhammad, S.; Noor, N.M. Calcification detection using convolutional neural network architectures in intravascular ultrasound images. Indones J. Electr. Eng. Comput. Sci. 2019, 17, 1313–1321. [Google Scholar] [CrossRef]
  112. The Royal Australian and New Zealand College of Radiologists. Ethical Principles for Artificial Intelligence in Medicine; RANZCR: Sydney, Australia, 2019; p. 10. [Google Scholar]
  113. Gao, Z.; Wang, X.; Sun, S.; Wu, D.; Bai, J.; Yin, Y.; Liu, X.; Zhang, H.; de Albuquerque, V.H.C. Learning physical properties in complex visual scenes: An intelligent machine for perceiving blood flow dynamics from static CT angiography imaging. Neural Netw. 2019, 123, 82–93. [Google Scholar] [CrossRef] [PubMed]
  114. Cerqueira, M.D.; Weissman, N.J.; Dilsizian, V.; Jacobs, A.K.; Kaul, S.; Laskey, W.K.; Pennell, D.J.; Rumberger, J.A.; Ryan, T.; Verani, M.S.; et al. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart. A statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation 2002, 105, 539–542. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  115. Wolterink, J.M.; Hamersvelt, R.W.V.; Viergever, M.A.; Leiner, T.; Išgum, I. Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier. Med. Image Anal. 2019, 51, 46–60. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  116. Ferguson, G.G.; Eliasziw, M.; Barr, H.W.; Clagett, G.P.; Barnes, R.W.; Wallace, M.C.; Taylor, D.W.; Haynes, R.B.; Finan, J.W.; Hachinski, V.C.; et al. The North American Symptomatic Carotid Endarterectomy Trial: Surgical results in 1415 patients. Stroke 1999, 30, 1751–1758. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  117. NASCET Collaborators; Barnett, H.J.M.; Taylor, D.W.; Haynes, R.B.; Sackett, D.L.; Peerless, S.J.; Ferguson, G.G.; Fox, A.J.; Rankin, R.N.; Hachinski, V.C. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N. Engl. J. Med. 1991, 325, 445–453. [Google Scholar] [CrossRef]
  118. Kirisli, H.A.; Schaap, M.; Metz, C.T.; Dharampal, A.S.; Meijboom, W.B.; Papadopoulou, S.L.; Dedic, A.; Nieman, K.; de Graaf, M.A.; Meijs, M.F.; et al. Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography. Med. Image Anal. 2013, 17, 859–876. [Google Scholar] [CrossRef] [Green Version]
  119. Schaap, M.; Metz, C.T.; van Walsum, T.; van der Giessen, A.G.; Weustink, A.C.; Mollet, N.R.; Bauer, C.; Bogunovic, H.; Castro, C.; Deng, X.; et al. Standardized evaluation methodology and reference database for evaluating coronary artery centerline extraction algorithms. Med. Image Anal. 2009, 13, 701–714. [Google Scholar] [CrossRef] [Green Version]
  120. Carson, J.M.; Pant, S.; Roobottom, C.; Alcock, R.; Javier Blanco, P.; Alberto Bulant, C.; Vassilevski, Y.; Simakov, S.; Gamilov, T.; Pryamonosov, R.; et al. Non-invasive coronary CT angiography-derived fractional flow reserve: A benchmark study comparing the diagnostic performance of four different computational methodologies. Int. J. Numer. Methods Biomed. Eng. 2019, 35. [Google Scholar] [CrossRef] [Green Version]
  121. Zreik, M.; Van Hamersvelt, R.W.; Wolterink, J.M.; Leiner, T.; Viergever, M.A.; Išgum, I. A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography. IEEE Trans. Med. Imaging 2019, 38, 1588–1598. [Google Scholar] [CrossRef] [Green Version]
  122. Oakden-Rayner, L.; Dunnmon, J.; Carneiro, G.; Re, C. Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging. Proc. ACM Conf. Health Inference Learn. 2020, 2020, 151–159. [Google Scholar] [CrossRef] [Green Version]
  123. Marcu, L.G.; Marcu, D. Points of view on artificial intelligence in medical imaging—One good, one bad, one fuzzy. Health Technol. 2021, 11, 17–22. [Google Scholar] [CrossRef]
Diagnostics 11 00551 g001 550
Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram of literature selection process.
Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram of literature selection process.
Diagnostics 11 00551 g001
Table
Table 1. Computed Tomography Angiography (CTA) characteristics.
Table 1. Computed Tomography Angiography (CTA) characteristics.
OrganML Prediction EndpointAuthor (Year)No. of Patients (M/F)ML Approach or SoftwareML ValidationGold StandardContrast Used?
BrainFaster clinician identificiation of intracranial aneurysmPark (2019)662
(157/505)		3-D CNN.75/14/11 train/validation/testClinicians’ segmentation (n = 8)Yes
Heart5 year ACMMotwani (2017)10,030
(5628/4402)		LogitBoostk-fold cross validation (n = 10)Existing clinical or cCTA metricsYes
ACMHan (2019)86,155
(59,745/26,410)		LogitBoost70/30 holdout validationMortality status at follow up (median 4.6 years)No
CAD—Calcium scoringShahzad (2013)366
(280/86)		k-Nearest Neighbour57/43 holdout validationExpert Calcium ScoringNo
Coronary vessel centreline extractionWolterink (2019)823-D CNN. + SVMMICCAI 2008 CAT08 dataset—32 pre segmented cCTA imagesClinicians’ segmentation (n = 3)Yes
FFR variation with kVpDe Geer (2019)351SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Functional stenosis significanceCoenen (2018)351
(258/93)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Hae (2018)1132
(860/272)		Light Gradient Boosting Machine83/17 Holdout validation k-fold cross validation (n = 3)79 external patients CAAS-5 softwareYes
Han (2018)252
(178/74)		SmartHeart SoftwareLeave one out cross validationInvasive coronary angiography FFRYes
Kurata
(2019)		74
(56/18)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
van Hamersvelt (2019)126
(97/29)		As described in Zreik (2018)k-fold cross validation (n = 50)Invasive coronary angiography FFRYes
Zreik (2018)166
(128/38)		3-D CNN + SVMManual segmentation of 40 patientsInvasive coronary angiography FFRYes
Dey (2018)254
(162/92)		Ensemble classification approach (Supervised ensemble learning)k-fold cross validation (n = 10)Invasive coronary angiography FFRYes
von Knebel Doeberitz (2018)84
(54/30)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Wardziak
(2019)		90
(61/29)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Yu (2018)129SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Yu (2019)180SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Hu (2018)105
(73/32)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Nous
(2019)		351
(258/93)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Zreik (2019)163
(126/37)		Recurrent CNN50/10/40 train/validation/testClinicians’ segmentation (n = 2)Yes
Functional stenosis significance (myocardial bridging)Zhou (2019)161
(103/58)		SyngoTM cFFR12,000 virtual coronary models41 control patients Clinicians segmentations (n = 2)Yes
MACE related lesions Tesche (2016)92
(57/35)		SyngoTM Coronary Plaque Analysis 2.0.3 Invasive coronary angiography FFRYes
von Knebel Doeberitz (2019)82
(51/30)		SyngoTM cFFR12,000 virtual coronary modelsInvasive coronary angiography FFRYes
Machine learning ischemia risk scoreKwan
(2020)		352
(238/114)		Ensemble classification approach (Supervised ensemble learning)k-fold cross validation (n = 10)Invasive coronary angiography FFRYes
Plaque based risk stratificationPriyatharshini (2017)76Active contour model-based region growing Agatston scoreYes
Plaque based risk stratificationZhang (2019)129Dense U-netk-fold cross validation (n = 5) 40 “orScore” database patients2 expert Agatston scoreNo
Plaque based risk stratificationvan Rosendael (2018)8844
(5102/3742)		Gradient boosted decision trees80/20 holdout validation k-fold cross validation (n = 5)Clinician segmentation + cCTA risk scoreYes
Plaque based risk stratificationWang (2019)5303D-Resnet deep neural network56/17/27 train/validation/testAgatston ScoreNo
Plaque stabilityAl’Aref
(2020)		468XGBoost80/20 holdout validation k-fold cross validation (n = 10)Invasive coronary angiographyYes
Rapidly progressing plaqueHan
(2020)		1083
(624/459)		LogitBoost, Naïve Bayes, BayesNet, AdaBoost, Random Forest, Bagging, Stacking, MLP, Sequential Minimimal Optimization, ADTree70/30 holdout validationAtherosclerotic cardiovascular disease risk score/duke coronary artery disease scoreYes
CNN = Convolutional Neural Network, ACM = All-cause mortality, cCTA = Coronary computed tomography angiography, CAD = Coronary artery disease, FFR = Fractional flow reserve, kVp = Kilovoltage peak, cFFR = Software computation of FFR, SVM = Support vector machine, MACE = Major adverse cardiovascular events, MLP = Multilayer perceptron, ADTree = Alternating decision tree.
Table
Table 2. Model performance statistics for machine learning (ML) quantitation using CTA.
Table 2. Model performance statistics for machine learning (ML) quantitation using CTA.
OrganML PredictionEndpointStatistics QuotedAuthor (Year)Sample SizeSensitivitySpecificityAccuracyAUC
MinMaxMinMaxMinMaxMinMax
BrainFaster aneurysm identificationPer clinicianPark (2019)818 exams 0.89 0.98 0.93
Heart5-year ACMPer patientMotwani (2017)10,030 patients 0.790.79
ACM (CAD)Per patientHan (2019)86,155 patients 0.740.78
CAD—Calcium scoringPer patientShahzad (2013)366 patients 0.84
Coronary vessel centreline extraction Wolterink (2019)82 patients
FFR variation with kVpPer vesselDe Geer (2019)525 vessels0.741.000.730.790.770.860.820.90
Functional stenosis significancePer patientCoenen (2018)525 lesions0.820.960.600.830.750.91
Hae (2018)1132 lesions0.730.840.760.850.740.840.800.91
Han (2018)252 patients0.520.710.610.850.640.68
Kurata (2019)91 lesions0.330.900.380.910.590.85
van Hamersvelt (2019)126 patients 0.85 0.48 0.72 0.76
Zreik (2018)166 patients 0.70 0.71 0.710.620.85
Per lesionDey (2018)2758 artery segments 0.84
von Knebel Doeberitz (2018)103 lesions0.620.880.330.68 0.610.93
Wardziak (2019)96 lesions 0.76 0.72 0.74 0.84
Yu (2018)166 lesions 0.850.88
Yu (2019)208 lesions0.810.940.820.870.830.860.870.94
Per vesselHu (2018)117 lesions0.61 0.91 0.82 0.860.92
Nous (2019)525 arteries0.790.880.720.800.750.830.820.88
Per segmentZreik (2019)676 lesions 0.620.80
Functional stenosis significance (myocardial bridging)Per lesionZhou (2019) 161 patients 0.650.77
MACEPer patientTesche (2016)258 lesions0.630.830.730.83 0.720.82
Per lesionvon Knebel Doeberitz (2019)82 patients 0.82 0.79 0.94
Machine learning ischemia risk scorePer vesselKwan (2020)352 patients 0.78
Plaque based risk stratificationPer patientPriyatharshini (2017)76 patients 0.91
Per lesionZhang (2019)129 patients0.860.91
Per vesselvan Rosendael (2018)8844 patients 0.77
Wang (2019)530 patients
Plaque stabilityPer lesionAl’Aref (2020)582 lesions 0.77
Rapidly progressing plaquePer patientHan (2020)1083 patients 0.790.83
ACM = All-cause mortality, CAD = Coronary artery disease, FFR = Fractional flow reserve, kVp = Kilovoltage peak, MACE = Major adverse cardiovascular event.
Table
Table 3. Model performance statistics using Siemens Syngo cFFR.
Table 3. Model performance statistics using Siemens Syngo cFFR.
OrganML Prediction EndpointStatistics QuotedAuthor (Year)Sample SizeSensitivitySpecificityAccuracyAUC
MinMaxMinMaxMinMaxMinMax
HeartFFR variation with kVpPer vesselDe Geer (2019)525 vessels0.741.000.730.790.770.860.820.90
Functional stenosis significancePer patientCoenen (2018)525 lesions0.820.960.600.830.750.91
Kurata (2019)91 lesions0.330.900.380.910.590.85
von Knebel Doeberitz (2018)103 lesions0.620.880.330.68 0.610.93
Wardziak (2019)96 lesions 0.76 0.72 0.74 0.84
Yu (2018)166 lesions 0.850.88
Yu (2019)208 lesions0.810.940.820.870.830.860.870.94
Per vesselHu (2018)117 lesions0.61 0.91 0.82 0.860.92
Nous (2019)525 arteries0.790.880.720.800.750.830.820.88
Functional stenosis significance (myocardial bridging)Per lesionZhou (2019) 161 patients 0.650.77
MACE related lesions Per lesionvon Knebel Doeberitz (2019)82 patients 0.82 0.79 0.94
FFR = Fractional flow reserve, kVp = Kilovoltage peak, MACE = Major adverse cardiovascular events.
Table
Table 4. Ultrasound characteristics.
Table 4. Ultrasound characteristics.
OrganML Prediction EndpointAuthor (Year)No. of Patients (M/F)US TypeML ApproachML ValidationGold Standard
BrainCarotid elastographyRoy-Cardinal (2019)66
(47/19)		B-ModeRandom forest0.632+ validationPatient symptoms
Carotid plaque echomorphologyGolemati (2020)77
(59/18)		B-ModeRandom forestLeave one outClinicians’ segmentations (n = 1)
Huang (2018)153B-Modek-nearest neighboursk-fold cross validation (n = 3)Grayscale median
Pedro (2014)109
(34/75)		B-ModeCutoff of ROCLeave one outClinician assignment of symptomatic plaque status
Carotid plaque segmentationMenchon-Lara (2016)67B-ModeNeural Network66/33 Holdout validationClinicians’ segmentations (repeated) (n = 2)
IMT measurement & plaque detectionHassan (2013)300B-ModeFuzzy C-mean & probabilistic neural network Clinicians’ segmentations (n = 1)
HeartProbability of OCT identified thin-cap fibroatheromaBae (2019)517 (382/135)IVUSANN, SVM, naïve bayesk-fold cross (n = 5) 80/20 HoldoutPresence of OCT thin-cap fibroatheroma
B-mode = Anatomical ultrasound, ROC = Receiver operating characteristic curve, IMT = Intima-media thickness, OCT = Optical coherence tomography, IVUS = Intravascular Ultrasound, ANN = Artificial neural network, SVM = Support vector machine.
Table
Table 5. Model performance statistics for ML quantitation using B-mode US and IVUS.
Table 5. Model performance statistics for ML quantitation using B-mode US and IVUS.
OrganML Prediction EndpointStatistics QuotedAuthor (Year)Sample SizeSensitivitySpecificityAccuracyAUC
MinMaxMinMaxMinMaxMinMax
BrainCarotid elastographyPer patientRoy-Cardinal (2019)66 patients 0.790.83
Carotid plaque echomorphologyPer lesionGolemati (2020)77 patients0.690.860.680.880.690.850.790.90
Per patientHuang (2018)315 frames0.680.810.630.890.720.850.710.83
Per imagePedro (2014)146 frames0.660.700.760.800.730.770.790.89
Carotid plaque segmentation Menchon-Lara (2016)67 patients
IMT measurement & plaque detectionPer patientHassan (2013)300 frames 0.98 0.98 0.98 0.98
HeartProbability of OCT identified thin-cap fibroatheromaPer imageBae (2019)41,101 frames0.810.840.610.790.760.820.740.82
IMT = Intima-media thickness, OCT = Optical coherence tomography.
Table
Table 6. Characteristics of modalities other than CT and ultrasound.
Table 6. Characteristics of modalities other than CT and ultrasound.
OrganML Prediction EndpointAuthor (Year)No. of Patients (M/F)Imaging ModalityML ApproachML ValidationGold Standard
BrainAtherosclerosis identificationWu (2019)1482MRI2.5D CNN (U-Net)90/10 holdout validationClinicians’ segmentations (n unknown)
Cerebral blood flow & cerebrovascular reactivityWaddle (2019)53
(10/43)		MRI LOO-CV k-fold cross validation (n = 3)Invasive coronary angiography
HeartFunctional stenosis significanceCho
(2019)		1501
(1157/344)		Invasive Angiography XGBoost 80/20 holdout validation k-fold cross validation (n = 5)79 external patients
Gao (2019)0Computer Generated CTA Recurrent Neural Net 180 external patients w/Invasive coronary angiography FFR
Presence of CADForssen (2017)3409NMR quantification of 256 metabolitesRandom Forest + Penalized Logistic Regressionk-fold logistic regression (n = 50)Coronary angiography reports
Probability of myocardial ischemiaNakajima (2015)106
(65/41)		NM
(Tc-MPI)		Artificial Neural Net. Clinicians’ segmentations (n = 3)
Nakajima (2017)1001
(751/250)		NM
(Tc-MPI)		Artificial Neural Net.364 (265/98) external patientsClinicians’ segmentations (n unknown)
Nakajima (2018)106
(65/41)		NM
(Tc-MPI)		Artificial Neural Net. Clinicians’ segmentations (n = 3)
Wang (2020)88
(83/5)		PET
(13N-NH3
&
18F-FDG)		SVM, Logistic Regression, Decision Tree, Linear Discriminant Analysis, Naïve Bayes, k-Nearest Neighbour, Random Forest60/40 holdout validationInvasive coronary angiography
MRI = Magnetic resonance imaging, CNN = Convolutional neural network, LOO-CV = Leave one out cross validation, CTA = Computed tomography angiography, CAD = Coronary artery disease, NMR = Nuclear magnetic resonance, NM = Nuclear Medicine, Tc-MPI = Technetium-99m myocardial perfusion imaging, PET = Positron emission tomography, 13N-NH3 = Nitrogen-13 ammonia, 18F-FDG = Fluorine-18 fluorodeoxyglucose, SVM = Support vector machine.
Table
Table 7. Model performance statistics for ML quantitation using modalities other than CT and ultrasound.
Table 7. Model performance statistics for ML quantitation using modalities other than CT and ultrasound.
OrganML Prediction EndpointStatistics QuotedAuthor (Year)Sample SizeSensitivitySpecificityAccuracyAUC
MinMaxMinMaxMinMaxMinMax
BrainAtherosclerosis identification Wu (2019)18,915 frames 0.810.890.870.95
Cerebral blood flow and cerebrovascular reactivity Waddle (2019)112 hemispheres0.430.70.670.83 0.650.71
HeartFunctional stenosis significancePer patientCho(2019)1501 frames0.720.840.770.890.810.850.870.90
Per patientGao (2019)13,000 synthetic trees0.840.920.750.89 0.890.94
Presence of CADPer patientForssen (2017)3409 patients0.940.940.210.280.710.730.680.71
Probability of myocardial ischemiaPer patientNakajima (2015)106 patients 0.69 0.62 0.660.880.97
Per patientNakajima (2017)1001 patients 0.900.93
Per patientNakajima (2018)106 patients0.780.870.960.980.890.920.890.96
Per patientWang (2020)159 vessels0.720.910.320.840.650.810.620.86
CAD = Coronary artery disease.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Boyd, C.; Brown, G.; Kleinig, T.; Dawson, J.; McDonnell, M.D.; Jenkinson, M.; Bezak, E. Machine Learning Quantitation of Cardiovascular and Cerebrovascular Disease: A Systematic Review of Clinical Applications. Diagnostics 2021, 11, 551. https://doi.org/10.3390/diagnostics11030551

AMA Style

Boyd C, Brown G, Kleinig T, Dawson J, McDonnell MD, Jenkinson M, Bezak E. Machine Learning Quantitation of Cardiovascular and Cerebrovascular Disease: A Systematic Review of Clinical Applications. Diagnostics. 2021; 11(3):551. https://doi.org/10.3390/diagnostics11030551

Chicago/Turabian Style

Boyd, Chris, Greg Brown, Timothy Kleinig, Joseph Dawson, Mark D. McDonnell, Mark Jenkinson, and Eva Bezak. 2021. "Machine Learning Quantitation of Cardiovascular and Cerebrovascular Disease: A Systematic Review of Clinical Applications" Diagnostics 11, no. 3: 551. https://doi.org/10.3390/diagnostics11030551

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop