Next Article in Journal
Systems Approaches to Treatment Response to Imatinib in Severe Asthma: A Pilot Study
Previous Article in Journal
Gut Microbiota and Non-Alcoholic Fatty Liver Disease Severity in Type 2 Diabetes Patients
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JPM
Volume 11
Issue 4
10.3390/jpm11040239
jpm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multi-View Convolutional Neural Networks in Rupture Risk Assessment of Small, Unruptured Intracranial Aneurysms

by
Jun Hyong Ahn
1,†,
Heung Cheol Kim
2,†,
Jong Kook Rhim
3,
Jeong Jin Park
4,
Dick Sigmund
5,
Min Chan Park
5,
Jae Hoon Jeong
5 and
Jin Pyeong Jeon
1,6,*
1
Department of Neurosurgery, College of Medicine, Hallym University, Chuncheon 24252, Korea
2
Department of Radioilogy, College of Medicine, Hallym University, Chuncheon 24252, Korea
3
Department of Neurosurgery, College of Medicine, Jeju National University, Jeju 63243, Korea
4
Department of Neurology, Konkuk University Medical Center, Seoul 05030, Korea
5
AIDOT Inc., Seoul 05854, Korea
6
Genetic and Research Inc., Chuncheon 24252, Korea
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
J. Pers. Med. 2021, 11(4), 239; https://doi.org/10.3390/jpm11040239
Submission received: 1 February 2021 / Revised: 17 March 2021 / Accepted: 23 March 2021 / Published: 24 March 2021
Browse Figures
Review Reports Versions Notes

Abstract

:
Auto-detection of cerebral aneurysms via convolutional neural network (CNN) is being increasingly reported. However, few studies to date have accurately predicted the risk, but not the diagnosis itself. We developed a multi-view CNN for the prediction of rupture risk involving small unruptured intracranial aneurysms (UIAs) based on three-dimensional (3D) digital subtraction angiography (DSA). The performance of a multi-view CNN-ResNet50 in accurately predicting the rupture risk (high vs. non-high) of UIAs in the anterior circulation measuring less than 7 mm in size was compared with various CNN architectures (AlexNet and VGG16), with similar type but different layers (ResNet101 and ResNet152), and single image-based CNN (single-view ResNet50). The sensitivity, specificity, and overall accuracy of risk prediction were estimated and compared according to CNN architecture. The study included 364 UIAs in training and 93 in test datasets. A multi-view CNN-ResNet50 exhibited a sensitivity of 81.82 (66.76–91.29)%, a specificity of 81.63 (67.50–90.76)%, and an overall accuracy of 81.72 (66.98–90.92)% for risk prediction. AlexNet, VGG16, ResNet101, ResNet152, and single-view CNN-ResNet50 showed similar specificity. However, the sensitivity and overall accuracy were decreased (AlexNet, 63.64% and 76.34%; VGG16, 68.18% and 74.19%; ResNet101, 68.18% and 73.12%; ResNet152, 54.55% and 72.04%; and single-view CNN-ResNet50, 50.00% and 64.52%) compared with multi-view CNN-ResNet50. Regarding F1 score, it was the highest in multi-view CNN-ResNet50 (80.90 (67.29–91.81)%). Our study suggests that multi-view CNN-ResNet50 may be feasible to assess the rupture risk in small-sized UIAs.

1. Introduction

Intracranial aneurysms (IAs) have been automatically detected using various algorithms based on magnetic resonance angiography (MRA) or computed tomography angiography (CTA). Previous works in the field have been successful at diagnosing and detecting aneurysms [1,2]. It is important to decide whether or not to treat unruptured intracranial aneurysms (UIAs) as well as automatically detect aneurysms clinically. Although UIAs are associated with a relatively lower rupture rate of less than 2%, ruptured aneurysms increase the mortality rate above 50% within the first six months, and morbidity is a major issue [3,4,5]. Moreover, higher rates of adverse effects (10%) after the treatment [6,7] also interfere with the treatment of UIAs, in particular, asymptomatic aneurysms before rupture. Therefore, it may be more helpful to select UIAs indicated for treatment based on risk classification using machine learning algorithms rather than simple automatic diagnosis [8].
Previously, we introduced single-view CNN for diagnosing aneurysm rupture (vs. UIA) based on three-dimensional (3D) digital subtraction angiography (DSA), which had a sensitivity of 78.76%, a specificity of 72.15%, and an overall diagnostic accuracy of 76.84% [5]. However, risk prediction based on a single input image may have limited value in clinical settings because physicians acquire multiple 3D-DSA images in multiple directions to determine the treatment plan, instead of single images. Moreover, in the previous study, there was no study that evaluated the risk of rupture in only UIAs. Multi-view CNN allows clinicians to draw conclusions based on various scanned images in multiple directions. Accordingly, compared with single-view CNN, multi-view CNN may be more useful in treatment decision-making for UIAs. Here, we propose a multi-view CNN in an effort to predict the rupture risk (high vs. non-high) of small-sized UIAs in the anterior circulation using 3D-DSA images.

2. Materials and Methods

2.1. Datasets

In this study, 3D-DSA images were used to develop the CNN algorithm. The images were acquired consecutively at three hospitals. Two datasets were prepared from the acquired images of training and test datasets. The training dataset included UIAs from January 2012 to December 2017, and the test dataset included those acquired from January 2018 to December 2019. Inclusion criteria were (1) adult patients more than 18 years old; (2) saccular aneurysm; (3) anterior circulation aneurysm; and (4) small aneurysms less than 7 mm in maximal diameter [5]. We excluded (1) fusiform and dissecting aneurysms; (2) traumatic aneurysms; (3) posterior circulation aneurysms; and (4) treated aneurysms by clipping or coiling.
A total of 457 UIAs consisting of 364 UIAs of the training dataset and 93 UIAs of the test dataset were included. Baseline characteristics of the two datasets are presented in Table 1. The size of the aneurysms was 5.2 ± 1.2 mm in the training dataset and 5.3 ± 1.3 mm in the test dataset. The UIAs in the training dataset were located in anterior cerebral artery (ACA) (n = 78, 21.4%), middle cerebral artery (MCA) (n = 115, 31.6%), and internal carotid artery (ICA) (n = 171, 47.0%). The distribution of the UIAs in the test dataset is as follows: ACA, n = 15 (16.1%); MCA, n = 40 (43.0%); and ICA, n = 38 (40.9%). The number of high-risk UIAs included 133 (36.5%) in the training and 44 (47.3%) in the test datasets. A total of 251 and 206 images were obtained from Siemens Healthcare and Philips Medical System, respectively.
We developed an automatic risk classification system for small-sized UIAs based on representative images with six directions acquired with 3D-DSA [5]. High-risk aneurysms were defined as (1) those with an irregular aneurysm wall with small bleb(s); (2) secondary aneurysms protruding from the fundus of saccular aneurysm; (3) bi- or multilobular aneurysms; and (4) those with an aspect ratio ≥ 1.6 [9,10] (Supplemental Figure S1). Risk interpretation of the UIAs was conducted by two blinded readers (KHC and JPG). Disagreements were resolved by the third reviewer (JKL) (Supplemental Data). Image acquisition and post-image processing are presented in the Supplemental Data. Briefly, DSA procedures were conducted with the Axiom Artis Zee (Siemens Healthcare, Erlangen, Germany) or the Allura Xper FD 20/20 (Philips Medical Systems, Best, The Netherlands) with standard injection protocols as described previously [5]. Post-processing of the 3D-DSA was conducted in an independent workstation equipped with InSpace 3D software [11,12,13]. The study design was listed on Clinical Research Information Service (registration number KCT0005084) prior to initiation of the study.

2.2. Multi-View Convolutional Neural Networks

We used a multi-view CNN architecture to assess the rupture risk in small-sized UIAs. Regions of interest (ROIs) including the aneurysms were selected by the neurosurgeon or neuroradiologist, extracted, and the cropped ROI was entered in parallel as input for the six neural networks, followed by convolution and pooling of the layers in each neural network. The multi-view CNN received six three-channel 224 ××× 224-sized input images, extracted from each different ROI view [5]. In the end, the pooled view was used to merge all six neural networks into a single one (Figure 1 and Supplemental Figure S2). More specifically, each view model in the multi-view CNN was initialized with the weights pre-trained from single-view CNN after feature extraction. During the multi-view CNN training, each view model weight was frozen, whereas the pooled layer view was fine-tuned. Hyper-parameters are combinations of values that are defined for training on CNN. The learning rate scheduler was applied. The learning rate varies depending on the setting of the learning rate scheduler. The learning rate warm-up was 5 epochs, and cosine decay was introduced after reaching the maximum learning rate. The total epoch was set to 100, and the cross-entropy function was selected as the loss function. Adam was used for the optimizer [14]. Additionally, all weights were initialized with ImageNet weights [15]. The batch size was set as 32. All training in the multi-view CNN was conducted with each aneurysm and not each image. Finally, label smoothing was added for the training [16].
ResNet50, AlexNet, and VGG16 were used as the components of multi-view CNN architecture for risk classification. ResNet101 and ResNet152 were then compared with ResNet50. Additionally, the diagnostic performance of multi-view CNN-ResNet50 was compared with that of single-view CNN-ResNet50. Pytorch was used as the main training framework [17].

2.3. Statistical Analysis

Descriptive analysis is presented as the numbers of subjects (percentage) for discrete and categorical variables and mean with standard deviation (SD). Two-by-two tables were generated to assess sensitivity, specificity, and overall accuracy [11]. The degree of agreement between the two readers was assessed using the k test (Supplemental Data) [14]. Confidence interval (CI) estimation was performed using the binomial proportions confidence interval, specifically the Clopper–Pearson interval, in which the success probability was estimated by the total number of trials and the number of successful trials based on the cumulative probabilities of the binomial distribution. A p-value < 0.05 was considered statistically significant. The analysis was conducted using SPSS version 19 (IBM, Armonk, NY, USA) and MedCalc (www.Medcalc.org, accessed on 1 February 2021).

3. Results

3.1. ResNet50 vs. AlexNet vs. VGG16

Comparative analyses of three different multi-CNN models of ResNet50, AlexNet, and VGG16 were performed. During the training, we achieved over 99% accuracy (Supplemental Figure S1). For the test, we evaluated every single model from each epoch, and each model in 93 UIAs to identify the best model (Table 2). Multi-view CNN-ResNet50 demonstrated a sensitivity of 81.82 (66.76–91.29)%, a specificity of 81.63 (67.50–90.76)%, and an overall accuracy of 81.72 (66.98–90.92)% (Table 3 and Figure 2). ResNet50 exhibited superior sensitivity in terms of risk prediction compared with other types of CNNs such as AlexNet (63.64 (47.74–77.17)%) and VGG16 (68.18 (52.29–80.93)%), but similar specificity. In terms of overall accuracy, ResNet50 was better than AlexNet (76.34 (62.31–88.19)%) and VGG16 (74.19 (58.93–85.60)%).

3.2. ResNet50 vs. ResNet101 vs. ResNet152

We compared the diagnostic performance of ResNet50 with the same type but different layer of ResNet101 and ResNet152. Detailed configurations of the three ResNets are presented in Table 4. ResNet50 was superior to ResNet101 (68.18 (52.29–80.93)%) and ResNet152 (54.55 (39.00–69.31)%) after the 50th epoch. ResNet50 had higher overall accuracy than ResNet101 (73.12 (57.71–84.66)%)) and ResNet152 (72.04 (58.18–85.68)%) (Table 3 and Figure 2). ResNet50 (80.90 (67.29–91.81)%) also exhibited higher F1 score than ResNet101 (70.59 (55.42–84.28)%) and ResNet152 (64.86 (47.46–79.79)%).

3.3. Multi-View CNN-ResNet50 vs. Single-View CNN-ResNet50

We further compared the diagnostic performance of multi-view CNN-ResNet50 and single-view CNN-ResNet50. Single-view CNN-ResNet50 showed a sensitivity of 50.00 (34.79–65.21)%, a specificity of 77.55 (63.01–87.75)%, and an overall accuracy of 64.52 (48.93–78.45)%, which were lower than those of multi-view CNN-ResNet50.

4. Discussion

To the best of our knowledge, this is the first study using multi-view CNN for the prediction of UIA risk. Multi-view CNN-ResNet50 demonstrated a sensitivity of 81.82%, a specificity of 81.63%, overall accuracy of 81.72%, and an F1 score of 80.90% for the classification of rupture risk involving small-sized UIAs.
Various machine learning algorithms have been increasingly implemented in disease diagnosis and predicting disease risk. Previous machine learning algorithms mainly focused on the auto-detection of cerebral aneurysms, but not risk prediction [1,18,19,20,21,22,23]. Based on MIP images acquired via MRA, CNN yielded superior detection rates (>90%) of aneurysm [1,19]. Recently, a two-stage CNN consisting of region localization and aneurysm detection was introduced for the auto-detection of aneurysms [24]. The regional average grayscale suppression was used to differentiate ROI of aneurysm and enlargement area [24]. Hu et al. [25] used Bayesian optimization for aneurysm detection based on DSA images, showing a sensitivity of 96.4% with a false-positive rate of 6.2%. However, in most cases in daily practice, DSA is additionally performed to acquire detailed information of aneurysm including relationship with nearby arteries to decide the treatment plan [26], but not the diagnosis of aneurysm itself [5]. Therefore, CNN based on DSA images can be used more frequently in clinical practice for the prediction of UIA risk, but not auto-detection. Kim et al. [5] proposed single-view CNN for differentiating ruptured aneurysm from UIA using DSA images based on AlexNet_v2 architecture. A single input image after data preprocessing using a histogram-oriented gradient resulted in two output parameters of ruptured aneurysm or UIA. Single-view CNN yielded better diagnostic accuracy than human evaluators based on the AUROC difference of 0.163 (p < 0.001) [5]. Nonetheless, its clinical usefulness is limited because the rupture risk of an aneurysm is evaluated by the clinician based on multiple images acquired simultaneously, and not each individual image. Therefore, we built six independent neural networks for six views in parallel and were trained separately in the front layers to evaluate the advantage of multi-view CNN for prediction of the risk of aneurysm rupture in patients with small-sized UIAs. Subsequently, the six individual models were combined together by pooling the views in the last layers.
Since 2012, most CNN algorithms have been evaluated using the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). AlexNet, VGG16, and ResNet50 were regarded as important CNN architectures and milestones (Table 4). As the 2012 ILSVRC winner, AlexNet represents a base architecture for various CNN models [27]. VGG16, which is ranked 2nd in 2014 ILSVRC, uses a different approach, with a reduced filter size [28]. Compared with AlexNet’s parallel architectures, VGG16 adopted a single architecture with smaller 3 × 3 filters, which reduced the computational cost and time. ResNet, ranked 1st in the 2015 ILSVRC, provided a completely different solution to the classification task, and suggested a residual block, with superior performance compared with others [29,30,31]. Compared with the plane layers, ResNet’s residual block included the identity block that preserves the existing information. As a result, both existing and newly trained information contributed to the total performance. ResNet can be differentiated into ResNet50, ResNet101, and ResNet152 depending on the number of layers. Additional layers require more computation and time. In many cases, image resolution is enhanced with architectures comprising a large number of layers, e.g., ResNet152. However, small architectures such as ResNet50 yielded better results involving low-resolution images. In our study, the average resolution of the cropped aneurysm was quite low, and therefore, ResNet50 predicted the rupture risk more accurately than ResNet101 and ResNet152 in small-sized UIAs. The superiority of ResNet50 compared with ResNet101 and ResNet152 can also be attributed to fewer trainable parameters in ResNet50. In our fine-grained classification task, ResNet50 with 23 million parameters could be optimized more effectively than ResNet152 with 60 million parameters.
The edges in 3D dimensions and roughness of the aneurysm surface are indicators for the assessment of rupture risk as well as aneurysm shape. As a main algorithm, we acquired six independent neural networks for six views in parallel, and they were trained separately in the front layers and combined via view pooling in the final layers. Clearer features were extracted in the front layers using this method, and the relationships between views were trained in the end as well. In short, both similar feature extractions and the relationship between the views enhanced accurate prediction of rupture risk using our multi-view CNN. Accordingly, multi-view CNN can be fine-tuned using pre-trained weights resulting in higher precision of risk prediction involving small-sized UIAs compared with single-view CNN. Kang et al. [32] also demonstrated higher accuracy of lung nodule classification using 3D multi-view CNN compared with single-view CNN. In addition, it is possible to apply our CNN system for rupture risk regardless of angiography machine due to post-processing of images via histogram equalization. Therefore, multi-view CNN will be more useful for clinicians who are less experienced in deciding the treatment plan for patients with UIAs.
In this study, we only included anterior circulation aneurysms measuring less than 7 mm in size. Tykocki et al. [33] reported that morphological factors such as parent artery size and aspect ratio varied between anterior and posterior circulation aneurysms. Usually, UIAs greater than 7 mm are indicated for treatment due to the risk of rupture [34,35]. However, the treatment policy for an aneurysm less than 7 mm in size is disputed. According to the second international study of unruptured intracranial aneurysms (ISUIA), small aneurysms measuring less than 7 mm without previous subarachnoid hemorrhage (SAH) are associated with 0% cumulative rupture rate over a period of 5 years [36,37]. Nevertheless, a small UIA can rupture during the follow-up period. Suzuki et al. [38] reported that bifurcated aneurysms and UIAs harboring blebs were associated with future rupture. Therefore, it may be helpful to selectively treat UIAs with a high risk of rupture in the future via CNN. Compared with other machine learning techniques, CNN is substantially more amenable to medical image processing and classification tasks, as well as easy customization for specific tasks. Our study demonstrated that ResNet50-based multi-view CNN had higher accuracy than AlexNet, VGG16, ResNet101, ResNet152, or single-view CNN.
The study limitations are as follows. First, the study included only small-sized UIAs in the anterior circulation. Morphometric variables, such as aspect ratio and parent artery size, which are associated with aneurysm rupture, vary according to aneurysm location and size [5]. Accordingly, the evaluation of an algorithm for rupture risk classification involving posterior circulation aneurysm is further required. Second, images acquired under specific conditions such as concomitant vascular disease or cerebral angiograms from General Electric Healthcare were not included. Thus, there may be some limitations in terms of real-world clinical effectiveness of our multi-view CNN. Third, no quantitative hemodynamic parameters were considered. In addition to morphological characteristics, hemodynamic factors such as wall share stress (WSS), oscillatory shear index, WSS gradient, or spatial WSS gradient are associated with aneurysm rupture. Although Chen et al. [39] did not report that machine learning was better than conventional logistic regression in predicting UIA rupture, a model is needed based on morphological and hemodynamic features in subsequent studies using artificial intelligence approaches. Fourth, we used histogram equalization rather than vesselness filtering. The main purpose of the histogram equalization applied in this study was to normalize the images obtained from two different angiography machines. In addition, we aimed to identify the risk of aneurysm rupture via the pattern and shape of the aneurysm, not the vessel itself. In future work, we will continue to explore other filtering methods to normalize images from different types of angiography machines as well as vesselness filtering to increase the diagnostic accuracy for high-risk aneurysms. Finally, we applied a user-selected ROI rather than automated diagnosis around the UIAs in DSA images for risk prediction. Clinically, DSA is generally used to decide treatment strategies rather than investigate the diagnosis [5]. Nonetheless, a study analyzing risk prediction of UIA based on automated detection is necessary to improve the clinical effectiveness of the CNN system.

5. Conclusions

We proposed a new multi-view CNN to evaluate rupture risk in small UIAs based on 3D-DSA images. The diagnostic performance of multi-view ResNet50 was better than that of the other types of CNNs. An external study is required to further corroborate our results before practical, clinical, and commercial applications can be envisaged.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jpm11040239/s1. Figure S1: Examples of rupture risk (high vs. non-high) involving small unruptured intracranial aneurysms. Figure S2: Post-image processing for uniform image conversion via histogram equalization based on different three-dimensional digital subtraction angiography data. Figure S3: Sensitivity of rupture risk differentiation (high vs. non-high) involving small UIAs in the training cohort using multi-view ResNet50. Methods: The process of risk stratification and inter-assessor agreements, image acquisition and post-image processing, detailed image acquisition protocol of 3D-DSA.

Author Contributions

Conceptualization, J.P.J. and H.C.K.; data curation, J.H.A.; J.K.R.; and J.J.P.; formal analysis, D.S.; M.C.P.; and J.H.J.; funding acquisition, J.P.J. and J.H.J.; methodology, D.S. and M.C.P.; supervision, J.P.J.; visualization, M.C.P.; writing—original draft, J.H.A. and J.P.J.; writing—review and editing, H.C.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Hallym University Research Fund (HURF-2019-22) and the Korea Technology and information Promotion Agency for SMEs (TIPA S2924458).

Institutional Review Board Statement

This study was approved by the Institutional Review Board (No. 2018-11-008).

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nakao, T.; Hanaoka, S.; Nomura, Y.; Sato, I.; Nemoto, M.; Miki, S.; Maeda, E.; Yoshikawa, T.; Hayashi, N.; Abe, O. Deep neural network-based computer-assisted detection of cerebral aneurysms in mr angiography. J. Magn. Reson. Imaging 2018, 47, 948–953. [Google Scholar] [CrossRef]
  2. Dai, X.; Huang, L.; Qian, Y.; Xia, S.; Chong, W.; Liu, J.; Ieva, A.D.; Hou, X.; Ou, C. Deep learning for automated cerebral aneurysm detection on computed tomography images. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 715–723. [Google Scholar] [CrossRef]
  3. Lantigua, H.; Ortega-Gutierrez, S.; Schmidt, J.M.; Lee, K.; Badjatia, N.; Agarwal, S.; Claassen, J.; Connolly, E.S.; Mayer, S.A. Subarachnoid hemorrhage: Who dies, and why? Crit. Care 2015, 9, 309. [Google Scholar] [CrossRef]
  4. Bonita, R.; Thomson, S. Subarachnoid hemorrhage: Epidemiology, diagnosis, management, and outcome. Stroke 1985, 16, 591–594. [Google Scholar] [CrossRef]
  5. Kim, H.C.; Rhim, J.K.; Ahn, J.H.; Park, J.J.; Moon, J.U.; Hong, E.P.; Kim, M.R.; Kim, S.G.; Lee, S.H.; Jeong, J.H.; et al. Machine learning application for rupture risk assessment in small-sized intracranial aneurysm. J. Clin. Med. 2019, 8, 683. [Google Scholar] [CrossRef]
  6. Katati, M.J.; Santiago-Ramajo, S.; Perez-Garcia, M.; Jofré, M.M.; Vilar-Lopez, R.; Coín-Mejias, M.A.; Cara cuel-Romero, A.; Arjona-Moron, V. Description of quality of life and its predictors in patients with aneurysmal subarachnoid hemorrhage. Cerebrovasc. Dis. 2007, 24, 66–73. [Google Scholar] [CrossRef]
  7. Johnston, S.C.; Zhao, S.; Dudley, R.A.; Berman, M.F.; Gress, D.R. Treatment of unruptured cerebral aneu rysms in california. Stroke 2001, 32, 597–605. [Google Scholar] [CrossRef] [PubMed]
  8. Liang, L.; Liu, M.; Martin, C.; Elefteriades, J.A.; Sun, W. A machine learning approach to investigate the rela tionship between shape features and numerically predicted risk of ascending aortic aneurysm. Biomech. Model. Mechanobiol. 2017, 16, 1519–1533. [Google Scholar] [CrossRef] [PubMed]
  9. Lindgren, A.E.; Koivisto, T.; Bjorkman, J.; Fraunberg, M.U.Z.; Helin, K.; Jääskeläinen, J.E.; Frösen, J. Irregular shape of intracranial aneurysm indicates rupture risk irrespective of size in a population-based cohort. Stroke 2016, 47, 1219–1226. [Google Scholar] [CrossRef] [PubMed]
  10. Huang, Z.Q.; Meng, Z.H.; Hou, Z.J.; Huang, S.; Chen, J.; Yu, H.; Feng, L.; Wang, Q.; Li, P.; Wen, Z. Geomet ric parameter analysis of ruptured and unruptured aneurysms in patients with symmetric bilateral intracranial aneurysms: A multicenter ct angiography study. AJNR Am. J. Neuroradiol. 2016, 37, 1413–1417. [Google Scholar] [CrossRef]
  11. Jeon, J.S.; Sheen, S.H.; Hwang, G.J.; Kim, H.C.; Kwon, B.J. Feasibility of intravenous flat panel detector ct angiography for intracranial arterial stenosis. AJNR Am. J. Neuroradiol. 2013, 34, 129–134. [Google Scholar] [CrossRef]
  12. Jou, L.D.; Mohamed, A.; Lee, D.H.; Mawad, M.E. 3d rotational digital subtraction angiography may under estimate intracranial aneurysms: Findings from two basilar aneurysms. AJNR Am. J. Neuroradiol. 2007, 28, 1690–1692. [Google Scholar] [CrossRef]
  13. Lu, L.; Zhang, L.J.; Poon, C.S.; Wu, S.Y.; Zhou, C.S.; Luo, S.; Wang, M.; Lu, G.M. Digital subtraction ct angi ography for detection of intracranial aneurysms: Comparison with three-dimensional digital subtraction angiography. Radiology 2012, 262, 605–612. [Google Scholar] [CrossRef]
  14. Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 2015 International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015; pp. 1–15. [Google Scholar]
  15. Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
  16. Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 2818–2826. [Google Scholar]
  17. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. Pytorch: An imperative style, high-performance deep learning library. arXiv 2019, arXiv:1912.01703. [Google Scholar]
  18. Cebral, J.R.; Mut, F.; Weir, J.; Putman, C.M. Association of hemodynamic characteristics and cerebral aneurysm rupture. AJNR Am. J. Neuroradiol. 2011, 32, 264–270. [Google Scholar] [CrossRef]
  19. Stember, J.N.; Chang, P.; Stember, D.M.; Liu, M.; Grinband, J.; Filippi, C.G.; Meyers, P.; Jambawalikar, S. Con volutional neural networks for the detection and measurement of cerebral aneurysms on magnetic resonance angiography. J. Digit. Imaging 2019, 32, 808–815. [Google Scholar] [CrossRef]
  20. Joo, B.; Ahn, S.S.; Yoon, P.H.; Sohi, B.; Sohn, B.; Lee, Y.E.; Bae, J.H.; Park, M.S.; Choi, H.S.; Lee, S. A deep learning algorithm may automate intracranial aneurysm detection on mr angiography with high diagnostic performance. Eur. Radiol. 2020, 30, 5785–5793. [Google Scholar] [CrossRef]
  21. Ueda, D.; Yamamoto, A.; Nishimori, M.; Shimono, T.; Doishita, S.; Shimazaki, A.; Katayama, Y.; Fukumoto, S.; Choppin, A.; Shimahara, Y.; et al. Deep learning for mr angiography: Automated detection of cerebral aneurysms. Radiology 2019, 290, 187–194. [Google Scholar] [CrossRef]
  22. Yang, J.; Xie, M.; Hu, C.; Alwalid, O.; Xu, Y.; Liu, J.; Jin, T.; Li, C.; Tu, D.; Liu, X.; et al. Deep learning for detecting cerebral aneurysms with ct angiography. Radiology 2021, 298, 155–163. [Google Scholar] [CrossRef] [PubMed]
  23. Kallmes, D.F.; Erickson, B.J. Automated Aneurysm Detection: Emerging from the Shallow End of the Deep Learning Pool. Radiology 2021, 298, 164–165. [Google Scholar] [CrossRef] [PubMed]
  24. Duan, H.; Huang, Y.; Liu, L.; Dai, H.; Chen, L.; Zhou, L. Automatic detection on intracranial aneurysm from digital subtraction angiography with cascade convolutional neural networks. Biomed. Eng. Online 2019, 18, 110. [Google Scholar] [CrossRef] [PubMed]
  25. Hu, T.; Yang, H.; Ni, W.; Lei, Y.; Jiang, Z.; Shi, K.; Yu, J.; Gu, Y.; Wang, Y. Automatic detection of intracranial aneurysms in 3d-dsa based on a bayesian optimized filter. Biomed. Eng. Online 2020, 19, 73. [Google Scholar] [CrossRef] [PubMed]
  26. Sugahara, T.; Korogi, Y.; Nakashima, K.; Hamatake, S.; Honda, S.; Takahashi, M. Comparison of 2d and 3d digital subtraction angiography in evaluation of intracranial aneurysms. AJNR Am. J. Neuroradiol. 2002, 23, 1545–1552. [Google Scholar] [PubMed]
  27. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Proceedings of the 2012 Advances in Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA, 3–8 December 2012; pp. 1097–1105. [Google Scholar]
  28. Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 2015 International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
  29. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar]
  30. Abrol, A.; Bhattarai, M.; Fedorov, A.; Du, Y.; Plis, S.; Calhoun, V. Deep residual learning for neuroimaging: An application to predict progression to alzheimer’s disease. J. Neurosci. Methods 2020, 339, 108701. [Google Scholar] [CrossRef]
  31. Ramzan, F.; Khan, M.U.G.; Rehmat, A.; Iqbal, S.; Saba, T.; Rehman, A.; Mehmood, Z. A deep learning ap proach for automated diagnosis and multi-class classification of alzheimer’s disease stages using resting-state fmri and residual neural networks. J. Med. Syst. 2019, 44, 37. [Google Scholar] [CrossRef]
  32. Kang, G.; Liu, K.; Hou, B.; Zhang, N. 3D multi-view convolutional neural networks for lung nodule classifica tion. PLoS ONE 2017, 12, e0188290. [Google Scholar] [CrossRef]
  33. Tykocki, T.; Kostkiewicz, B. Aneurysms of the anterior and posterior cerebral circulation: Comparison of the morphometric features. Acta Neurochir. 2014, 156, 1647–1654. [Google Scholar] [CrossRef]
  34. Williams, L.N.; Brown, R.D., Jr. Management of unruptured intracranial aneurysms. Neurol. Clin. Pract. 2013, 3, 99–108. [Google Scholar] [CrossRef] [PubMed]
  35. Investigators, U.J.; Morita, A.; Kirino, T.; Hashi, K.; Aoki, N.; Fukuhara, S.; Hashimoto, N.; Nakayama, T.; Sakai, M.; Teramoto, A.; et al. The natural course of unruptured cerebral aneurysms in a Japanese cohort. N. Engl. J. Med. 2012, 366, 2474–2482. [Google Scholar] [CrossRef]
  36. Wiebers, D.O.; Whisnant, J.P.; Huston, J., 3rd; Meissner, I.; Brown, R.D., Jr.; Forbes, P.G.S.; Thielen, K.; Nichols, D.; O’Fallon, W.M.; Peacock, J.; et al. Unruptured intracranial aneurysms: Natural history, clinical outcome, and risks of surgical and endovascular treatment. Lancet 2003, 362, 103–110. [Google Scholar] [CrossRef]
  37. Dolati, P.; Pittman, D.; Morrish, W.F.; Wong, J.; Sutherland, G.R. The frequency of subarachnoid hemorrhage from very small cerebral aneurysms (<5 mm): A population-based study. Cureus 2015, 7, e279. [Google Scholar] [PubMed]
  38. Suzuki, T.; Takao, H.; Rapaka, S.; Fujimura, S.; Nita, C.I.; Uchiyama, Y.; Ohno, H.; Otani, K.; Dahmani, C.; Mihalef, V.; et al. Rupture risk of small unruptured intracranial aneurysms in japanese adults. Stroke 2020, 51, 641–643. [Google Scholar] [CrossRef] [PubMed]
  39. Chen, G.; Lu, M.; Shi, Z.; Xia, S.; Ren, Y.; Liu, Z.; Liu, X.; Li, Z.; Mao, L.; Li, X.L.; et al. Development and validation of machine learning prediction model based on computed tomography angiography-derived hemodynamics for rupture status of intracranial aneurysms: A chinese multicenter study. Eur. Radiol. 2020, 30, 5170–5182. [Google Scholar] [CrossRef] [PubMed]
Jpm 11 00239 g001 550
Figure 1. Overview of our proposed multi-view CNN architecture and simplified example of the convolution and pooling layers.
Figure 1. Overview of our proposed multi-view CNN architecture and simplified example of the convolution and pooling layers.
Jpm 11 00239 g001
Jpm 11 00239 g002 550
Figure 2. Comparative analyses of multi-view CNN-ResNet50 for predicting rupture risk type involving small UIAs under various conditions: other types of AlexNet and VGG16 (A,B), same type but different layers (ResNet101 and ResNet152) (C,D), and single-view CNN (E,F).
Figure 2. Comparative analyses of multi-view CNN-ResNet50 for predicting rupture risk type involving small UIAs under various conditions: other types of AlexNet and VGG16 (A,B), same type but different layers (ResNet101 and ResNet152) (C,D), and single-view CNN (E,F).
Jpm 11 00239 g002
Table
Table 1. Clinical and radiological characteristics: training vs. test datasets.
Table 1. Clinical and radiological characteristics: training vs. test datasets.
VariablesTraining Dataset (n = 364)Test Dataset (n = 93)
Clinical findings
Female206 (56.6%)54 (58.1%)
Age, years57.2 ± 14.5 56.8 ± 15.3
Hypertension97 (26.7%)28 (30.1%)
Diabetes mellitus35 (9.6%)10 (10.8%)
Hyperlipidemia42 (11.5%)10 (10.8%)
Smoking48 (13.2%)12 (12.9%)
Radiologic findings
High-risk UIA 133 (36.5%)44 (47.3%)
Size (mm)5.2 ± 1.25.3 ± 1.3
Aneurysm location
Anterior cerebral artery78 (21.4%)15 (16.1%)
Middle cerebral artery115 (31.6%)40 (43.0%)
Internal carotid artery 171 (47.0%)38 (40.9%)
Imaging platform
Siemens Healthcare210 (57.7%)41 (44.1%)
Philips Medical Systems154 (42.3%)52 (55.9%)
Table
Table 2. Accuracy of multi-view CNN-ResNet50 for the prediction of rupture.
Table 2. Accuracy of multi-view CNN-ResNet50 for the prediction of rupture.
CNNResNet50 Classification
Diagnosis High RiskNon-High RiskTotal
High risk36844
Non-high risk94049
Total454893
Table
Table 3. Binary classification of rupture risk involving small UIAs in the test dataset using various CNN architectures. CI, confidence interval.
Table 3. Binary classification of rupture risk involving small UIAs in the test dataset using various CNN architectures. CI, confidence interval.
TypeModelsSensitivity (95% CI)Specificity (95% CI)Overall Accuracy (95% CI)F1 Score (95% CI)
Multi-viewResNet5081.82 (66.76–91.2)%81.63 (67.50–90.76)%81.72 (66.98–90.92)%80.90 (67.29–91.81)%
Multi-viewAlexNet63.64 (47.74–77.17)%87.76 (74.54–94.92)%76.34 (62.31–88.19)%71.79 (55.13–85.00)%
Multi-viewVGG1668.18 (52.29–80.93)%79.59 (65.24–89.28)%74.19 (58.93–85.60)%71.43 (55.42–84.28)%
Multi-viewResNet10168.18 (52.29–80.93)%77.55 (63.01–87.75)%73.12 (57.71–84.66)%70.59 (55.42–84.28)%
Multi-viewResNet15254.55 (39.00–69.31)%87.76 (74.54–94.92)%72.04 (58.18–85.68)%64.86 (47.46–79.79)%
Single-viewResNet5050.00 (34.79–65.21)%77.55 (63.01–87.75)%64.52 (48.93–78.45)%57.14 (40.82–73.69)%
Table
Table 4. Detailed configuration of ResNet50, ResNet101, ResNet152, AlexNet, and VGG16. Convolution and pooling layers of ResNet block 2, 3, 4, and 5 were performed with a stride of 2.
Table 4. Detailed configuration of ResNet50, ResNet101, ResNet152, AlexNet, and VGG16. Convolution and pooling layers of ResNet block 2, 3, 4, and 5 were performed with a stride of 2.
BlockOutput SizeResNet50ResNet101ResNet152VGG16Output SizeAlexNet
1 112   × 1127 × 7, 64, stride 27 × 7, 64, stride 27 × 7, 64, stride 2 3 × 3 ,   64   × 2 55   × 5511 × 11, 96, stride 4
2 56   × 563 × 3 max-pooling, stride 23 × 3 max-pooling, stride 23 × 3 max-pooling, stride 22 × 2 max-pooling, stride 2 27   × 273 × 3 max-pooling, stride 2
  1 × 1 ,   64 3 × 3 ,   64 1 × 1 ,   256   × 3   1 × 1 ,   64 3 × 3 ,   64 1 × 1 ,   256   × 3   1 × 1 ,   64 3 × 3 ,   64 1 × 1 ,   256   × 3   3 × 3 ,   128   × 2 5 × 5, 256, stride 1
3 28   × 28   1 × 1 ,   128 3 × 3 ,   128 1 × 1 ,   512   × 4   1 × 1 ,   128 3 × 3 ,   128 1 × 1 ,   512   × 4   1 × 1 ,   128 3 × 3 ,   128 1 × 1 ,   512   × 8 2 × 2 max-pooling, stride 2 13   × 133 × 3 max-pooling, stride 2
  3 × 3 ,   256   × 3   3 × 3 ,   384 3 × 3 ,   384 3 × 3 ,   256   , stride 1
4 14   × 14   1 × 1 ,   256 3 × 3 ,   256 1 × 1 ,   1024   × 6   1 × 1 ,   256 3 × 3 ,   256 1 × 1 ,   1024   × 23   1 × 1 ,   256 3 × 3 ,   256 1 × 1 ,   1024   × 36 2 × 2 max-pooling, stride 2 1   × 13 × 3 max-pooling, stride 2
average pooling
  3 × 3 ,   512   × 3 [4096 fully connected,
ReLU ]   × 2
n-class fully connected, softmax
5 7   × 7   1 × 1 ,   512 3 × 3 ,   512 1 × 1 ,   2048   × 3   1 × 1 ,   512 3 × 3 ,   512 1 × 1 ,   2048   × 3   1 × 1 ,   512 3 × 3 ,   512 1 × 1 ,   2048   × 3   3 × 3 ,   512   × 3
6 1   × 1average-pooling2 × 2 max-pooling, stride 2
[4096 fully connected, ReLU] × 2
n-class fully connected, softmaxn-class fully connected, softmax
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ahn, J.H.; Kim, H.C.; Rhim, J.K.; Park, J.J.; Sigmund, D.; Park, M.C.; Jeong, J.H.; Jeon, J.P. Multi-View Convolutional Neural Networks in Rupture Risk Assessment of Small, Unruptured Intracranial Aneurysms. J. Pers. Med. 2021, 11, 239. https://doi.org/10.3390/jpm11040239

AMA Style

Ahn JH, Kim HC, Rhim JK, Park JJ, Sigmund D, Park MC, Jeong JH, Jeon JP. Multi-View Convolutional Neural Networks in Rupture Risk Assessment of Small, Unruptured Intracranial Aneurysms. Journal of Personalized Medicine. 2021; 11(4):239. https://doi.org/10.3390/jpm11040239

Chicago/Turabian Style

Ahn, Jun Hyong, Heung Cheol Kim, Jong Kook Rhim, Jeong Jin Park, Dick Sigmund, Min Chan Park, Jae Hoon Jeong, and Jin Pyeong Jeon. 2021. "Multi-View Convolutional Neural Networks in Rupture Risk Assessment of Small, Unruptured Intracranial Aneurysms" Journal of Personalized Medicine 11, no. 4: 239. https://doi.org/10.3390/jpm11040239

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop