A Convolutional Neural Network for Anterior Intra ‐ Arterial Thrombus Detection and Segmentation on Non ‐ Contrast Computed Tomography of Patients with Acute Ischemic Stroke

: The aim of this study was to develop a convolutional neural network (CNN) that automatically detects and segments intra ‐ arterial thrombi on baseline non ‐ contrast computed tomography (NCCT) scans. We retrospectively collected computed tomography (CT) ‐ scans of patients with an anterior circulation large vessel occlusion (LVO) from the Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands trial, both for training (n = 86) and validation (n = 43). For testing we included patients with (n = 58) and without (n = 45) an LVO from our comprehensive stroke center. Ground truth was established by consensus between two experts using both CT angiography and NCCT. We evaluated the CNN for correct identification of a thrombus, its location and thrombus segmentation and compared these with the results of a neurologist in training and expert neuroradiologist. Sensitivity of the CNN thrombus detection was 0.86, vs . 0.95 and 0.79 for the neuroradiologists. Specificity was 0.65 for the network vs . 0.58 and 0.82 for the neuroradiologists. The CNN correctly identified the location of the thrombus in 79% of the cases, compared to 81% and 77% for the neuroradiologists. The sensitivity and specificity for thrombus identification and the rate for correct thrombus location assessment by the CNN were similar to those of expert neuroradiologists. for (1) identification of intra ‐ arterial thrombus in NCCT of the brain, (2) thrombus location assessment, and (3) thrombus segmentation on baseline NCCT images. of a thrombus in the scan. The neuroradiologists were blinded for any clinical information and were not aware of the ratio of non ‐ thrombus versus thrombus scans. We compared the results from the CNN with those of the experts.


Introduction
The outcome of patients with an acute ischemic stroke (AIS) caused by a large vessel occlusion (LVO) of the anterior circulation has been significantly improved since the introduction of endovascular treatment (EVT) [1,2]. However, the number of patients that reach a favorable functional outcome after treatment is still low [2]. Reducing the time to treatment is currently the main goal in optimizing treatment of AIS [3][4][5].
Fast recognition of the occluding thrombus is key in diagnosis and treatment selection. Patients with symptoms of AIS are most commonly brought to the nearest primary stroke center [6,7]. In most hospitals, computed tomography (CT) is the modality of choice for AIS diagnosis because of its speed and broad availability at emergency rooms [8]. A non-contrast CT (NCCT) is initially acquired to exclude hemorrhagic stroke. An intra-arterial thrombus might already be recognized on NCCT by a hyperdense artery sign (HAS) [9][10][11]. An additional CT angiography (CTA) is normally used to determine the location of the occlusion and to evaluate the accessibility to the thrombus with EVT. If the stroke is caused by an LVO, patients might need to be transferred to a comprehensive stroke center to receive EVT. Consequently, if an LVO is missed by the radiologist, a patient might be withheld from EVT and not receive appropriate treatment.
Furthermore, delays in diagnosis and assessment of treatment eligibility reduces the chance on favorable outcome [12][13][14]. In many primary stroke centers, radiologists are less familiar in the detection of thrombus. Moreover, specialized neuroradiologists are often limitedly available during non-office hours. The automated detection and segmentation of thrombus on baseline NCCT images has the potential to reduce time to treatment and improve appropriate treatment selection.
Convolutional neural networks (CNN) have shown high potential for the automated analysis of medical images. They have been applied with high performance for multiple segmentation tasks, including brain lesion and vessel segmentation [15][16][17][18]. The aim of this study was to assess the accuracy of a CNN for (1) identification of intra-arterial thrombus in NCCT of the brain, (2) thrombus location assessment, and (3) thrombus segmentation on baseline NCCT images.

Patient Selection and Data Sets
For training and validation of the CNN we used baseline NCCT from the Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands (MR CLEAN) trial. MR CLEAN included patients from the Dutch stroke treatment centers with an LVO in the anterior circulation confirmed by CTA [19]. The training and validation set consisted of 86 and 43 NCCT images of stroke patients, respectively. The training and validation set were randomly selected and separated, such that the optimization of the hyperparameters were not influenced by the validation set and the accuracy assessment was performed solely on unseen data. We used the validation set to evaluate network performance during the optimalisation process.
The accuracy of the trained CNNs was assessed using a test set, which consisted of 58 NCCT scans with a proven LVO and 45 NCCT scans from patients with stroke mimics. For testing, we included patients with AIS or stroke mimics from our comprehensive stroke center. The imaging protocol of our test data was similar to the imaging protocol of the MR CLEAN trial [19]. The study was conducted in accordance with the Declaration of Helsinki. The medical ethical committee of each participating center approved the MR CLEAN trial (MEC-2010-041) and written informed consent was obtained for each participant. For the additional test set, the center's medical ethical committee approved the use of the anonymized datasets in this study and informed consent was waived (W19_255 # 19.307).
We only included scans with a slice thickness less than 2.5 mm and with a maximum time difference between NCCT and CTA of 30 min. We excluded scans with artefacts, excessive noise, and poor contrast enhancement. We also excluded scans of stroke patients with migration of the thrombus between NCCT and CTA acquisition.

Pre-Processing
All scans were registered to an atlas so that the NCCT and CTA for each patient were aligned and the voxel dimensions were equal for all scans. The registration ensured that the midplane of the brain coincided with the y-z plane halfway the x-direction. As a result, the hemispheres were symmetrically oriented. For the registration we used ELASTIX ® software [20].
For each patient, we obtained a brain mask using a region growing approach: First, we considered all voxels with intensity value above 160 Hounsfield units (HU) as skull. We filled all foramina within the skull by a morphological dilatation with a 7 mm radius. A seed with a 7 mm radius was placed at the center of gravity of the segmented skull to segment the brain with region growing. The region expanded until the border of the skull was reached. To assure that calcified arteries were included in the mask, we performed an additional closing operation that filled all holes that existed after the region growing step. For the brain mask, all segmentations within slices below the foramen magnum were excluded. The location of the foramen magnum was considered the first slice with a brain segmentation area below 900 mm 2 below the slice with the largest segmented brain area.

CNN Design
The thrombus detection was based on two observations: (1) the asymmetry between the hemispheres and (2) the HAS of the thrombus on NCCT images. Therefore, we developed two CNNs: an asymmetry detection network and a HAS detection network. Both networks were patch-based. The assymetry detection network consisted of two parrallel convolutional pathways, one for each hemisphere. The output of both pathways was combined and fed to the fully connected layers. The HAS network consisted of a single convolutional pathway. Patch size was 24 × 24 × 24 voxels, corresponding to 12 × 12 × 12 mm.
Both networks had a design similar to AlexNet [21] and consisted of five 3D convolutional layers with a 5 × 5 × 5 dimension followed by a Rectified Linear Unit (ReLU) layer (see Figure 1). Within the first three layers, we decreased the dimension of the feature maps and increased the number of feature maps by a factor two for each layer. Subequently, we decreased the number of feature maps by half within each subsequent layer. To reduce the chance for overfitting, we added an additional dropout after the first fully connected layer. We used the default settings of the pytorch library, which is 0.5 for fully connected layers.
The fully connected part of both networks consisted of two fully connected layers followed by ReLU. The input of the fully connected part of the HAS detection network consisted of a single output from the convolutional layers. For the assymetry detection network, we concatenated the output from the two parallel convolutional pathways before feeding it to the fully connected layers. We added an additional dropout after the first fully connected layer. The CNNs were trained and validated seperately. During training, we alternated thrombus patches with non-thrombus patches for input so that the number of thrombus and non-thrombus was balanced. Because the thrombus is very small, we only required a small number of non-thrombus patches to balance the data. As it is possible to extract an abundant number of non-thrombus patches from a single scan, we did not need additional scans to extract negative patches. During the training process, the hyperparameters of the CNN were optimized for correctly discriminating thrombus patches from non-thrombus patches.

Ground Truth
Ground truth thrombus segmentations in the MR CLEAN data have been previously obtained in a study performed by Santos et al. [22]. The ground truth thrombus segmentations for the test set was established by joint reading of two experts with >5 years of experience. Because the thrombus location is more apparent for the human eye on CTA images, we used the combination of CTA with the NCCT images to determine the ground truth thrombus location. The thrombus segmentations were performed using ITK-SNAP [23].

Classification Pipeline
To speed up processing, we performed the thrombus segmentation in two steps. For each scan, we first performed a coarse segmentation by feeding both networks with non-overlapping patches. If both networks or only the HAS detection network classified a patch within the scan as "thrombus present", we performed an additional voxel-wise segmentation for that specific patch by the same networks. Therefore, the number of patches per scan to be processed by the networks was reduced.

Network Evaluation
We evaluated the CNN for the correct identification of a thrombus, thrombus location assessment and thrombus segmentations. An expert neuroradiologist with >15 years of experience (expert 1) and a neuroradiologist in training with >4 years of experience (expert 2) performed the same task on the same test set independently: the experts evaluated the scans for presence of a thrombus and performed manual segmentations of the thrombus if they assumed the presence of a thrombus in the scan. The neuroradiologists were blinded for any clinical information and were not aware of the ratio of non-thrombus versus thrombus scans. We compared the results from the CNN with those of the experts.
We assessed the sensitivity, specificity and precision for the detection of a thrombus within a NCCT scan. A thrombus was considered as "present" if at least one voxel within the NCCT scan was classified as "thrombus". The thrombus location was considered correct if at least a single voxel of the segmentation overlapped with the ground truth segmentation. Sensitivity was defined as the ratio of thrombi correctly found "present" by the CNN or neuroradiologists and the total number of scans with truly present thrombi according to the ground truth. Specificity was defined as the ratio of thrombi correctly found "absent" by the CNN or neuroradiologists and the total number of scans with absent thrombi according to the ground truth. The accuracy of the thrombus location assessment was defined as the percentage of scans with correct thrombus location. To evaluate the thrombus segmentation of the CNN and both neuroradiologists we assessed the agreement of the thrombus volume and mean density by the computation of the two-way intraclass correlation coefficient (ICC) agreement using the 'irr' library from R CRAN repository [24] for thrombus volume and mean density in HU. The ICC of the thrombus volume assessment was computed for all patients within the test set with a proven thrombus. If a thrombus was not correctly detected, we set the volume to zero. For the ICC of the mean thrombus density, we only included the thrombus density measurements for patients with a proven thrombus for which the CNN and both neuroradiologists correctly detected the thrombus. We additionally created scatter and Bland-Altman plots to study the relation between the measurements made by network or neuroradiologists and the ground truth.

Results
The sensitivity of the CNN thrombus detection was 0.86, vs. 0.95 and 0.79 for neuroradiologist 1 and 2 respectively (Table 1). Specificity was 0.65 for the network vs. 0.58 and 0.82 for the neuroradiologists. Precision was 0.75, 0.74 and 0.81 for the CNN, neuroradiologist 1 and neuroradiologist 2, respectively. The CNN correctly identified the location of the thrombus in 79% of the cases, compared to 81% and 77% for the neuroradiologists. An example of the results is given in Figure 2. The agreement in thrombus volume was fair (ICC: 0.49) for the CNN, poor for neuroradiologist 1 (ICC: 0.37) and fair for neuroradiologist 2 (ICC: 0.55). For the thrombus density, the ICC was poor (ICC: 0.14) for the CNN and fair for neuroradiologist 1 and 2 (ICC: 0.45 and ICC: 0.40, respectively). The scatter and Bland-Altman plots for the segmented thrombus volumes and for the thrombus density are shown in Figures 3 and 4, respectively. The scatter plots show the relation of the thrombus volumes and densities that were computed from the thrombus segmentations by one of the neuroradiologists or CNN and those of the ground truth. The diagonal line within the scatter plots, represent the perfect match of the results between the neuroradiologists or CNN and ground truth. The Bland-Altman plots show the difference in measurements, comparing the results of the neuroradiologists or CNN with the ground truth. The mean and standard deviation of the differences are also shown within the plots, represented as bias and limits of agreements. The exact values of the bias and limits of agreements for thrombus volume and density, together with the other results are shown in Table 1.

Discussion
We have implemented and evaluated a CNN-based thrombus detection and segmentation approach on baseline NCCT image data. The accuracy was similar to expert neuroradiologists.
Of the two neuroradiologists who evaluated the images, one had highest sensitivity for thrombus identification and thrombus location assessment, while the other had the highest specificity for thrombus detection. In all cases, the network scored second best. The network performed better on sensitivity than specificity, which means that the network is better in detecting true positives than true negatives.
The thrombus volumetric agreement, represented by the ICC, of the segmentations of the network was similar to the agreement amongst the neuroradiologists. However, the agreement of the thrombus density as determined with the network was poor and less accurate than the density agreements of the neuroradiologists. This suggests that, even though the thrombus volume was similar, the thrombus was not segmented at the exact same location as the ground truth segmentation. Moreover, the Bland-Altman plots for the CNN show that the segmentation generated by the CNN generally caused an overestimation of the thrombus volume and underestimation of the thrombus density. The segmentations of both neuroradiologists generally underestimated the thrombus volume and overestimated the thrombus density. Overall, the results suggest that our network is less suitable for exact thrombus segmentation and should mainly be used for thrombus detection.
The evaluation of correct thrombus segmentation is challenging because of class imbalance. A common segmentation evaluation is the DICE coefficient, which quantifies the overlap between ground truth and segmentation [25]. However, because the number of background voxels is substantially larger than the number of thrombus voxels the DICE score will be strongly penalized for false positives. Because thrombus volume and density have been associated with treatment outcome [26,27], we have decided that these parameters were more appropriate evaluation parameters.
Before the presented CNN was established, we evaluated multiple CNN architectures. Hereby, we varied patch size, learning rate, batch size, dropout rate and inclusion of batch normalization. In the end, the presented CNN architecture performed best in discriminating thrombus from non-thrombus patches.
There are a few studies that describe the detection of LVO. Murray et al. published an extensive systematic review in which they summarized the available tools for LVO detection [28]. However, most of these methods do not point out where the thrombus is located. Machine learning methods, such as random forest learning and support vector machines, are used to detect LVO based on the Alberta stroke program early CT score or the detection of diffusion-perfusion mismatch from CT perfusion images. The use of these so-called black-box LVO detection is suboptimal since radiologists have to search for the thrombus location themselves after the detection of the LVO. Moreover, previous approaches detect thrombi in CTA images only, whereas the availability of contrast-enhanced imaging is suboptimal in off-hours situations [29,30].
Lisowska et. al. [31] presented a thrombus segmentation method with a similar approach. Their network performs thrombus detection based on asymmetry in NCCT images. Their network resulted in a high area under the receiver operating characteristic curve, but low precision-sensitivity area under the curve (AUC). Concretely, the network showed high performance for balanced data, but low performance in unbalanced data. This makes their network less suitable for the segmentation of entire scans.
In addition to the approach presented by Lisowska et al. we combined asymmetry detection with HAS detection. Moreover, the presented architecture of our CNNs was based on AlexNet [21], which is a well-known and validated architecture for image recognition tasks. Because our results were obtained by the joined result of both CNN's, we were not able to create an AUC similar to Lisowska et al. and directly compare the results. More specifically, since the two CNN's each have their own threshold, these two thresholds would result in a 3D type of AUC. To compare our results, we computed the Youden's J statistic for our results and made an estimation of this statistic from the AUC presented by Lisowska et al. From this we were able to conclude that our network outperformed their network. More details on this comparison can be found within the Supplementary Materials (Analysis S1).
The presented CNN has the potential to be used in clinical practice to support non-experts in thrombus detection on baseline NCCT. The detection of thrombus on NCCT has several benefits. First, it can speed up the clinical workup, as NCCT is always performed first. When the thrombus is clearly detected on NCCT, the patient can be immediately sent to a comprehensive stroke center and the intervention team can already be informed before CTA imaging is performed and analyzed. In addition, it could be beneficial for smaller community hospitals, which operate in less developed settings to detect LVOs, without the need for an additional CTA.
The CNN could be incorporated within the current workflow. The application of our network requires registration and pre-processing steps as described in the methods section. With the appropriate hardware typical analysis time is less than 2 min.
Still, the presented results showed that in about 15% of patients the thrombus is missed and approximately 35% of the detected thrombi are actual false positives. Therefore, this network is not yet accurate enough to replace experts and should be considered a support tool. Moreover, in current practice, all patients with a clinical suspicion of an LVO, require a CTA. Before such a CNN could be integrated within daily clinical practice, its value should be assessed in a prospective study to determine if the use of the CNN advances correct thrombus detection, faster treatment and reduction in workload of expert neuroradiologists.
The future availability of accurate thrombus volume segmentation may support the analysis of thrombus image characteristics in large image data sets present in randomized controlled trials. Thrombus density, perviousness and volume have previously been associated with functional outcome and recanalization [26,27,32]. Currently, thrombus image characteristics are analyzed using semi-manual annotation, which is tedious, time consuming and is prone to observer variability. This makes analysis of large number of images less feasible. In contrast, automated segmentation allows for fast segmentation and thrombus biomarker extraction.
Automated accurate thrombus volume segmentation may also support machine learning approaches for the extraction of histological information, based on thrombus image characteristics. For example, Liebeskind et al. [33] showed that thrombus density might characterize thrombi as blood-cell rich or fibrin-rich, indicating whether a thrombus would be susceptible for intravenous thrombolysis. In future work, the CNN could be fed with histological information in combination with the segmentation and location of the thrombus to train for thrombus histology information extraction from imaging data alone.
This study suffers some limitations. First, the sample size of this study was limited. CNN networks require large number of samples to obtain a good accuracy. The more samples are included for training, the more generalizable the CNN will be, since it will be trained on a larger variation of thrombus cases. Since our main goal was to evaluate the accuracy of a tool for thrombus detection and the thrombus segmentations were merely evaluated for exploration for potential future work, we think that the current training dataset is sufficient enough for this proof of concept. We thereby kept track of the training and validation accuracy to prevent from overfitting.
Second, the number of experts that were included to determine the accuracy of manual detection was low. To accurately assess the performance of manual detection for clinical practice, readings by more observers would be required.
Third, the non-thrombus patches used within our training data were obtained from images of patients with an LVO. We made the assumption that the pixel values of patches with no thrombus pixels were comparable to patches that would be sampled from scans acquired from patients with stroke mimics. A limitation of this assumption is that we did not take any parenchymal and/or vascular changes into account that could have been caused by the occlusion.
Finally, we only included patients with anterior circulation LVOs. Future research should extend the task to identifying posterior circulation LVOs and more distal occlusions.

Conclusions
We have proposed a CNN for the detection and segmentation of thrombus in the anterior intra-arterial vasculature on NCCT in order to provide a tool for less experienced radiologists for the fast detection of AIS. The accuracy of thrombus detection is similar to expert neuroradiologists. Therefore, the presented CNN is a promising tool to be included in future clinical workflows. However, the sensitivity and specificity are currently too low to exclude CTA imaging in the work up of patients suspected with an LVO.
Moreover, the results of the thrombus segmentations showed a low volumetric agreement, similar to expert neuroradiologists, and low thrombus density agreement with ground truth. Therefore, the presented network is less suitable for exact thrombus segmentation. Funding: This study was part of the CONTRAST consortium which is supported by Netherlands Cardiovascular Research Initiative (CVON), an initiative of the Dutch Heart Foundation, and by the Brain Foundation Netherlands. AMC and Erasmus MC received additional unrestricted funding on behalf of CONTRAST, for the execution of MR CLEAN NO-IV from Stryker European Operations BV. The MR CLEAN trial was supported by the Dutch Heart Foundation and by unrestricted grants from AngioCare Covidien/ev3, Medac/Lamepro, and Penumbra.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.