Inter-Rater Reliability of Collateral Status Assessment Based on CT Angiography: A Retrospective Study of Middle Cerebral Artery Ischaemic Stroke

The importance of assessing the collateral status (CS) in patients with ischaemic stroke (IS) has repeatedly been emphasised in clinical guidelines. Various publications offer qualitative or semiquantitative scales with gradations corresponding to the different extents of the collaterals, visualised mostly on the basis of CTA images. However, information on their inter-rater reliability is limited. Therefore, the aim of this study is to investigate the inter-rater reliability of the scales for collateral assessment. CTA images of 158 patients in the acute period of IS were used in the study. The assessment of CS was performed by two experts using three methodologies: the modified Tan scale, the Miteff scale, and the Rosenthal scale. Cohen’s kappa, weighted kappa and Krippendorff’s alpha were used as reliability measures. For the modified Tan scale and the Miteff and Rosenthal scales, the weighted kappa values were 0.72, 0.49 and 0.59, respectively. Although the best measure of consistency was found for the modified Tan scale, no statistically significant differences were revealed among the scales. The impact of the CS on the degree of neurological deficit at discharge was shown for the modified Tan and Rosenthal scales. In conclusion, the analysis showed a moderate inter-rater reliability of the three scales, but was not able to distinguish the best one among them.


Introduction
Ischaemic stroke (IS) affects over 7.6 million people annually [1] and dominates in the structure of mortality and disability in the population [1][2][3].Even though mortality rates have decreased by 32% over the last decade [2], up to 78% of survivors have some physical or cognitive deficiency and desperately need extensive rehabilitation [4][5][6].Reperfusion therapy for IS is aimed at the restoration of the blood flow to the ischaemic brain tissue, and includes two main techniques: intravenous thrombolysis (EVT) with recombinant tissue plasminogen activator and mechanical thrombectomy (MT) [7].Timely reperfusion therapy can significantly improve the patient's chances of a complete or partial recovery, but the efficiency of treatment depends not only on the time passed before intervention (onset-to-reperfusion time), but also on the condition of the collateral blood flow that supports the viability of ischaemic areas [8][9][10][11].The opening of collateral vessels and anastomoses between cerebral arterial vascular territories after thrombosis is one of the pathophysiological stages of IS [12,13].
Digital subtraction angiography (DSA) is considered the gold standard in determining collateral status (CS) [12,14].However, DSA's disadvantages, which include invasiveness (it is performed only in patients already selected for reperfusion) and expensiveness, and the fact that it allows imaging only in the territory of the contrasted artery, make the use of other radiological techniques reasonable [15].Computed tomography angiography (CTA), which is quite common in routine practice and is not time consuming, has demonstrated good concordance with DSA [16,17].CTA allows high-resolution visualisation of the entire cerebral vasculature and allows precise localisation of stenosis or occlusion.Some studies have found that CTA-assessed CS is even more informative in predicting outcome than assessment performed by means of DSA [18,19].
The importance of assessing CS has been repeatedly emphasised in clinical guidelines [33][34][35][36][37], as its main practical relevance is in supporting proper patient selection for reperfusion.Nevertheless, none of the scales has yet been universally accepted.The main reason for this is their low reliability and a very limited number of studies devoted to a comparative analysis of different scales.To date, only a few studies have assessed the inter-rater reliability of existing scales [38][39][40].Most often, these assessments have been performed by the authors of the scales themselves [20,25,30,32], or in the context of a small number of reviews [38][39][40].In some studies, the choice of methods for assessing reliability, including the use of intraclass correlation coefficients (ICC) designed to deal with quantitative variables with normal distribution [41], in the task of assessing qualitative scales [29,38,42], is questionable.
The relation of collateral capacity to treatment efficacy and clinical presentation is of great interest in terms of the practical relevance of the scales.To assess it, the authors have most often used the information on the degree of functional independence graded on the modified Rankin Scale (mRs) 3 months after the IS [19,20,39,43].In reperfusion interventions, favourable outcomes (mRs 0-2) have been found to be significantly more frequent in cases with good collaterals [20,43].However, it has recently been shown that the NIHSS (National Institutes of Health Stroke Scale) [44] is equally informative as a target outcome variable, with the most pronounced dynamics in the degree of neurological deficit observed after reperfusion interventions and during hospitalisation.To date, there are few works in the literature reflecting the impact of the CS on the clinical performance assessed using the NIHSS upon discharge.
Therefore, the aim of this work was to study the inter-rater reliability of CS scales in IS patients with the use of CTA and analyse their relation to treatment efficacy.The degree of neurological deficit was assessed by neurologists with the NIHSS immediately after admission.The standard treatment protocol also included NIHSS score assessment at the following time points: just before reperfusion, every 15 min during EVT, every 30-60 min throughout the first day in the intensive care unit, and then daily.

Description of the Sample
In the current research, we used the information on the NIHSS score at discharge to assess the association of CS with the treatment efficacy.In the case of death due to the underlying disease, a score of 42 was used as the final NIHSS score.In cases of death due to complications (e.g., pulmonary embolism), the data from the last neurological examination before the acute deterioration were recorded.
Non-contrast CT conducted right after the initial neurologic examination and assessed by radiologists showed the absence of visible signs of ischaemia in 82.3% of cases.In 94 cases (59.5%),CTA revealed the presence of MCA occlusion, in 7 cases (4.4%) there was only ICA occlusion.On the follow-up MRI, all patients had confirmed areas of ischaemia in the MCA vascular territory, with an average lesion volume of 111 (IQR 58-164) cm 3 .
Reperfusion therapy was performed in 48.1% of patients: 26 had thrombolysis (EVT) alone, 28 had MT (thromboextraction/thromboaspiration) and 22 had both.The decision on treatment tactics was taken by a team of neurologists and endovascular surgeons in accordance with the National Guidelines [37].In the case of a positive decision on reperfusion therapy, intravenous thrombolysis was performed by neurologists and resuscitators, and mechanical thrombectomy was performed by endovascular surgeons.For the reperfusion group, the median time from symptoms' onset to CTA was 95 (IQR 62-143) minutes.EVT was performed right after CT and CTA scanning (directly in the CT room); MT required approximately 20 additional minutes for the transfer to the operating room.
The clinical characteristics of the sample are shown in Table 1 (parameters of statistical tests are available in Table A1 of the Appendix A).
The study was approved by the biomedical ethics committee of the State Budget Institution "N.V. Sklifosovsky Research Institute for Emergency Medicine of the Moscow Health Department" (protocol # 8-22 of 19 July 2022).

Assessment of Collateral Status with CTA Images
The research was performed on a 64-slice Aquilion CXL CT scanner (Toshiba Med.Inc., Tokyo, Japan) with a slice thickness of 0.5 mm.During the first stage, a standard non-contrast brain scan was performed to exclude haemorrhagic components.This was followed by a CTA scan of the brachiocephalic arteries with the aortic arch captured.Data were acquired with 64 × 0.5 mm collimation, and tube voltage and current strength were 120 kVp and 100 mAs, respectively.In all cases, a non-ionic contrast agent with an iodine concentration of at least 350 mg/mL was used that was administered intravenously using an automatic bolus injector at a rate of 4 mL per second, in a volume of 80 mL.An automatic aortic arch bolus tracking system was used to initiate the scan at the right time.
The state of the collateral blood flow was assessed using three scales: (1) The modified Tan scale [25] is based on the binary classification proposed in 2002 by Shramm et al. [21].This classification is both the simplest and, according to the published works, one of the most reliable [38,39], probably due to the minimal number of grades.A 'good' grade is given if collaterals are seen on more than 50% of the MCA basin.(2) The Miteff scale [20]: the authors propose characterising CS on the basis of the maximum-intensity projection (MIP) according to the following three grades: poor collaterals-only distal superficial branches of the MCA are visible; moderate collaterals-in addition to superficial arteries, the branches of the MCA in the Sylvian sulcus are also visualised; good collaterals-MCA is visualised immediately after the occlusion site.Previously, this system also demonstrated a high inter-rater reliability [39].(3) The Rosenthal scale [27]: unlike the previous two, this classification is based on the comparison of the affected side with the intact one and includes five gradations: (1) absent vessels; (2) vessels are less than on the contralateral normal side; (3) vessels are represented to the same extent as on the intact side; (4) vessels are greater than on the contralateral normal side; (5) exuberant vessels on the affected side.The authors originally proposed using only CTA-SI, but as MIP and MPR are now firmly integrated into any workstation routine, they were also assessed.Examples of gradations for each scale are presented in Figure 1.Thus, the modified Tan scale and the Rosenthal scale can both be used for cases with or without visible occlusions, while the Miteff scale is intended only for cases with visible occlusions.The choice of these scales was motivated by the low number of grades, as using them seems not to be so time consuming as that for more detailed scales.Additionally, the modified Tan scale and the Miteff scale were selected because of their relatively high prevalence in the literature and the availability of previously published articles claiming their high reliability, while the Rosenthal scale offered more explicit definitions of grades.
Two radiologists (M.R. and V.S., with 17 and 7 years of experience, respectively) performed the estimation of CS after attending a 1-h training session on the aforementioned scores.The assessment was performed independently and retrospectively, with no additional details on the clinical picture or the results of follow-up radiological studies.The only information provided to the radiologists was the side of the lesion.MIP and MPR were constructed on a Gamma-Multivox workstation.CTA series of three clinical cases, as examples of radiologists' disagreement, are presented in Appendix B Figures A1-A3.In the subsequent analysis of the association of CS with the severity of neurological impairment, the scales were binarised, with cases where collaterals were considered poor by at least one expert being combined with those where the experts agreed in their opinion on poor collaterals.Thus, the modified Tan scale and the Rosenthal scale can both be used for cases with or without visible occlusions, while the Miteff scale is intended only for cases with visible occlusions.The choice of these scales was motivated by the low number of grades, as using them seems not to be so time consuming as that for more detailed scales.Additionally, the modified Tan scale and the Miteff scale were selected because of their relatively high prevalence in the literature and the availability of previously published articles claiming their high reliability, while the Rosenthal scale offered more explicit definitions of grades.
Two radiologists (M.R. and V.S., with 17 and 7 years of experience, respectively) performed the estimation of CS after attending a 1-h training session on the aforementioned scores.The assessment was performed independently and retrospectively, with no additional details on the clinical picture or the results of follow-up radiological studies.The only information provided to the radiologists was the side of the lesion.MIP and MPR were constructed on a Gamma-Multivox workstation.CTA series of three clinical cases, as examples of radiologists' disagreement, are presented in Appendix B Figures A1-A3.In the subsequent analysis of the association of CS with the severity of neurological impairment, the scales were binarised, with cases where collaterals were considered poor by at least one expert being combined with those where the experts agreed in their opinion on poor collaterals.

Methods for Assessing Inter-Rater Reliability
There are several methods for assessing the inter-rater reliability of qualitative scales.The most common method is the Cohen's kappa calculation [45], which is designed to evaluate the consistency of the conclusions of two experts.The interpretation of kappa values is ambiguous, as kappa is sensitive to the balance of groups (the highest values are

Methods for Assessing Inter-Rater Reliability
There are several methods for assessing the inter-rater reliability of qualitative scales.The most common method is the Cohen's kappa calculation [45], which is designed to evaluate the consistency of the conclusions of two experts.The interpretation of kappa values is ambiguous, as kappa is sensitive to the balance of groups (the highest values are observed for balanced samples) and to the number of gradations (the more gradations, the lower the kappa) [46].One of the most common interpretations is the Fleiss version [47], according to which: (1) κ < 0.4 is accepted as poor reliability; (2) 0.4 ≤ κ < 0.75 indicates fair to good reliability; (3) κ ≥ 0.75 indicates excellent reliability.
When a scale has more than two gradations, this methodology does not consider the degree of disagreement between rates.For such cases, it is recommended that a weighted kappa be used [48], the calculation of which involves a system of 'penalties' for disagreements (the greater the disagreement between opinions, the greater the 'penalty').
Another measure of reliability is Krippendorff's alpha [49].This is advantageous when applied to any number of raters, for both quantitative and qualitative data, and is also able to deal with omissions in the data.

Statistical Processing of the Results
Quantitative variables are described in this paper using the median and interquartile range (IQR) because of the small number of cases in most comparison groups and the absence of a normal distribution revealed with the Shapiro-Wilk test.The values of qualitative characteristics are given as absolute and relative frequencies.The Mann-Whitney U test was used to compare quantitative variables, including the NIHSS scores between different groups of CS.A comparison of qualitative characteristics was performed with the Pearson's chi-square test and Fisher's exact test.For statistical hypothesis testing, a significance threshold of p o = 0.05 was used.A Holm-corrected value of the threshold was also calculated (p o = 0.017) in order to address the multiple comparison problem [50].
Cohen's kappa and weighted kappa were calculated by means of the statistical software package of MedCalc (online version) [51].The Krippendorffsalpha library was used to assess Krippendorff's alpha.A point estimate and a 95% confidence interval (95% CI) were calculated for each consistency measure.
The results are visualised using box-and-whiskers plots in which the lower and upper bounds of the box correspond to the first and third quartiles, the line in the middle of the box indicates the median, and the lower and upper whiskers indicate the minimum and maximum values.Outliers are indicated with coloured dots (for observation 1.5 times the interquartile range less than the first quartile or greater than the third quartile) and asterisks (for observation 3 times the interquartile range less than the first quartile or greater than the third quartile).
The statistical analysis was performed in the R Studio environment (version 3.6.1).

Results
The distribution of the radiologists' assessments for each of the three scales is given in Tables 2-4.This distribution shows that the sample as a whole was not balanced: there were far fewer cases with poor collaterals.For the Rosenthal scale, the rarest variant was the presence of collaterals greater than on the intact side (Table 4).Additionally, there were no cases with exuberant collaterals on the ischaemic side (grade 5).
The distribution of cases on the Miteff scale for patients with MCA occlusion was relatively even.
The calculation of agreement rates for the three scales (Table 5) showed that the highest coefficient values were observed when using the modified Tan scale.The resulting weighted kappa values correspond to a moderate degree of inter-rater consistency.However, due to the overlapping confidence intervals, no statistically significant differences can be established with certainty.
The following grouping of the obtained assessments was performed to estimate the influence of CS on the degree of neurological deficit, while taking into account the differences in the examiners' opinions.For the modified Tan scale and the Miteff scale, both the series where the radiologists' opinions were unanimous and the series for which only one of the examiners determined the status to be poor were included in the group of cases with poor collaterals.According to the Rosenthal scale, the cases in which at least one examiner rated "no vessels" and "vessels less pronounced than on the intact side" were united in a single group.All other cases were assigned to the group with good CS.Thus, all scales were binarised: only two gradations were identified in each, with conventionally good and poor collaterals, and each case was assigned to one of the gradations.Patients in these groups were compared on the basis of their degree of neurological deficit at discharge with regard to the type of treatment (Figure 2).
The distribution of patients and the descriptive statistics for NIHSS scores on discharge are presented in Table 6 (the parameters of statistical tests are presented in Table A2 in the Appendix A).
The evaluation of the impact of CS on the degree of neurological deficit on discharge showed statistically significant differences for the modified Tan scale and Rosenthal scale: regardless of whether reperfusion interventions were performed or not, the NIHSS score at discharge was significantly lower in the group of patients with good collaterals (marked green in the box diagrams).For the Miteff scale, a significant difference was registered only for the patients under conservative treatment (p = 0.031), possibly due to the small number of observations in the groups.The distribution of patients and the descriptive statistics for NIHSS scores on discharge are presented in Table 6 (the parameters of statistical tests are presented in Table A2 in the Appendix A).The evaluation of the impact of CS on the degree of neurological deficit on discharge showed statistically significant differences for the modified Tan scale and Rosenthal scale: regardless of whether reperfusion interventions were performed or not, the NIHSS score at discharge was significantly lower in the group of patients with good collaterals (marked green in the box diagrams).For the Miteff scale, a significant difference was registered only for the patients under conservative treatment (p = 0.031), possibly due to the small number of observations in the groups.

Discussion
The state of CS is one of the key factors in the course of IS.Unfortunately, the definition of this term remains one of the vaguest.It is important to note that when analysing

Discussion
The state of CS is one of the key factors in the course of IS.Unfortunately, the definition of this term remains one of the vaguest.It is important to note that when analysing the CT and MRI images, we did not refer to the visualisation of leptomeningeal anastomoses (also known as pial collateral vessels), which are less than 1 mm in diameter, but rather larger-calibre vessels that were retrogradely filled from them [13,15].
Most often, in the context of collateral blood flow assessment, CT and MR perfusion are mentioned, as they allow the visualisation of the core and penumbra zones, with the latter being more prominent due to good collaterals [11].However, not every hospital is capable of performing these studies, making the analysis of CTA images more relevant.Previous works on the application of scales in the assessment of CS by CTA indicate the presence of the relationship of obtained scores with the outcome of IS and the effectiveness of reperfusion interventions [43,52,53].However, there is limited information on the interrater agreement of the scales as a measure of their reliability [20,28,39,40,43].
In this study, we investigated the three scales with the lowest numbers of grades; two of them have demonstrated high reliability on the metrics used in several publications.For example, in the study by Weiss et al. [40], the weighted kappa values for the modified Tan scale and the Miteff scale were 0.86 and 0.81, respectively (95% CI not presented), while Yeo et al. [39] reported even higher reliability values in a CTA analysis of 200 patients: the Cohen kappa values for the modified Tan scale and the Miteff scale were 0.93 (95% CI 0.91-0.95) and 0.91 (95% CI 0.86-0.93),respectively.The authors of the Miteff scale themselves reported similar results (Cohen's kappa = 0.93) on data from 92 patients [20], which may be due to the homogeneous nature of the sample: only patients with occlusion in the M1 segment were included.
The inter-rater reliability assessed in the current study was characterised by far more modest results.The weighted Cohen's kappa values for the modified Tan scale and the Miteff scale were 0.72 (95% CI 0.59, 0.84) and 0.56 (95% CI 0.41, 0.71), respectively, indicating that both point estimates and confidence intervals were significantly lower than the reported values.At the same time, the Multicentre Randomised Clinical Trial of Endovascular Treatment of Acute Ischaemic Stroke (MR CLEAN) study of 493 patients was also known to have a moderate inter-rater consistency, with a weighted kappa of only 0.49 for the modified Tan scale [43].
One possible reason for such a wide distribution of consistency measurements is the somewhat variable manner in which the scales are used: some articles simply mention the imaging of the arteries in the ischaemic area when describing the collateral assessment method [20,25], while others explicitly state the need for a comparison with the intact side for the same scales [40].The viewing parameters (SI alone [21,25], MPR alone [25,29], MIP alone [19,20,25] or MPR along with MIP [40]) may also have some influence.As the generation of MPR and MIP has become a routine practice for radiologists, in our work, we used both.At the same time, there are no clear recommendations regarding the thickness of MIP reconstruction for the assessment of the intracranial vascular bed.In our work, MIP images with a thickness of at least 10 mm were the best, in the opinion of the examining radiologists.
For the Rosenthal five-grade scale, inter-rater agreement had not been assessed before, either by the authors of the scale or by other research teams, so the current work is the first study of the reliability of this scale [27].In terms of the results obtained, no significant difference in reliability was detected compared with the other two.In the literature, classification is widely performed based on the five Rosenthal grades: according to the 10-point M.B.Maas scale, the Sylvian sulcus vessels and leptomeningeal arteries [28] are to be separately characterised using the five-point scale; however, the assessment of the inter-rater reliability of this scale was beyond the scope of this study.
The comparison with most other scales, including more detailed ones, does not suggest the use of the latter to be preferable over these scales.For example, in a study by Yang et al., in which the inter-rater reliability was estimated on the basis of data from 100 CTA series, the original four-grade Tan scale and the authors' proposed expanded six-grade version of this scale produced weighted kappa values of 0.65 (95% CI 0.42-0.88)and 0.7 (95% CI 0.59-0.81),respectively [30].Unfortunately, it was impossible to correlate our results with those presented in some publications in which the same collateral status scales were used due to incorrect statistical analyses and the uninformative nature of the indicators provided by the authors [38].
It is important to note that in the majority of studies, the authors, who had initially used more detailed gradations, enlarged the groups and reduced the number of gradations to two in the subsequent analysis of the association with the degree of functional independence (mRs), as well as in attempts to construct prognostic rules [19,52,54].
The use of the NIHSS score at the end of hospitalisation as an output variable is a distinctive feature of the current study: previously published studies focused on mRs 3 months after IS in order to assess the impact of CS on clinical outcome [19,39,55].The possibility of using the NIHSS as a surrogate endpoint instead of mRs has already been demonstrated in the literature [44].We believe that, despite the differences in the purpose of these scales (NIHSS for describing neurological deficits and mRs for describing functional independence), this substitution has several justifications.Firstly, it is during the intrahospital period rather than during rehabilitation that the most pronounced progression in the patient's status is observed; secondly, the need to follow patients up for 3 months often results in the loss of some cases, and therefore smaller sample sizes.In addition, the use of mRs is time consuming and costly.There are few studies analysing the relationship between CS and NIHSS assessment one day after symptom onset (early neurological improvement) [32,56].Since our study included patients on conservative therapy, who do not commonly show dramatic improvements one day after onset, we assessed the degree of neurological deficit at the time of discharge.
Another particularity of our work was the inclusion of both patients with visible occlusions for whom endovascular interventions had been performed and patients who, for various reasons, were receiving conservative treatment alone.Most works are devoted to the assessment of the association of collaterals with the clinical outcome in patients after reperfusion [19,39,55,56]; however, in cases where interventions are contraindicated or the occlusion site cannot be visualised, the outcome of IS will be almost entirely determined by collateral reserves.Our data showed that the presence of good collaterals assessed on the basis of the modified Tan and Rosenthal scales was a significant predictor of less severe neurological symptoms in this group of patients.
Overall, our results are fully consistent with those of other authors based on the data from the mRs: the presence of good collaterals causes a less severe clinical picture and a greater efficacy of reperfusion interventions.
None of the publications we analysed used Krippendorff's alpha to assess consistency, and the lack of the original cross-tabulations prevented us from calculating it by ourselves.Since there remains some ambiguity in the assessment of CS, additional multicentre studies involving multiple radiologists and employing both operator-dependent scales and more quantitative methods of collateral assessment, which are clearly the trend of recent research, are necessary.
One of the most promising solutions to the problem of the subjectivity of CS assessment is the development of algorithms for the automatic assessment of cerebral vessels [57,58].So far, only a few clinically validated software products have been presented (e.g., e-STROKE by Brainomix Ltd., StrokeViewer by Nico-lab), and the results of their implementation into routine practice indicate a significant potential of this direction of CTA image analysis [10,30,42].Moreover, similar software can be applied for IS in vertebrobasilar system, for which there are currently different scales from those for CMA IS [59][60][61][62].In these cases, the use of the Krippendorff's alpha will allow the results of different scales and different study designs to be compared.
The limitations of this study are as follows.Firstly, it was a relatively small sample, in which the distribution of patients into comparison groups was unbalanced: gradations with poor collaterals, assessed using the modified Tan and Rosenthal scales, were much less frequent.However, judging by the fact that a similar distribution has been observed in many other studies [20,43,52,56,57], it may reflect the situation in the population in general.Nevertheless, small groups and the non-normality of the distribution of variables resulted in nonparametric tests having less statistical power than parametric ones.Secondly, it is obvious that the single-phase CTA is sensitive to temporal information and does not allow the visualisation of vessel filling in later phases.There is evidence in the literature suggesting two-and three-phase CTA as being more informative [32,[63][64][65], but in practice these studies are not as widespread as single-phase CTA.Finally, the study design was single-centred and non-randomised, which may result in biases.
Additionally, it should be mentioned that the study design did not take into account the structure of the circle of Willis, which acts as the primary collateral system, as well as the presence of extra-intracranial anastomoses [9].Their influence on the stroke outcome was studied in previously published articles [23,66,67], but due to the considerable variety in their anatomies (including combinations of aplasia/hypoplasia of anterior and posterior communicating arteries, as well as of A1 and P1 segments), an integrated approach and representative samples are necessary for future research.
We believe that the findings presented in the current work can help in planning further studies with higher evidential value, in particular through preliminary power analysis and meta-analysis.These steps will make it possible to come closer to solving the problem of performing reliable collateral assessment and proper patient selection for reperfusion therapy, which is especially important for those who have been admitted outside the therapeutic window or who have an unknown time of symptom onset (wake-up strokes).

Conclusions
The analysis of the CS assessment by two radiologists using CTA images demonstrated a moderate degree of inter-rater reliability.The highest rates of consistency were observed for the modified Tan scale, but there were no statistically significant differences among the scales.Significant differences were revealed when assessing the relationship between the CS and the dynamics of neurological deficit: patients with good collaterals on the modified Tan and Rosenthal scales received lower values for their NIHSS score.This pattern was observed in both the group of patients after reperfusion and in the group undergoing conservative treatment alone.The objectification of the CS assessment based on CTA images remains an unsolved and urgent task that requires further research.
This retrospective study used CTA DICOM-images of 158 consecutive patients admitted between 2018 and 2021 to the N.V. Sklifosovsky Research Institute of Emergency Medicine in the acute period of IS.The median age was 72 (IQR 63-81) years, and 46.8% of patients were males.The inclusion criteria were as follows: (1) age of 18 years or older; (2) diagnosis of middle cerebral artery (MCA) ischaemic stroke confirmed with follow-up MRI study (2-3 days after the admission); (3) admission within 6 h after symptoms onset; (4) CTA performed on admission.The exclusion criteria were: (1) poor-quality CTA images and presence of artifacts; (2) absence of CTA images on admission and MR images in dynamics in the hospital PACS (picture archiving and communication system); (3) lacunar pathogenetic variant of IS according to Trial of Org 10,172 in Acute Stroke Treatment (TOAST) classification; (4) lesion not only in MCA territory, but also in the zones of anterior or posterior cerebral artery; (5) bilateral IS.

Figure 1 .
Figure 1.CT angiograms showing examples of collateral status grades for the modified Tan scale, the Miteff scale, and the Rosenthal scale.The areas indicated with a red dashed line indicate the side of the stroke.

Figure 1 .
Figure 1.CT angiograms showing examples of collateral status grades for the modified Tan scale, the Miteff scale, and the Rosenthal scale.The areas indicated with a red dashed line indicate the side of the stroke.

Figure 2 .
Figure 2. Comparison of patients with poor and good collaterals by NIHSS score on discharge.The symbols * and ** indicate statistically significant differences at p < 0.05 and p < 0.017 (Holm-corrected value), respectively.Coloured dots and asterisks mark outliers.

Figure 2 .
Figure 2. Comparison of patients with poor and good collaterals by NIHSS score on discharge.The symbols * and ** indicate statistically significant differences at p < 0.05 and p < 0.017 (Holm-corrected value), respectively.Coloured dots and asterisks mark outliers.

Figure A1 .
Figure A1.An example of radiologists' disagreement on the score obtained using the modified Tan scale.Male, 80 years old, with the occlusion of M3 segment of the right MCA.Both experts characterised collateral status as "Good, MCA is visualised immediately after the occlusion site" on the Miteff scale and as "Less than the contralateral normal side" on the Rosenthal scale, but disagreed on the modified Tan scale score.

Figure A2 .
Figure A2.An example of radiologists' disagreement on the score obtained using the Miteff scale.Female, 76 years old, with the occlusion of M1 segment of the left MCA.Both experts characterised the collateral status as "Poor" on the modified Tan scale and as "Less than the contralateral normal side" on the Rosenthal scale.Nevertheless, one of the radiologists assessed collateral status as "Moderate.Vessels can be seen at the Sylvian fissure", and the other as "Poor.Only distal superficial branches of the MCA are visible".

Figure A2 .
Figure A2.An example of radiologists' disagreement on the score obtained using the Miteff scale.Female, 76 years old, with the occlusion of M1 segment of the left MCA.Both experts characterised the collateral status as "Poor" on the modified Tan scale and as "Less than the contralateral normal side" on the Rosenthal scale.Nevertheless, one of the radiologists assessed collateral status as "Moderate.Vessels can be seen at the Sylvian fissure", and the other as "Poor.Only distal superficial branches of the MCA are visible".

Figure A3 .
Figure A3.An example of radiologists' disagreement on the score obtained using the Rosenthal scale.Male, 69 years old, with occlusion of M1 segment of the left MCA.Both experts characterised collateral status as "Poor" on the modified Tan scale and as "Poor.Only distal superficial branches of the MCA are visible" on the Miteff scale.On the Rosenthal scale, there were two opinions: "Less than the contralateral normal side" and "Absent vessels".

Table 1 .
Characteristics of the study population (comparison of groups was performed using the Mann-Whitney U test for quantitative variables, the Pearson's chi-squared test and the Fisher's exact test for qualitative variables.The symbol * denotes statistically significant results).
NIHSS, National Institutes of Health Stroke Scale; CTA, computed tomography angiography, ASPECTS, The Alberta Stroke Program Early CT Score; IS, ischaemic stroke; TOAST, Trial of Org 10,172 in Acute Stroke Treatment; MCA, middle cerebral artery; ICA, internal carotid artery.

Table 2 .
Distribution of radiologists' opinions on the CS score for the modified Tan scale.

Table 3 .
Distribution of radiologists' opinions on CS score for the Miteff scale.

Table 4 .
Distribution of radiologists' opinions on CS score for the Rosenthal scale.

Table 5 .
Inter-rater reliability scores for the three scales assessing collateral status.The 95% confidence interval is shown in parentheses.

Table 6 .
Distribution of NIHSS scores on discharge in patients, grouped by CS grade.The symbols * and ** indicate statistically significant differences at p < 0.05 and p < 0.017 (Holm-corrected value), respectively.
NIHSS, National Institutes of Health Stroke Scale.

Table 6 .
Distribution of NIHSS scores on discharge in patients, grouped by CS grade.The symbols * and ** indicate statistically significant differences at p < 0.05 and p < 0.017 (Holm-corrected value), respectively.
NIHSS, National Institutes of Health Stroke Scale.
, National Institutes of Health Stroke Scale; CTA, computed tomography angiography, ASPECTS, The Alberta Stroke Program Early CT Score; IS, ischaemic stroke; TOAST, Trial of Org 10,172 in Acute Stroke Treatment; MCA, middle cerebral artery; ICA, internal carotid artery. NIHSS