Evaluation of ACR TI-RADS for Predicting Malignancy in Thyroid Nodules: Insights from Fine-Needle Aspiration Cytology and Histopathology Results

Abstract

The American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) stratifies patients with thyroid nodules for further evaluation. We evaluated the accuracy of this risk stratification and compared it with that of fine-needle aspiration cytology (FNAC) and final pathology. The nodules (n = 381) of 329 patients (mean age, 45.5 y; 251 female) who underwent thyroid surgery from January 2021 to September 2023 were retrospectively analyzed for the ACR TI-RADS, FNAC, and final pathology results. The nodules were categorized according to risk of malignancy as 75.9% ACR TI-RADS 5, 72.6% ACR TI-RADS 4, 38.9% ACR TI-RADS 3, and 33.3% ACR TI-RADS 2. Considering ACR TI-RADS 5 and 4 as high-risk categories, the sensitivity, specificity, positive-predictive value, and negative-predictive value of the ACR TI-RADS categories were 77.9%, 57.3%, 73.8%, and 62.8%, respectively. With both ACR TI-RADS and FNAC high-risk categories combined, the sensitivity and specificity improved to 82.4% and 58.6%, respectively. The ACR TI-RADS score effectively predicts malignancy in higher-risk categories; however, some malignant cases were found within lower-risk categories, implying the need for careful utilization of this system. Combining high-risk ACR TI-RADS and FNAC results improves the diagnostic accuracy of thyroid nodules. Confirmation from further prospective studies is required.

1. Introduction

Thyroid nodules are frequently found in the general population. The reported prevalence of thyroid nodules among the normal population varies depending on the detection method, with ultrasonography (US) showing prevalence rates of 19–35% and autopsy examination revealing rates of 8–65% [1,2]. The introduction of high-resolution ultrasonography has led to the incidental discovery of a large number of thyroid nodules. Deng et al. evaluated data from 195 countries and described an upward trend in thyroid cancer cases from 1990 to 2017 [3]. Similar upward trends have been reported in Saudi Arabia [4]. However, this increased detection rate has also led to a notable rise in the overdiagnosis of thyroid nodules. Such overdiagnosis often identifies indolent or clinically insignificant cases, leading to a rise in the number of thyroid cancer diagnoses reported in various studies [5,6]. As a result, the increase in thyroid surgeries has raised both the health and financial burden [7]. The American Thyroid Association has suggested simple ultrasound (US) criteria for risk assessment of thyroid nodules and for planning subsequent management [1]. In 2017, the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) was released. This system provides a uniform way to describe the US characteristics of thyroid nodules. The ACR TI-RADS categorizes thyroid nodules into five distinct classes based on the points of sonographic features, including shape, echogenic foci, composition, echogenicity, and margins of nodules. Recommendation of further intervention by fine-needle aspiration cytology (FNAC), biopsy, or US surveillance depends on the risk levels of these nodules [8]. Many studies have demonstrated the usefulness of ACR TI-RADS in decreasing the necessity for further invasive evaluation of thyroid nodules and subsequently lowering the need for thyroidectomy [9,10,11,12]. This risk classification model was also reported to outperform other models, such as those of the European Thyroid Association, the American Thyroid Association, and the Korean Society of Thyroid Radiology [13]. On the other hand, some studies have reported that it has low sensitivity and specificity [14]. Inter-observer variability in interpreting the US features that affect the final ACR TI-RADS score has been reported [15,16]. Moreover, the ACR TI-RADS does not consider certain features that can increase the risk of malignancy, such as the nature of the solid component of mixed nodules; it also has minimal utility for assessing multinodular goiter with multiple thyroid nodules of similar features [17,18].

Our main objective was to evaluate the diagnostic effectiveness of the ACR TI-RADS, specifically by examining its sensitivity, specificity, positive-predictive value (PPV), and negative-predictive value (NPV), and comparing it with FNAC and surgical pathologies. The secondary aim was to evaluate whether combining high-risk ACR TI-RADS categories with high-risk cytology results could enhance the diagnostic accuracy for thyroid malignancy. Overall, this research could have significant implications for the diagnosis and treatment of thyroid nodules, potentially leading to more accurate diagnoses, reduced unnecessary interventions, and improved patient outcomes.

2. Materials and Methods

This retrospective cross-sectional study was conducted in King Abdulaziz Medical City in the National Guard Health Affairs (KAMC-NGHA) and King Fahad Medical City (KFMC) in Riyadh, Saudi Arabia. Data from the KAMC-NGHA were retrospectively collected from the Best Care electronic medical records (Ezcaretech Co., Seoul, Republic of Korea) from January 2021 to December 2022. Data from KFMC were collected from Epic Hyperspace^® electronic medical records (Epic Systems Corporation, Verona, WI, USA) from January 2021 to September 2023.

The institutional review board (IRB) of King Abdullah International Medical Research Center approved the study in KAMC-NGHA (Study Number NRC23R/500/07), and the IRB of King Abdulaziz City for Science and Technology approved the study in KFMC (registration number: H-01-R-012). Informed consent was obtained from each patient for FNAC and thyroid surgery; however, the IRB waived the need for consent for retrospective data collection in this study.

We included patients with information on thyroid nodules and corroborating US imaging results according to the ACR TI-RAD classification, Bethesda categories, and final histopathological findings. Patients who underwent US, FNAC, or thyroidectomy outside our institution were excluded. Additionally, nodules with a US-described location that did not correlate to its gross anatomic location postoperatively were excluded. All FNAC procedures were performed under US guidance. ACR TI-RADS scoring, FNAC, and histopathology final results were evaluated and approved at both centers by senior radiologists and pathologists, respectively.

The medical records of all patients were reviewed, and data on patient demographics and thyroid nodule profile, including size, ACR TI-RADS classification, Bethesda score, final histopathology results, and positivity of lymph nodes, were extracted. The matching process was carefully maintained for each nodule regarding its ACR TI-RADS score, FNAC, and the final pathology results.

We categorized patients with ACR TI-RADS 5 and 4 and Bethesda scores 6 and 5 as high-risk groups, while those with ACR TI-RADS 3, 2, and 1, and Bethesda scores 4, 3, and 2 were considered low-risk groups. Non-diagnostic FNAC results were excluded from the analysis. The Statistical Package for Social Sciences (SPSS) version 26 (IBM Corp., Armonk, NY, USA) was used to analyze the data.. Categorical variables are described in percentages and frequencies, while continuous variables are presented as the mean ± standard deviation, median, and interquartile range. The risk (prevalence) of malignancy was interpreted as a proportion by dividing the number of malignant nodules by the total number of nodules. To determine the diagnostic accuracy, sensitivity, specificity, predictive values, and likelihood ratios with their 95% confidence intervals (CIs) were calculated. In addition, a receiver-operating characteristic (ROC) curve was drawn, and the area under the curve (AUC) and its 95% CI were determined.

3. Results

In total, 381 nodules from 329 patients were included in this study. Most of the included patients were female (76.3%). The mean age of the patients was 45.50 ± 14.28 years. The mean nodule size was 3.13 ± 1.90 cm, and 27.8% of the nodules were ≥4 cm in size. Nodules categorized as ACR TI-RADS 5 constituted 22.8%, whereas the Bethesda score indicating highly suspicious categories (Bethesda 5 and 6) constituted 31.2% of cases. Most nodules (60.2%) were malignant.

Table 1. Demographic and histopathological features of the included samples.

Characteristics	Subgroups	% (n) or Mean ± SD, Median (IQR)
Sex (n = 329)	Male	23.7 (78)
	Female	76.3 (251)
Age, years (n = 329)	45.50 ± 14.28, 45.00 (34.0–56.0)
Nodule size (n = 381)	3.13 ± 1.90, 2.80 (1.70–4.0), 0.30–17.4
Nodule size categories (n = 381)	<4 cm	72.2 (275)
	≥4 cm	27.8 (106)
ACR TI-RADS category (n = 381)	1	1.0 (4)
	2	3.9 (15)
	3	31.0 (118)
	4	41.2 (157)
	5	22.8 (87)
Bethesda category (n = 381)	1	2.4 (9)
	2	31.0 (118)
	3	27.0 (103)
	4	8.4 (32)
	5	6.0 (23)
	6	25.2 (96)
Histopathology (n = 381)	Benign	40.4 (154)
	Malignant	59.6 (227)

ACR TI-RADS, American College of Radiology Thyroid Imaging Reporting and Data System; SD, standard deviation; IQR, interquartile range.

shows the detailed features of the participants (n = 329) and the nodules (n = 381).

Regarding the risk of malignancy among the ACR TI-RADS and Bethesda categories, the incidence of malignancy was the highest among the high-risk groups in both systems However, some cases of Bethesda category 1 and ACR TI-RADS 2 (44.4% and 33.3%, respectively) were also malignant.

Table 2. Risk of malignancy in the ACR TI-RADS and Bethesda categories.

Bethesda Categories	Benign	Malignant	Risk of Malignancy	ACR TI-RADS Categories	Benign	Malignant	Risk of Malignancy
1 (n = 9)	5	4	44.4%	1 (n = 4)	4	0	0.0%
2 (n = 118)	91	27	22.9%	2 (n = 15)	10	5	33.3%
3 (n = 103)	44	59	57.3%	3 (n = 118)	72	46	38.9%
4 (n = 32)	9	23	71.9%	4 (n = 157)	43	114	72.6%
5 (n = 23)	1	22	95.7%	5 (n = 87)	21	66	75.9%
6 (n = 96)	0	96	100.0%

ACR TI-RADS, American College of Radiology Thyroid Imaging Reporting and Data System.

shows the risk of malignancy among different groups of the ACR TI-RADS and Bethesda systems.

Concerning the diagnostic accuracy of the ACR TI-RADS system for malignancy, the sensitivity, specificity, and PPV were 77.9%, 57.3%, and 73.8%, respectively, while those of the Bethesda system were 51.98%, 99.31%, and 99.16%, respectively. This aligns with the findings derived from the ROC curve (Figure 1), where a larger AUC was observed for the Bethesda system (0.857) than for the ACR TI-RADS (0.692).

When both the ACR TI-RADS and Bethesda system were used in combination, the sensitivity and specificity of the combined test increased to 82.4% and 58.6%, respectively, as compared to the ACR TI-RADS alone. Details of the diagnostic accuracy parameters of the ACR TI-RADS, Bethesda score, and combined systems, each with their 95% CIs, are shown in

Table 3. Diagnostic accuracy measures of the ACR TI-RADS, Bethesda, and combined systems.

Statistic	ACR TI-RADS Value (95% CI)	Bethesda Value (95% CI)	Combined Systems Value (95% CI)
Sensitivity	77.9% (72.01% to 83.10%)	51.98% (45.27% to 58.64%)	82.4 (76.79% to 87.10%)
Specificity	57.3% (49.01% to 65.36%)	99.31% (96.22% to 99.98%)	58.6% (50.15% to 66.73%)
Positive-Likelihood Ratio	1.8 (1.50 to 2.23)	75.37 (10.65 to 533.63)	2.0 (1.63 to 2.44)
Negative-Likelihood Ratio	0.39 (0.29 to 0.51)	0.48 (0.42 to 0.55)	0.30 (0.22 to 0.41)
Positive-Predictive Value	73.8% (69.77% to 77.41%)	99.16% (94.34% to 99.88%)	75.7% (71.79% to 79.24%
Negative-Predictive Value	62.8% (56.06% to 69.03%)	56.92% (53.55% to 60.22%)	68.0% (60.85% to 74.39%)
Total accuracy	69.8% (64.93% to 74.39%)	70.43% (65.51% to 75.02%)	73.1% (68.31% to 77.56%)

ACR TI-RADS, American College of Radiology Thyroid Imaging Reporting and Data System.

.

4. Discussion

Since the release of the ACR TI-RADS classification in 2017, many studies have evaluated its utility in the evaluation of thyroid nodules, particularly in the context of indeterminant cytology results. Despite the expected high occurrence of malignancy among the high-risk categories (ACR TI-RADS 4 and 5), we found a concerning proportion of malignant nodules among those categorized as ACR TI-RADS 2 and 3.

In a meta-analysis of 16 studies evaluating the accuracy of ACR TI-RADS, which included more than 20,000 nodules, Li et al. reported significant heterogeneity in both sensitivity and specificity, which ranged from 42% to 99% and from 35% to 92%, respectively. When all studies were pooled, the sensitivity and specificity were 89% (95% CI, 0.81–0.93) and 70% (95% CI, 0.60–0.78), respectively [14]. Heterogeneity in the reported sensitivity and specificity can be explained in different ways. First, some studies have considered the cutoff indicating high risk as ACR TI-RADS 4 or above [10,12,19,20,21,22], while others have considered it to be ACR TI-RADS 5 [23]. Second, some studies have considered US and FNAC with cytology as the gold standard [13,19,21,24], while other studies reported surgery and FANC along with final pathology or the final pathology alone as the gold standard [20,23,25,26]. This would lead to selection bias, as additional risk factors not covered by the ACR TI-RADS, including nodule size, family history, and radiation exposure, can influence the surgical decision, thereby potentially increasing the number of malignant cases. Third, the interpretation and interobserver variability in reporting sonographic features can strongly influence the results [27].

The ACR TI-RADS white paper estimated malignancy risk as approximately ≤2.0% for ACR TI-RADS 1 and ACR TI-RADS 2, 2.1–5.0% for ACR TI-RADS 3, 5.1–20.0% for ACR TI-RADS 4, and >20.0% for ACR TI-RADS 5 [8]. Except for ACR TI-RADS 1, our reported percentages were higher: the risk of malignancy for ACR TI-RADS 2, 3, 4, and 5 was 33.3%, 38.9%, 72.6%, and 75.9%, respectively. One of the reasons for the higher percentages in our study is that we reported the risk among the operated cases only. In a study of 1758 operated patients, Gao et al. reported higher malignancy percentages for ACR TI-RADS categories 1, 2, 3, 4, and 5 of 0%, 1.3%, 9.1%, 52.5%, and 88.8%, respectively [23]. Although it is difficult to generalize the results of our ACR TI-RADS 2 group because of the small numbers, the risk of malignancy in our ACR TI-RADS 3 group is also concerning.

Considering the primary use of the ACR TI-RADS is to reduce the need for further workup [8], malignant cases may be missed if the current ACR TI-RADS recommendations are followed. The 2015 American Thyroid Association suggested that active surveillance can be used for malignant cytology of nodules that are <1 cm in size in carefully selected patients. This recommendation remains controversial and requires further studies to define low-risk patients who may benefit from this strategy [1,28]. Sahli et al. found that 17% of nodules found to be malignant on the final histopathological investigation would have been overlooked if ACR TI-RADS recommendations were followed [29]. In our study, 35 (15%) malignant nodules would have been missed if we followed ACR TI-RADS recommendations. Although 14 of these cases in our study were microcarcinomas, the 21 remaining malignant nodules were >1 cm, and in 7 cases, lymph node metastasis was found in the final pathology results.

Regarding the ROC curve, our results were comparable to those reported by George et al. [30], indicating that the Bethesda system provides superior diagnostic accuracy compared to that of the ACR TI-RADS. We thus evaluated whether combining the high-risk ACR TI-RADS with high-risk cytology results would improve the diagnostic accuracy of thyroid malignancy. By this means, we demonstrated that combining these modalities increased the sensitivity and specificity and also resulted in minor improvements in all other diagnostic accuracy measures. To the best of our knowledge, no previous study has evaluated the combination of both diagnostic criteria. It would be interesting to evaluate the diagnostic power after combining ACR TI-RADS with not only Bethesda results but also with those of high-risk background history and molecular studies, particularly for the indeterminant risk group.

While this study provides valuable insights into the utility of the ACR TI-RADS, several limitations should be noted. First, our study was limited to operated cases, which may restrict the generalizability of our findings to a broader population of thyroid nodule patients, particularly those managed non-surgically. This could introduce a selection bias, as operated cases are often more likely to be high-risk or suspicious. Second, the retrospective design of our study may have introduced challenges in accurately correlating ultrasound-detected nodules with their final histopathological locations, potentially leading to misclassification or incomplete data. Additionally, small sample sizes in specific subgroups, such as Bethesda 1, ACR TI-RADS 1, and ACR TI-RADS 3, may have reduced the statistical power, limiting our ability to draw precise conclusions for these categories.

Despite the above limitations, our study contributed to the growing body of evidence on thyroid nodule evaluation and management. Future research should focus on overcoming these limitations to improve the reliability and applicability of ACR TI-RADS in clinical practice.

5. Conclusions

In summary, our study indicated that the ACR TI-RADS is useful for decision-making to workup thyroid nodules further; however, the results should be applied carefully where ACR TI-RADS 2 and 3 nodules are concerned, and additional risk factors should be considered in decision-making to further workup these patients to avoid missing high-risk malignant cases. Large prospective studies are necessary to assess the accuracy of the ACR TI-RADS within these categories. Additionally, our findings emphasize that combining different diagnostic tools enhances the accuracy of evaluating thyroid nodules.