An Interobserver Comparison of the Ultrasound Lexicon Classification of Thyroid Nodules: A Single-Center Prospective Validation Study
Abstract
:1. Introduction
2. Materials and Methods
3. Results
3.1. Study Population
3.2. Agreement for US Characteristic According to the I-TIRADS Lexicon
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
ATA | American Thyroid Association |
ACR-TIRADS | American College of Radiology Thyroid Imaging Reporting and Data System |
CI | Confidence Interval |
CT | Computed Tomography |
EU-TIRADS | European Thyroid Imaging Reporting and Data System |
FDG-PET/CT | 18F-Fluorodeoxyglucose Positron Emission Tomography/Computed Tomography |
FNAB | Fine Needle Aspiration Biopsy |
IQR | Interquartile Range |
I-TIRADS | International Thyroid Imaging Reporting and Data System |
K-TIRADS | Korean Thyroid Imaging Reporting and Data System |
MRI | Magnetic Resonance Imaging |
RSSs | Risk Stratification Systems |
SPSS | Statistical Package for the Social Sciences |
US | Ultrasonography |
References
- Pizzato, M.; Li, M.; Vignat, J.; Laversanne, M.; Singh, D.; La Vecchia, C.; Vaccarella, S. The epidemiological landscape of thyroid cancer worldwide: GLOBOCAN estimates for incidence and mortality rates in 2020. Lancet Diabetes Endocrinol. 2022, 10, 264–272. [Google Scholar] [CrossRef]
- Frates, M.C.; Benson, C.B.; Charboneau, J.W.; Cibas, E.S.; Clark, O.H.; Coleman, B.G.; Cronan, J.J.; Doubilet, P.M.; Evans, D.B.; Goellner, J.R.; et al. Management of thyroid nodules detected at US: Society of Radiologists in Ultrasound consensus conference statement. Ultrasound Q. 2006, 22, 231–238. [Google Scholar] [CrossRef]
- Marqusee, E.; Benson, C.B.; Frates, M.C.; Doubilet, P.M.; Larsen, P.R.; Cibas, E.S.; Mandel, S.J. Usefulness of Ultrasonography in the Management of Nodular Thyroid Disease. Ann. Intern. Med. 2000, 133, 696–700. [Google Scholar] [CrossRef]
- Papini, E.; Guglielmi, R.; Bianchini, A.; Crescenzi, A.; Taccogna, S.; Nardi, F.; Panunzi, C.; Rinaldi, R.; Toscano, V.; Pacella, C.M. Risk of Malignancy in Nonpalpable Thyroid Nodules: Predictive Value of Ultrasound and Color-Doppler Features. J. Clin. Endocrinol. Metab. 2002, 87, 1941–1946. [Google Scholar] [CrossRef]
- Nam-Goong, I.S.; Kim, H.Y.; Gong, G.; Lee, H.K.; Hong, S.J.; Kim, W.B.; Shong, Y.K. Ultrasonography-guided fine-needle aspiration of thyroid incidentaloma: Correlation with pathological findings. Clin. Endocrinol. 2003, 60, 21–28. [Google Scholar] [CrossRef]
- Khoo, M.L.C.; Asa, S.L.; Witterick, I.J.; Freeman, J.L. Thyroid calcification and its association with thyroid carcinoma. Head Neck 2002, 24, 651–655. [Google Scholar] [CrossRef] [PubMed]
- Kim, S.J.; Kim, E.K.; Park, C.S.; Chung, W.Y.; Oh, K.K.; Yoo, H.S. Ultrasound-Guided Fine-Needle Aspiration Biopsy in Nonpalpable Thyroid Nodules: Is It Useful in Infracentimetric Nodules? Yonsei Med. J. 2003, 44, 635–640. [Google Scholar] [CrossRef] [PubMed]
- Kim, E.-K.; Park, C.S.; Chung, W.Y.; Oh, K.K.; Kim, D.I.; Lee, J.T.; Yoo, H.S. New Sonographic Criteria for Recommending Fine-Needle Aspiration Biopsy of Nonpalpable Solid Nodules of the Thyroid. Am. J. Roentgenol. 2002, 178, 687–691. [Google Scholar] [CrossRef] [PubMed]
- Frates, M.C.; Benson, C.B.; Doubilet, P.M.; Cibas, E.S.; Marqusee, E. Can Color Doppler Sonography Aid in the Prediction of Malignancy of Thyroid Nodules? J. Ultrasound Med. 2003, 22, 127–131. [Google Scholar] [CrossRef]
- Brander, A.; Viikinkoski, P.; Nickels, J.; Kivisaari, L. Thyroid gland: US screening in a random adult population. Radiology 1991, 181, 683–687. [Google Scholar] [CrossRef] [PubMed]
- Ezzat, S. Thyroid incidentalomas. Prevalence by palpation and ultrasonography. Arch. Intern. Med. 1994, 154, 1838–1840. [Google Scholar] [CrossRef] [PubMed]
- Tomimori, E.; Pedrinola, F.; Cavaliere, H.; Knobel, M.; Medeiros-Neto, G. Prevalence of Incidental Thyroid Disease in a Relatively Low Iodine Intake Area. Thyroid 1995, 5, 273–276. [Google Scholar] [CrossRef]
- Gnarini, V.L.; Brigante, G.; Della Valle, E.; Diazzi, C.; Madeo, B.; Carani, C.; Rochira, V.; Simoni, M. Very high prevalence of ultrasound thyroid scan abnormalities in healthy volunteers inModena, Italy. J. Endocrinol. Investig 2013, 36, 722–728. [Google Scholar] [CrossRef]
- Youserm, D.M.; Huang, T.; A Loevner, L.; Langlotz, C.P. Clinical and economic impact of incidental thyroid lesions found with CT and MR. Am. J. Neuroradiol. 1997, 18, 1423–1428. [Google Scholar] [PubMed]
- Yoon, D.Y.; Chang, S.K.; Choi, C.S.; Yun, E.J.; Seo, Y.L.; Nam, E.S.; Cho, S.J.; Rho, Y.-S.; Ahn, H.Y. The Prevalence and Significance of Incidental Thyroid Nodules Identified on Computed Tomography. J. Comput. Assist. Tomogr. 2008, 32, 810–815. [Google Scholar] [CrossRef]
- Shie, P.; Cardarelli, R.; Sprawls, K.; Fulda, K.G.; Taur, A. Systematic review: Prevalence of malignant incidental thyroid nodules identified on fluorine-18 fluorodeoxyglucose positron emission tomography. Nucl. Med. Commun. 2009, 30, 742–748. [Google Scholar] [CrossRef]
- Soelberg, K.K.; Bonnema, S.J.; Brix, T.H.; Hegedüs, L. Risk of Malignancy in Thyroid Incidentalomas Detected by 18F-Fluorodeoxyglucose Positron Emission Tomography: A Systematic Review. Thyroid 2012, 22, 918–925. [Google Scholar] [CrossRef]
- Russ, G.; Bonnema, S.J.; Erdogan, M.F.; Durante, C.; Ngu, R.; Leenhardt, L. European Thyroid Association Guidelines for Ultrasound Malignancy Risk Stratification of Thyroid Nodules in Adults: The EU-TIRADS. Eur. Thyroid. J. 2017, 6, 225–237. [Google Scholar] [CrossRef] [PubMed]
- Haugen, B.R.; Alexander, E.K.; Bible, K.C.; Doherty, G.M.; Mandel, S.J.; Nikiforov, Y.E.; Pacini, F.; Randolph, G.W.; Sawka, A.M.; Schlumberger, M.; et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016, 26, 1–133. [Google Scholar] [CrossRef] [PubMed]
- Shin, J.H.; Baek, J.H.; Chung, J.; Ha, E.J.; Kim, J.-H.; Lee, Y.H.; Lim, H.K.; Moon, W.-J.; Na, D.G.; Park, J.S.; et al. Ultrasonography Diagnosis and Imaging-Based Management of Thyroid Nodules: Revised Korean Society of Thyroid Radiology Consensus Statement and Recommendations. Korean J. Radiol. 2016, 17, 370–395. [Google Scholar] [CrossRef]
- Gharib, H.; Papini, E.; Garber, J.R.; Duick, D.S.; Harrell, R.M.; Hegedüs, L.; Paschke, R.; Valcavi, R.; Vitti, P. American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi Medical Guidelines for Clinical Practice for the Diagnosis and Management of Thyroid Nodules—2016 Update Appendix. Endocr. Pract. 2016, 22, 1–60. [Google Scholar] [CrossRef]
- Tessler, F.N.; Middleton, W.D.; Grant, E.G.; Hoang, J.K.; Berland, L.L.; Teefey, S.A.; Cronan, J.J.; Beland, M.D.; Desser, T.S.; Frates, M.C.; et al. ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. J. Am. Coll. Radiol. 2017, 14, 587–595. [Google Scholar] [CrossRef] [PubMed]
- Zhou, J.; Yin, L.; Wei, X.; Zhang, S.; Song, Y.; Luo, B.; Li, J.; Qian, L.; Cui, L.; Chen, W.; et al. 2020 Chinese guidelines for ultrasound malignancy risk stratification of thyroid nodules: The C-TIRADS. Endocrine 2020, 70, 256–279. [Google Scholar] [CrossRef] [PubMed]
- Hoang, J.K.; Asadollahi, S.; Durante, C.; Hegedüs, L.; Papini, E.; Tessler, F.N. An International Survey on Utilization of Five Thyroid Nodule Risk Stratification Systems: A Needs Assessment with Future Implications. Thyroid 2022, 32, 675–681. [Google Scholar] [CrossRef]
- Durante, C.; Hegedüs, L.; Na, D.G.; Papini, E.; Sipos, J.A.; Baek, J.H.; Frasoldati, A.; Grani, G.; Grant, E.; Horvath, E.; et al. International Expert Consensus on US Lexicon for Thyroid Nodules. Radiology 2023, 309, e231481. [Google Scholar] [CrossRef]
- McHugh, M.L. Interrater reliability: The kappa statistic. Biochem. Medica 2012, 22, 276–282. [Google Scholar] [CrossRef]
- Grani, G.; Lamartina, L.; Cantisani, V.; Maranghi, M.; Lucia, P.; Durante, C. Interobserver agreement of various thyroid imaging reporting and data systems. Endocr. Connect. 2018, 7, 1–7. [Google Scholar] [CrossRef]
- Persichetti, A.; Di Stasio, E.; Coccaro, C.; Graziano, F.M.; Bianchini, A.; Di Donna, V.; Corsello, S.M.; Valle, D.; Bizzarri, G.; Frasoldati, A.; et al. Inter- and Intraobserver Agreement in the Assessment of Thyroid Nodule Ultrasound Features and Classification Systems: A Blinded Multicenter Study. Thyroid 2020, 30, 237–242. [Google Scholar] [CrossRef] [PubMed]
- Ha, E.J.; Chung, S.R.; Na, D.G.; Ahn, H.S.; Chung, J.; Lee, J.Y.; Park, J.S.; Yoo, R.-E.; Baek, J.H.; Baek, S.M.; et al. 2021 Korean Thyroid Imaging Reporting and Data System and Imaging-Based Management of Thyroid Nodules: Korean Society of Thyroid Radiology Consensus Statement and Recommendations. Korean J. Radiol. 2021, 22, 2094–2123. [Google Scholar] [CrossRef] [PubMed]
- Koh, J.; Kim, S.-Y.; Lee, H.S.; Kim, E.-K.; Kwak, J.Y.; Moon, H.J.; Yoon, J.H. Diagnostic performances and interobserver agreement according to observer experience: A comparison study using three guidelines for management of thyroid nodules. Acta Radiol. 2017, 59, 917–923. [Google Scholar] [CrossRef]
- Choi, S.H.; Kim, E.-K.; Kwak, J.Y.; Kim, M.J.; Son, E.J. Interobserver and Intraobserver Variations in Ultrasound Assessment of Thyroid Nodules. Thyroid 2010, 20, 167–172. [Google Scholar] [CrossRef]
- Solymosi, T.; Hegedűs, L.; Bonnema, S.J.; Frasoldati, A.; Jambor, L.; Karanyi, Z.; Kovacs, G.L.; Papini, E.; Rucz, K.; Russ, G.; et al. Considerable interobserver variation calls for unambiguous definitions of thyroid nodule ultrasound characteristics. Eur. Thyroid. J. 2023, 12, e220134. [Google Scholar] [CrossRef] [PubMed]
- Furuya-Kanamori, L.; Bell, K.J.; Clark, J.; Glasziou, P.; Doi, S.A. Prevalence of Differentiated Thyroid Cancer in Autopsy Studies Over Six Decades: A Meta-Analysis. J. Clin. Oncol. 2016, 34, 3672–3679. [Google Scholar] [CrossRef] [PubMed]
- Tappouni, R.R.; Itri, J.N.; McQueen, T.S.; Lalwani, N.; Ou, J.J. ACR TI-RADS: Pitfalls, Solutions, and Future Directions. Radiographics 2019, 39, 2040–2052. [Google Scholar] [CrossRef]
- Jegerlehner, S.; Bulliard, J.-L.; Aujesky, D.; Rodondi, N.; Germann, S.; Konzelmann, I.; Chiolero, A.; NICER Working Group. Overdiagnosis and overtreatment of thyroid cancer: A population-based temporal trend study. PLoS ONE 2017, 12, e0179387. [Google Scholar] [CrossRef]
- Wildman-Tobriner, B.; Ahmed, S.; Erkanli, A.; Mazurowski, M.A.; Hoang, J.K. Using the American College of Radiology Thyroid Imaging Reporting and Data System at the Point of Care: Sonographer Performance and Interobserver Variability. Ultrasound Med. Biol. 2020, 46, 1928–1933. [Google Scholar] [CrossRef] [PubMed]
- Kim, S.H.; Park, C.S.; Jung, S.L.; Kang, B.J.; Kim, J.Y.; Choi, J.J.; Kim, Y.I.; Oh, J.K.; Oh, J.S.; Kim, H.; et al. Observer Variability and the Performance between Faculties and Residents: US Criteria for Benign and Malignant Thyroid Nodules. Korean J. Radiol. 2010, 11, 149–155. [Google Scholar] [CrossRef]
- Beland, M.D.; Kwon, L.; Delellis, R.A.; Cronan, J.J.; Grant, E.G. Nonshadowing Echogenic Foci in Thyroid Nodules. J. Ultrasound Med. 2011, 30, 753–760. [Google Scholar] [CrossRef] [PubMed]
- Tahvildari, A.M.; Pan, L.; Kong, C.S.; Desser, T. Sonographic-Pathologic Correlation for Punctate Echogenic Reflectors in Papillary Thyroid Carcinoma. J. Ultrasound Med. 2016, 35, 1645–1652. [Google Scholar] [CrossRef]
- Piticchio, T.; Russ, G.; Radzina, M.; Frasca, F.; Durante, C.; Trimboli, P. Head-to-head comparison of American, European, and Asian TIRADSs in thyroid nodule assessment: Systematic review and meta-analysis. Eur. Thyroid. J. 2024, 13, e230242. [Google Scholar] [CrossRef] [PubMed]
- Tong, W.J.; Wu, S.H.; Cheng, M.Q.; Huang, H.; Liang, J.Y.; Li, C.Q.; Guo, H.L.; He, D.N.; Liu, Y.H.; Xiao, H.; et al. Integration of Artificial Intelligence Decision Aids to Reduce Workload and Enhance Efficiency in Thyroid Nodule Management. JAMA Netw. Open 2023, 6, e2313674. [Google Scholar] [CrossRef] [PubMed]
Parameter | Minimum | Maximum | Distribution † |
---|---|---|---|
Age (years) | 16 | 80 | 53 (16–80) |
Nodule size (mm) | 5 | 70 | 18 (5–70) |
Gender | p | ||
---|---|---|---|
Male (n = 86, 21.9%) | Female (n = 307, 78.1%) | ||
Distribution * | |||
Age (years) | 59.5 (27–80) | 83 (16–79) | <0.001 a |
Nodule size (mm) | 23 (7–70) | 17 (5–80) | 0.003 a |
Number of multiple nodules | 79(91.9%) | 273 (89.9%) | |
Number of solitary nodules | 7(8.1%) | 34(11.1%) | |
Nodule size (categorized) | |||
<1 cm | 5 (5.8%) | 31 (10.1%) | 0.224 b |
≥1 cm | 81 (94.2%) | 276 (89.9%) |
US Feature | Interobserver Agreement (%) | Κ | %95 CI | p Value |
---|---|---|---|---|
Composition | 69.80% | 0.538 | 0.511–0.593 | <0.001 |
Echogenicity | 55.26% | 0.343 | 0.318–0.367 | <0.001 |
Margin | 77.78% | 0.431 | 0.398–0.464 | <0.001 |
Direction of growth | 90.88% | 0.481 | 0.441–0.521 | <0.001 |
Echogenic foci/calcifications | 58.78% | 0.389 | 0.364–0.413 | <0.001 |
Extrathyroidal extension | 87.32% | 0.333 | 0.298–0.368 | <0.001 |
Lymph nodes | 88.25% | 0.452 | 0.419–0.485 | <0.001 |
US Feature | Interobserver Agreement (%) | Κ | %95 CI | p Value |
---|---|---|---|---|
Composition | 81.59% | 0.728 | 0.691–0.765 | <0.001 |
Echogenicity | 73.54% | 0.565 | 0.527–0.603 | <0.001 |
Margin | 83.12% | 0.585 | 0.539–0.632 | <0.001 |
Direction of growth | 92.88% | 0.657 | 0.600–0.715 | <0.001 |
Echogenic foci/calcifications Foci/calcifications | 77.44% | 0.391 | 0.367–0.415 | <0.001 |
Extrathyroidal extension | 91.09% | 0.571 | 0.521–0.622 | <0.001 |
Lymph nodes | 90.75% | 0.625 | 0.578–0.671 | <0.001 |
US Feature | Subgroup | Κ | %95 CI | p Value |
---|---|---|---|---|
Composition | Solid | 0.552 | 0.512–0.593 | <0.001 |
Mixed predominantly solid | 0.498 | 0.457–0.538 | <0.001 | |
Mixed predominantly cystic | 0.729 | 0.689–0.77 | <0.001 | |
Spongiform | 0.497 | 0.456–0.537 | <0.001 | |
Pure cyst | 0.292 | 0.252–0.333 | <0.001 | |
Echogenicity | Markedly hypoechoic | 0.514 | 0.474–0.554 | <0.001 |
Mildly hypoechoic | 0.352 | 0.311–0.392 | <0.001 | |
Isoechoic | 0.349 | 0.309–0.389 | <0.001 | |
Hyperechoic | 0.151 | 0.11–0.191 | <0.001 | |
Anechoic | 0.341 | 0.3–0.381 | <0.001 | |
Margin | Irregular | 0.352 | 0.311–0.392 | <0.001 |
defined | 0.361 | 0.321–0.402 | <0.001 | |
Smooth | 0.511 | 0.471–0.552 | <0.001 | |
Direction of Growth | Taller-than-wide | 0.481 | 0.441–0.521 | <0.001 |
Wider-than-tall | 0.481 | 0.441–0.521 | <0.001 | |
Echogenic Foci/Calcifications | Punctate echogenic foci/microcalcifications | 0.448 | 0.407–0.488 | <0.001 |
Macrocalcifications | 0.532 | 0.491–0.572 | <0.001 | |
Peripheral (rim) calcifications | 0.321 | 0.281–0.362 | <0.001 | |
Echogenic foci with comet-tail artifacts | 0.394 | 0.354–0.434 | <0.001 | |
Without foci | 0.275 | 0.235–0.315 | <0.001 | |
Extrathyroidal extension | Gross extrathyroidal extension | 0.298 | 0.257–0.338 | <0.001 |
Suspicious minor extrathyroidal extension | 0.288 | 0.247–0.328 | <0.001 | |
Capsula contact | 0.378 | 0.338–0.419 | <0.001 | |
Lymph Nodes | Suspicious lymph node | 0.483 | 0.442–0.523 | <0.001 |
Indeterminate lymph node | 0.364 | 0.323–0.404 | <0.001 | |
Nonsuspicious lymph node | 0.504 | 0.464–0.545 | <0.001 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Uysal, E.; Yangoz, B.; Sagan, M.; Duman, I.; Alparslan, A.S. An Interobserver Comparison of the Ultrasound Lexicon Classification of Thyroid Nodules: A Single-Center Prospective Validation Study. J. Clin. Med. 2025, 14, 1222. https://doi.org/10.3390/jcm14041222
Uysal E, Yangoz B, Sagan M, Duman I, Alparslan AS. An Interobserver Comparison of the Ultrasound Lexicon Classification of Thyroid Nodules: A Single-Center Prospective Validation Study. Journal of Clinical Medicine. 2025; 14(4):1222. https://doi.org/10.3390/jcm14041222
Chicago/Turabian StyleUysal, Ender, Burak Yangoz, Mustafa Sagan, Ismet Duman, and Ahmet Sukru Alparslan. 2025. "An Interobserver Comparison of the Ultrasound Lexicon Classification of Thyroid Nodules: A Single-Center Prospective Validation Study" Journal of Clinical Medicine 14, no. 4: 1222. https://doi.org/10.3390/jcm14041222
APA StyleUysal, E., Yangoz, B., Sagan, M., Duman, I., & Alparslan, A. S. (2025). An Interobserver Comparison of the Ultrasound Lexicon Classification of Thyroid Nodules: A Single-Center Prospective Validation Study. Journal of Clinical Medicine, 14(4), 1222. https://doi.org/10.3390/jcm14041222