The Diagnostic Efficiency of Ultrasound Computer–Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis
Abstract
:1. Introduction
2. Materials and Methods
2.1. Literature Search
2.2. Inclusion and Exclusion Criteria
2.3. Data Extraction
2.4. Quality Assessment
2.5. Data Analysis Approach
3. Results
3.1. Literature Search
3.2. Study Characteristics
3.3. Quality Assessment
3.4. Study Findings
3.5. Performance of Sole Computerised Ultrasound Features
3.5.1. Echogenicity
3.5.2. Echogenic Foci
3.5.3. Doppler Ultrasound Feature
3.6. General Performance of CAD
3.6.1. Performance between CAD and Radiologists (Clinicians)
3.6.2. Performance Based on Different TIRADS Guidelines
4. Discussion
4.1. Overview of Principal Findings
4.2. Potential Factors that May Influence CAD Diagnostic Performance
4.3. Clinical Implications and Suggested Directions for Future Research
4.4. Strengths and Limitations
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Brito, J.P.; Gionfriddo, M.R.; Al Nofal, A.; Boehmer, K.R.; Leppin, A.L.; Reading, C.; Callstrom, M.; Elraiyah, T.A.; Prokop, L.J.; Stan, M.N.; et al. The Accuracy of Thyroid Nodule Ultrasound to Predict Thyroid Cancer: Systematic Review and Meta–Analysis. J. Clin. Endocrinol. Metab. 2014, 99, 1253–1263. [Google Scholar] [CrossRef]
- Brito, J.P.; Morris, J.C.; Montori, V.M. Thyroid cancer: Zealous imaging has increased detection and treatment of low risk tumours. BMJ (Online) 2013, 347, 1–6. [Google Scholar] [CrossRef]
- Hoang, J.K.; Nguyen, X.V.; Davies, L. Overdiagnosis of Thyroid Cancer. Acad. Radiol. 2015, 22, 1024–1029. [Google Scholar] [CrossRef]
- Durante, C.; Grani, G.; Lamartina, L.; Filetti, S.; Mandel, S.J.; Cooper, D.S. The Diagnosis and Management of Thyroid Nodules. JAMA 2018, 319, 914–924. [Google Scholar] [CrossRef]
- Haugen, B.R.; Alexander, E.K.; Bible, K.C.; Doherty, G.M.; Mandel, S.J.; Nikiforov, Y.E.; Pacini, F.; Randolph, G.W.; Sawka, A.M.; Schlumberger, M.; et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer the American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016, 26, 1–133. [Google Scholar] [CrossRef]
- Gharib, H.; Papini, E.; Garber, J.R.; Duick, D.S.; Harrell, R.M.; Hegedüs, L.; Paschke, R.; Valcavi, R.; Vitti, P. AACE/ACE/AME Guidelines American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi Medical Guidelines for Clinical Practice for the Diagnosis and Management of Thyroid Nodules–2016 Update AP. Endocr. Pract. 2016, 22, 622–639. [Google Scholar] [CrossRef]
- Sebag, F.; Vaillant–Lombard, J.; Berbis, J.; Griset, V.; Henry, J.F.; Petit, P.; Oliver, C. Shear Wave Elastography: A New Ultrasound Imaging Mode for the Differential Diagnosis of Benign and Malignant Thyroid Nodules. J. Clin. Endocrinol. Metab. 2010, 95, 5281–5288. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Xie, C.; Cox, P.; Taylor, N.; LaPorte, S. Ultrasonography of thyroid nodules: A pictorial review. Insights Imaging 2016, 7, 77–86. [Google Scholar] [CrossRef] [PubMed]
- Tessler, F.N.; Middleton, W.D.; Grant, E.G.; Hoang, J.K.; Berland, L.L.; Teefey, S.A.; Cronan, J.J.; Beland, M.D.; Desser, T.S.; Frates, M.C.; et al. ACR Thyroid Imaging, Reporting and Data System (TI–RADS): White Paper of the ACR TI–RADS Committee. J. Am. Coll. Radiol. 2017, 14, 587–595. [Google Scholar] [CrossRef]
- Kwak, J.Y.; Jung, I.; Baek, J.H.; Baek, S.M.; Choi, N.; Choi, Y.J.; Jung, S.L.; Kim, E.K.; Kim, J.A.; Kim, J.H.; et al. Image reporting and characterization system for ultrasound features of thyroid nodules: Multicentric Korean retrospective study. Korean J. Radiol. 2013, 14, 110–117. [Google Scholar] [CrossRef]
- Shin, J.H.; Baek, J.H.; Chung, J.; Ha, E.J.; Kim, J.H.; Lee, Y.H.; Lim, H.K.; Moon, W.J.; Na, D.G.; Park, J.S.; et al. Ultrasonography diagnosis and imaging–based management of thyroid nodules: Revised Korean society of thyroid radiology consensus statement and recommendations. Korean J. Radiol. 2016, 17, 370–395. [Google Scholar] [CrossRef] [PubMed]
- Russ, G.; Bonnema, S.J.; Erdogan, M.F.; Durante, C.; Ngu, R.; Leenhardt, L. European Thyroid Association Guidelines for Ultrasound Malignancy Risk Stratification of Thyroid Nodules in Adults: The EU–TIRADS. Eur. Thyroid J. 2017, 6, 225–237. [Google Scholar] [CrossRef] [PubMed]
- Faust, O.; Acharya, U.R.; Tamura, T. Formal design methods for reliable computer–aided diagnosis: A review. IEEE Rev. Biomed. Eng. 2012, 5, 15–28. [Google Scholar] [CrossRef] [PubMed]
- Chang, T.-C. The Role of Computer—Aided Detection and Diagnosis System in the Differential Diagnosis of Thyroid Lesions in Ultrasonography. J. Med. Ultrasound 2015, 23, 177–184. [Google Scholar] [CrossRef]
- Ying, M.; Ng, D.K.; Yung, D.M.; Lee, E.S. A semi—Quantitative approach to compare high—Sensitivity power Doppler sonography and conventional power Doppler sonography in the assessment of thyroid vascularity. Thyroid 2009, 19, 1265–1269. [Google Scholar] [CrossRef] [PubMed]
- Lam, J.; Ying, M.; Cheung, S.Y.; Yeung, K.H.; Yu, P.H.; Cheng, H.C.; Ahuja, A.T. A Comparison of the Diagnostic Accuracy and Reliability of Subjective Grading and Computer—Aided Assessment of Intranodal Vascularity in Differentiating Metastatic and Reactive Cervical Lymphadenopathy. Ultraschall Med. 2016, 37, 63–67. [Google Scholar] [CrossRef]
- Sollini, M.; Cozzi, L.; Chiti, A.; Kirienko, M. Texture analysis and machine learning to characterize suspected thyroid nodules and differentiated thyroid cancer: Where do we stand? Eur. J. Radiol. 2018, 99, 1–8. [Google Scholar] [CrossRef]
- Acharya, U.R.; Swapna, G.; Sree, S.V.; Molinari, F.; Gupta, S.; Bardales, R.H.; Witkowska, A.; Suri, J.S. A Review on Ultrasound–Based Thyroid Cancer Tissue Characterization and Automated Classification. Technol. Cancer Res. Treat. 2014, 13, 289–301. [Google Scholar] [CrossRef]
- Faust, O.; Acharya, U.R.; Meiburger, K.M.; Molinari, F.; Koh, J.E.W.; Yeong, C.H.; Kongmebhol, P.; Ng, K.H. Comparative assessment of texture features for the identification of cancer in ultrasound images: A review. Biocybern. Biomed. Eng. 2018, 38, 275–296. [Google Scholar] [CrossRef]
- Zhao, W.-J.; Fu, L.-R.; Huang, Z.-M.; Zhu, J.-Q.; Ma, B.-Y. Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound. Medicine 2019, 98, 1–13. [Google Scholar] [CrossRef]
- McInnes, M.D.F.; Moher, D.; Thombs, B.D.; McGrath, T.A.; Bossuyt, P.M.; Clifford, T.; Cohen, J.F.; Deeks, J.J.; Gatsonis, C.; Hooft, L.; et al. Preferred Reporting Items for a Systematic Review and Meta—Analysis of Diagnostic Test Accuracy Studies The PRISMA–DTA Statement. JAMA J. Am. Med. Assoc. 2018, 319, 388–396. [Google Scholar] [CrossRef] [PubMed]
- Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Prisma Group, T.P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, 264–269. [Google Scholar] [CrossRef] [PubMed]
- Whiting, P.F.; Rutjes, A.W.S.; Westwood, M.E.; Mallett, S.; Deeks, J.J.; Reitsma, J.B.; Leeflang, M.M.G.; Sterne, J.A.C.; Bossuyt, P.M.M. Quadas-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann. Intern. Med. 2011, 155, 529–536. [Google Scholar] [CrossRef]
- Maki, K.; Okumura, Y.; Sato, S.; Yoneda, A.; Kurose, T.; Iguchi, T.; Akaki, S.; Takeda, Y.; Kanazawa, S.; Hiraki, Y. Quantitative evaluation by Tl–201 scintigraphy in the diagnosis of thyroid follicular nodules. Ann. Nucl. Med. 2003, 17, 91–98. [Google Scholar] [CrossRef]
- Palermo, F. Diagnostic efficacy of dynamic radiothallium uptake in thyroid nodules determined by computer-assisted scintigraphy. Reevaluation of a radioisotopic procedure. Nuklearmed. Nucl. Med. 1989, 28, 114–119. [Google Scholar]
- Ding, J.; Cheng, H.D.; Huang, J.; Zhang, Y.; Liu, J. An improved quantitative measurement for thyroid cancer detection based on elastography. Eur. J. Radiol. 2012, 81, 800–805. [Google Scholar] [CrossRef]
- Peng, W.; Liu, C.; Xia, S.; Shao, D.; Chen, Y.; Liu, R.; Zhang, Z. Thyroid nodule recognition in computed tomography using first order statistics. Biomed. Eng. Online 2017, 16, 67. [Google Scholar] [CrossRef]
- Granchi, S.; Vannacci, E.; Biagi, E. Characterization of benign thyroid nodules with HyperSPACE (Hyper Spectral Analysis for Characterization in Echography) before and after percutaneous laser ablation: A pilot study. Med. Ultrasonogr. 2017, 19, 172–178. [Google Scholar] [CrossRef]
- Lee, J.H.; Baek, J.H.; Kim, J.H.; Shim, W.H.; Chung, S.R.; Choi, Y.J.; Lee, J.H. Deep Learning–Based Computer–Aided Diagnosis System for Localization and Diagnosis of Metastatic Lymph Nodes on Ultrasound: A Pilot Study. Thyroid 2018, 28, 1332–1338. [Google Scholar] [CrossRef]
- Lavarello, R.J.; Ridgway, W.R.; Sarwate, S.S.; Oelze, M.L. Characterization of thyroid cancer in mouse models using high–frequency quantitative ultrasound techniques. Ultrasound Med. Biol. 2013, 39, 2333–2341. [Google Scholar] [CrossRef]
- Wang, W.; Ozolek, J.A.; Slepcev, D.; Lee, A.B.; Chen, C.; Rohde, G.K. An optimal transportation approach for nuclear structure–based pathology. IEEE Trans. Med. Imaging 2011, 30, 621–631. [Google Scholar] [CrossRef] [PubMed]
- Choi, J.W.; Lee, J.H. Response to a Letter Regarding Diagnostic Accuracy of Ultrasound and 18–F–FDG PET or PET/CT for Patients with Suspected Recurrent Papillary Thyroid Carcinoma. Ultrasound Med. Biol. 2011, 37, 841. [Google Scholar] [CrossRef]
- Iakovidis, D.K.; Keramidas, E.G.; Maroulis, D. Fusion of fuzzy statistical distributions for classification of thyroid ultrasound patterns. Artif. Intell. Med. 2010, 50, 33–41. [Google Scholar] [CrossRef] [PubMed]
- Song, G.; Xue, F.; Zhang, C. A Model Using Texture Features to Differentiate the Nature of Thyroid Nodules on Sonography. J. Ultrasound Med. 2015, 34, 1753–1760. [Google Scholar] [CrossRef]
- Bibicu, D.; Moraru, L.; Biswas, A. Thyroid Nodule Recognition Based on Feature Selection and Pixel Classification Methods. J. Digit. Imaging 2012, 26, 119–128. [Google Scholar] [CrossRef] [Green Version]
- Liu, Y.I.; Kamaya, A.; Desser, T.S.; Rubin, D.L. A bayesian network for differentiating benign from malignant thyroid nodules using sonographic and demographic features. AJR Am. J. Roentgenol. 2011, 196, 598–605. [Google Scholar] [CrossRef]
- Savelonas, M.; Maroulis, D.; Sangriotis, M. A computer—Aided system for malignancy risk assessment of nodules in thyroid US images based on boundary features. Comput. Methods Programs Biomed. 2009, 96, 25–32. [Google Scholar] [CrossRef]
- Raghavendra, U.; Acharya, U.R.; Gudigar, A.; Tan, J.H.; Fujita, H.; Hagiwara, Y.; Molinari, F.; Kongmebhol, P.; Ng, K.H. Fusion of spatial gray level dependency and fractal texture features for the characterization of thyroid lesions. Ultrasonics 2017, 77, 110–120. [Google Scholar] [CrossRef]
- Ardakani, A.A.; Gharbali, A.; Mohammadi, A. Application of texture analysis method for classification of benign and malignant thyroid nodules in ultrasound images. Iran. J. Cancer Prev. 2015, 8, 116–124. [Google Scholar]
- Acharya, U.R.; Faust, O.; Sree, S.V.; Molinari, F.; Garberoglio, R.; Suri, J.S. Cost—Effective and non–invasive automated benign & malignant thyroid lesion classification in 3D contrast—Enhanced ultrasound using combination of wavelets and textures: A class of thyroscan™ algorithms. Technol. Cancer Res. Treat. 2011, 10, 371–380. [Google Scholar] [CrossRef]
- Acharya, U.R.; Molinari, F.; Garberoglio, R.; Witkowska, A.; Suri, J.S. Automated benign & malignant thyroid lesion characterization and classification in 3D contrast–enhanced ultrasound. In Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, 28 August–1 September 2012; pp. 452–455. [Google Scholar]
- Acharya, U.R.; Faust, O.; Sree, S.V.; Molinari, F.; Suri, J.S. ThyroScreen system: High resolution ultrasound thyroid image characterization into benign and malignant classes using novel combination of texture and discrete wavelet transform. Comput. Methods Programs Biomed. 2012, 107, 233–241. [Google Scholar] [CrossRef] [PubMed]
- Ardakani, A.A.; Mohammadzadeh, A.; Yaghoubi, N.; Ghaemmaghami, Z.; Reiazi, R.; Jafari, A.H.; Hekmat, S.; Shiran, M.B.; Bitarafan-Rajabi, A. Predictive quantitative sonographic features on classification of hot and cold thyroid nodules. Eur. J. Radiol. 2018, 101, 170–177. [Google Scholar] [CrossRef] [PubMed]
- Abbasian Ardakani, A.; Reiazi, R.; Mohammadi, A. A Clinical Decision Support System Using Ultrasound Textures and Radiologic Features to Distinguish Metastasis from Tumor—Free Cervical Lymph Nodes in Patients with Papillary Thyroid Carcinoma. J. Ultrasound Med. 2018, 37, 2527–2535. [Google Scholar] [CrossRef]
- Kim, S.-Y.; Kim, E.-K.; Moon, H.J.; Yoon, J.H.; Kwak, J.Y. Application of Texture Analysis in the Differential Diagnosis of Benign and Malignant Thyroid Nodules: Comparison with Gray-Scale Ultrasound and Elastography. AJR Am. J. Roentgenol. 2015, 205, 343–351. [Google Scholar] [CrossRef]
- Prochazka, A.; Gulati, S.; Holinka, S.; Smutek, D. Classification of Thyroid Nodules in Ultrasound Images Using Direction—Independent Features Extracted by Two—Threshold Binary Decomposition. Technol. Cancer Res. Treat. 2019, 18. [Google Scholar] [CrossRef]
- Prochazka, A.; Gulati, S.; Holinka, S.; Smutek, D. Patch–based classification of thyroid nodules in ultrasound images using direction independent features extracted by two–threshold binary decomposition. Comput. Med. Imaging Graph. 2019, 71, 9–18. [Google Scholar] [CrossRef]
- Ardakani, A.A.; Gharbali, A.; Mohammadi, A. Classification of Benign and Malignant Thyroid Nodules Using Wavelet Texture Analysis of Sonograms. J. Ultrasound Med. 2015, 34, 1983–1989. [Google Scholar] [CrossRef]
- Yu, Q.; Jiang, T.; Zhou, A.; Zhang, L.; Zhang, C.; Xu, P. Computer–aided diagnosis of malignant or benign thyroid nodes based on ultrasound images. Eur. Arch. Oto Rhino Laryngol. 2017, 274, 2891–2897. [Google Scholar] [CrossRef]
- Lim, K.J.; Choi, C.S.; Yoon, D.Y.; Chang, S.K.; Kim, K.K.; Han, H. Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Acad. Radiol. 2008, 15, 853–858. [Google Scholar] [CrossRef]
- Chi, J.; Walia, E.; Babyn, P.; Wang, J.; Groot, G.; Eramian, M. Thyroid Nodule Classification in Ultrasound Images by Fine–Tuning Deep Convolutional Neural Network. J. Digit. Imaging 2017, 30, 477–486. [Google Scholar] [CrossRef]
- Galimzianova, A.; Siebert, S.M.; Kamaya, A.; Desser, T.S.; Rubin, D.L. Toward Automated Pre–Biopsy Thyroid Cancer Risk Estimation in Ultrasound. AMIA Annu. Symp. Proc. AMIA Symp. 2017, 2017, 734–741. [Google Scholar] [PubMed]
- Maroulis, D.E.; Savelonas, M.A.; Iakovidis, D.K.; Karkanis, S.A.; Dimitropoulos, N. Variable background active contour model for computer-aided delineation of nodules in thyroid ultrasound images. IEEE Trans. Inf. Technol. Biomed. Publ. IEEE Eng. Med. Biol. Soc. 2007, 11, 537–543. [Google Scholar] [CrossRef]
- Tsantis, S.; Cavouras, D.; Kalatzis, I.; Piliouras, N.; Dimitropoulos, N.; Nikiforidis, G. Development of a support vector machine-based image analysis system for assessing the thyroid nodule malignancy risk on ultrasound. Ultrasound Med. Boil. 2005, 31, 1451–1459. [Google Scholar] [CrossRef] [PubMed]
- Hirning, T.; Zuna, I.; Schlaps, D.; Lorenz, D.; Meybier, H.; Tschahargane, C.; van Kaick, G. Quantification and classification of echographic findings in the thyroid gland by computerized B–mode texture analysis. Eur. J. Radiol. 1989, 9, 244–247. [Google Scholar] [PubMed]
- Ma, J.; Wu, F.; Jiang, T.A.; Zhao, Q.; Kong, D. Ultrasound image–based thyroid nodule automatic segmentation using convolutional neural networks. Int. J. Comput. Assist. Radiol. Surg. 2017, 12, 1895–1910. [Google Scholar] [CrossRef] [PubMed]
- Keramidas, E.G.; Maroulis, D.; Iakovidis, D.K. TND: A thyroid nodule detection system for analysis of ultrasound images and videos. J. Med. Syst. 2012, 36, 1271–1281. [Google Scholar] [CrossRef] [PubMed]
- Chang, Y.; Paul, A.K.; Kim, N.; Baek, J.H.; Choi, Y.J.; Ha, E.J.; Lee, K.D.; Lee, H.S.; Shin, D.; Kim, N. Computer-aided diagnosis for classifying benign versus malignant thyroid nodules based on ultrasound images: A comparison with radiologist–based assessments. Med. Phys. 2016, 43, 554–567. [Google Scholar] [CrossRef]
- Ding, J.; Cheng, H.; Ning, C.; Huang, J.; Zhang, Y. Quantitative measurement for thyroid cancer characterization based on elastography. J. Ultrasound Med. 2011, 30, 1259–1266. [Google Scholar] [CrossRef]
- Koundal, D.; Gupta, S.; Singh, S. Computer aided thyroid nodule detection system using medical ultrasound images. Biomed. Signal Process. Control 2018, 40, 117–130. [Google Scholar] [CrossRef] [Green Version]
- Lin, Y.-H.; Tsai, Y.-C.; Lin, K.J.; Der Lin, J.; Wang, C.-C.; Chen, S.-T. Computer-Aided Diagnostic Technique in 2-Deoxy-2-[18F] fluoro-d-glucose-Positive Thyroid Nodule: Clinical Experience of 74 Non-thyroid Cancer Patients. Ultrasound Med. Biol. 2019, 45, 108–121. [Google Scholar] [CrossRef]
- Wu, M.-H.; Chen, C.-N.; Chen, K.-Y.; Ho, M.-C.; Tai, H.-C.; Chung, Y.-C.; Lo, C.-P.; Chen, A.; Chang, K.-J. Quantitative Analysis of Dynamic Power Doppler Sonograms for Patients with Thyroid Nodules. Ultrasound Med. Biol. 2013, 39, 1543–1551. [Google Scholar] [CrossRef] [PubMed]
- Wu, M.H.; Chen, C.N.; Chen, K.Y.; Ho, M.C.; Tai, H.C.; Wang, Y.H.; Chen, A.; Chang, K.J. Quantitative analysis of echogenicity for patients with thyroid nodules. Sci. Rep. 2016, 6, 35632. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chen, K.Y.; Chen, C.N.; Wu, M.H.; Ho, M.C.; Tai, H.C.; Huang, W.C.; Chung, Y.C.; Chen, A.; Chang, K.J. Computerized Detection and Quantification of Microcalcifications in Thyroid Nodules. Ultrasound Med. Biol. 2011, 37, 870–878. [Google Scholar] [CrossRef]
- Yoo, Y.J.; Ha, E.J.; Cho, Y.J.; Kim, H.L.; Han, M.; Kang, S.Y. Computer–Aided Diagnosis of Thyroid Nodules via Ultrasonography: Initial Clinical Experience. Korean J. Radiol. 2018, 19, 665–672. [Google Scholar] [CrossRef]
- Baig, F.N.; Van Lunenburg, J.T.J.; Liu, S.Y.W.; Yip, S.P.; Law, H.K.W.; Ying, M. Computer–aided assessment of regional vascularity of thyroid nodules for prediction of malignancy. Sci. Rep. 2017, 7, e14350. [Google Scholar] [CrossRef]
- Choi, Y.J.; Baek, J.H.; Park, H.S.; Shim, W.H.; Kim, T.Y.; Shong, Y.K.; Lee, J.H. A Computer–Aided Diagnosis System Using Artificial Intelligence for the Diagnosis and Characterization of Thyroid Nodules on Ultrasound: Initial Clinical Assessment. Thyroid 2017, 27, 546–552. [Google Scholar] [CrossRef]
- Jeong, E.Y.; Kim, H.L.; Ha, E.J.; Park, S.Y.; Cho, Y.J.; Han, M. Computer–aided diagnosis system for thyroid nodules on ultrasonography: Diagnostic performance and reproducibility based on the experience level of operators. Eur. Radiol. 2019, 29, 1978–1985. [Google Scholar] [CrossRef]
- Lyshchik, A.; Moses, R.; Barnes, S.L.; Higashi, T.; Asato, R.; Miga, M.I.; Gore, J.C.; Fleischer, A.C. Quantitative Analysis of Tumor Vascularity in Benign and Malignant Solid Thyroid Nodules. J. Ultrasound Med. 2007, 26, 837–846. [Google Scholar] [CrossRef]
- Gao, L.; Liu, R.; Jiang, Y.; Song, W.; Wang, Y.; Liu, J.; Wang, J.; Wu, D.; Li, S.; Hao, A.; et al. Computer–aided system for diagnosing thyroid nodules on ultrasound: A comparison with radiologist–based clinical assessments. Head Neck 2018, 40, 778–783. [Google Scholar] [CrossRef]
- Gitto, S.; Grassi, G.; De Angelis, C.; Monaco, C.G.; Sdao, S.; Sardanelli, F.; Sconfienza, L.M.; Mauri, G. A computer–aided diagnosis system for the assessment and characterization of low–to–high suspicion thyroid nodules on ultrasound. Radiol. Med. 2019, 124, 118–125. [Google Scholar] [CrossRef]
- Choi, W.J.; Park, J.S.; Kim, K.G.; Kim, S.Y.; Koo, H.R.; Lee, Y.J. Computerized analysis of calcification of thyroid nodules as visualized by ultrasonography. Eur. J. Radiol. 2015, 84, 1949–1953. [Google Scholar] [CrossRef] [PubMed]
- Sultan, L.R.; Xiong, H.; Zafar, H.M.; Schultz, S.M.; Langer, J.E.; Sehgal, C.M. Vascularity Assessment of Thyroid Nodules by Quantitative Color Doppler Ultrasound. Ultrasound Med. Biol. 2015, 41, 1287–1293. [Google Scholar] [CrossRef] [PubMed]
- Reverter, J.L.; Vázquez, F.; Puig-Domingo, M. Diagnostic Performance Evaluation of a Computer–Assisted Imaging Analysis System for Ultrasound Risk Stratification of Thyroid Nodules. Am. J. Roentgenol. 2019, 1–6. [Google Scholar] [CrossRef] [PubMed]
- Wang, L.; Yang, S.; Yang, S.; Zhao, C.; Tian, G.; Gao, Y.; Chen, Y.; Lu, Y. Automatic thyroid nodule recognition and diagnosis in ultrasound imaging with the YOLOv2 neural network. World J. Surg. Oncol. 2019, 17, e12. [Google Scholar] [CrossRef]
- Na, D.G.; Baek, J.H.; Sung, J.Y.; Kim, J.-H.; Kim, J.K.; Choi, Y.J.; Seo, H. Thyroid Imaging Reporting and Data System Risk Stratification of Thyroid Nodules: Categorization Based on Solidity and Echogenicity. Thyroid 2016, 26, 562–572. [Google Scholar] [CrossRef]
- Ying, M.; Cheng, S.C.H.; Ahuja, A.T. Diagnostic accuracy of computer–aided assessment of intranodal vascularity in distinguishing different causes of cervical lymphadenopathy. Ultrasound Med. Biol. 2016, 42, 2010–2016. [Google Scholar] [CrossRef]
- Gao, L.; Xi, X.; Jiang, Y.; Yang, X.; Wang, Y.; Zhu, S.; Lai, X.; Zhang, X.; Zhao, R.; Zhang, B. Comparison among TIRADS (ACR TI–RADS and KWAK- TI–RADS) and 2015 ATA Guidelines in the diagnostic efficiency of thyroid nodules. Endocrine 2019, 64, 90–96. [Google Scholar] [CrossRef]
- Wang, Y.; Lei, K.R.; He, Y.P.; Li, X.L.; Ren, W.W.; Zhao, C.K.; Bo, X.W.; Wang, D.; Sun, C.Y.; Xu, H.X. Malignancy risk stratification of thyroid nodules: Comparisons of four ultrasound Thyroid Imaging Reporting and Data Systems in surgically resected nodules. Sci. Rep. 2017, 7, e11560. [Google Scholar] [CrossRef]
- Chng, C.L.; Tan, H.C.; Too, C.W.; Lim, W.Y.; Chiam, P.P.S.; Zhu, L.; Nadkarni, N.V.; Lim, A.Y.Y. Diagnostic performance of ATA, BTA and TIRADS sonographic patterns in the prediction of malignancy in histologically proven thyroid nodules. Singap. Med. J. 2018, 59, 578–583. [Google Scholar] [CrossRef] [Green Version]
- Wu, X.L.; Du, J.R.; Wang, H.; Jin, C.X.; Sui, G.Q.; Yang, D.Y.; Lin, Y.Q.; Luo, Q.; Fu, P.; Li, H.Q.; et al. Comparison and preliminary discussion of the reasons for the differences in diagnostic performance and unnecessary FNA biopsies between the ACR TIRADS and 2015 ATA guidelines. Endocrine 2019, 65, 121–131. [Google Scholar] [CrossRef]
- Grani, G.; Lamartina, L.; Ascoli, V.; Bosco, D.; Biffoni, M.; Giacomelli, L.; Maranghi, M.; Falcone, R.; Ramundo, V.; Cantisani, V.; et al. Reducing the number of unnecessary thyroid biopsies while improving diagnostic accuracy: Toward the “Right” TIRADS. J. Clin. Endocrinol. Metab. 2019, 104, 95–102. [Google Scholar] [CrossRef] [PubMed]
- Xu, T.; Wu, Y.; Wu, R.-X.; Zhang, Y.-Z.; Gu, J.-Y.; Ye, X.-H.; Tang, W.; Xu, S.-H.; Liu, C.; Wu, X.-H. Validation and comparison of three newly–released Thyroid Imaging Reporting and Data Systems for cancer risk determination. Endocrine 2018, 64, 1–9. [Google Scholar] [CrossRef] [PubMed]
- Xu, T.; Gu, J.Y.; Ye, X.H.; Xu, S.H.; Wu, Y.; Shao, X.Y.; Liu, D.Z.; Lu, W.P.; Hua, F.; Shi, B.M.; et al. Thyroid nodule sizes influence the diagnostic performance of TIRADS and ultrasound patterns of 2015 ATA guidelines: A multicenter retrospective study. Sci. Rep. 2017, 7, e43183. [Google Scholar] [CrossRef] [PubMed]
- Ozel, A.; Erturk, S.M.; Ercan, A.; Yilmaz, B.; Basak, T.; Karpat, Z.; Basak, M. The Diagnostic Efficiency of Ultrasound in Characterization of Thyroid Nodules: How Many Criteria Are Needed? Med. Ultrasonogr. 2012, 14, 24–28. [Google Scholar] [CrossRef]
Author(s) | Ref. Year | Patients Total (n) | Mean Age-Years (SD/Range) | Nodules (n) | Mean Size of Nodules-cm (SD) | |||||
---|---|---|---|---|---|---|---|---|---|---|
Overall | Benign | Malignant | Total | Benign | Malignant | Benign | Malignant | |||
Lyshchik et al. | 2007 [69] | 56 | 53.1 ± 11.6 | NA | NA | 86 | 40 | 46 | NA | NA |
Chen et al. | 2011 [64] | 225 | NA | 50.6 ± 12.52 | 46.7 ± 15.22 | 256 | 173 | 83 | 2.35 ± 0.98 | 1.94 ± 0.86 |
Wu et al. | 2013 [62] | 208 | 49.6 ± 13.4 | 51.0 ± 12.7 | 47.0 ± 14.2 | 238 | 159 | 79 | NA | NA |
Choi et al. | 2015 [72] | 85 | 52 (29–81) | NA | NA | 99 | 21 | 78 | NA | NA |
Sultan et al. | 2015 [73] | 99 | 54 ± 15.5 | 56.6 ± 14.6 | 50.7 ± 16.4 | 100 | 58 | 42 | 1.81 ± 0.73 | 1.77 ± 0.74 |
Wu et al. | 2016 [63] | 333 | 48.37 (11–81) | NA | NA | 411 | 254 | 157 | NA | NA |
Baig et al. | 2017 [66] | 111 | NA | 51.2 ± 12 | 56.6 ± 17.6 | 111 | 84 | 27 | NA | NA |
Gao et al. | 2018 [70] | 262 | NA | 48.4 ± 12.3 | 43.2 ± 10.4 | 342 | 103 | 239 | 1.7 ± 1.4 | 1 ± 0.7 |
Choi et al. | 2017 [67] | 89 | 45.3 | NA | NA | 102 | 59 | 43 | 1.5 ± 0.8 | 0.9 ± 0.4 |
Gitto et al. | 2019 [71] | 62 | 60 ± 12 | NA | NA | 62 | 48 | 14 | NA | NA |
Yoo et al. | 2018 [65] | 50 | 43.2 (22–81) | NA | NA | 117 | 67 | 50 | 1.2 ± 1.0 | 1.1 ± 0.8 |
Jeong et al. | 2019 [68] | 76 | 46 | NA | NA | 100 | 56 | 44 | 1.8 ± 0.8 | 1.5 ± 0.8 |
Reverter et al. | 2019 [74] | 300 | NA | 55 ± 11 | 56 ± 12 | 300 | 165 | 135 | 2.8 ± 0.4 | 3.2 ± 1.0 |
Wang et al. | 2019 [75] | 276 | 46.3 (20–71) | 50 ± 10.6 | 44.3 ± 11.5 | 351 | 109 | 242 | 3.37 ± 1.81 | 1.17 ± 0.87 |
Author(s) | Type of Ultrasound Machine | Type of CAD | Reference Standard | Diagnosis Parameter |
---|---|---|---|---|
Lyshchik et al., 2007 [69] | Siemens Sonoline Elegra with a 5–9MHz linear array transducer (7.5L40) | Algorithm for manual segmentation of tumor and Doppler quantification in MATLAB | Histopathology | Doppler–visual and quantitative intranodular vascularization (vascular index-VI) |
Chen et al., 2011 [64] | Philips HDI 5000 (2000 model) with a 5–12 MHz linear probe (L12–5) | AmCAD-UT (grey scale CAD of microcalcifications) | FNAC (75)/Histopathology (181) | Qualitative and computed calcification analysis (calcification index-CI) |
Wu et al., 2013 [62] | Philips HDI 5000 (2000 model) with a 5–12 MHz linear probe (L12–5) | Stand-alone AmCAD-UV (Doppler CAD) | FNAC/Histopathology | Doppler–quantitative intranodular vascularization (vascular index-VI) |
Choi et al., 2015 [72] | Philips HDI 5000 | CAD based on artificial intelligence for calcification assessment | Histopathology | Computed grey scale calcification analysis |
Sultan et al., 2015 [73] | Philips HDI 5000 (68), Philips iu22 (30), GE LOGIC E9, GE LOGIC 9 | IDL-based software computer program for vascular analysis | Histopathology/FNAC | Qualitative and quantitative vascular area analysis |
Wu et al., 2016 [63] | Philips HDI 5000 (2000 model) with a 5–12 MHz linear probe (L12–5) | Stand-alone AmCAD-UT (grey scale CAD of echogenicity) | FNAC/Histopathology | Qualitative and quantitative echogenicity analysis (echogenicity index-EI) |
Baig et al., 2017 [66] | Supersonic Imagine Aixplorer with 4–15 MHz linear transducer | Custom-developed Doppler algorithm for use in MATLAB | FNAC (62–benign)/Histopathology (49) | Quantitative regional Doppler vascularization analysis (vascular index-VI) |
Gao et al., 2018 [70] | Philips HDI 5000, GE Logiq 9 and GE Logiq 7 with a 5–12 MHz or 8–15 MHz linear array transducer | CAD-based on artificial intelligence | Histopathology | Qualitative and computed grey scale analysis |
Choi et al., 2017 [67] | Samsung RS80A with 3–12 MHz linear transducer | S-Detect for Thyroid CAD embedded in Samsung US scanner | Histopathology/FNAC/US findings | Qualitative and computed grey scale feature analysis |
Gitto et al., 2019 [71] | Samsung RS80A with 3–8 MHz linear transducer | S-Detect for Thyroid CAD embedded in Samsung US scanner | FNAC | Qualitative and computed grey scale feature analysis |
Yoo et al., 2018 [65] | Samsung RS80A with a 5–12 MHz linear probe (Samsung Medison Co., Ltd.) | S-Detect for Thyroid CAD embedded in Samsung US scanner | FNAC (14)/Histopathology (103) | Qualitative and computed grey scale feature analysis |
Jeong et al., 2019 [68] | Samsung RS80A with 5–12 MHz linear transducer | S-Detect for Thyroid CAD embedded in Samsung US scanner | Histopathology/FNAC | Qualitative and computed grey scale feature analysis |
Reverter et al., 2019 [74] | GE Logiq E9 with 5–15 MHz linear transducer | AmCAD-UT | Histopathology | Qualitative and computed grey scale analysis |
Wang et al., 2019 [75] | GE Logiq E8, Philips iE Elite, and Philips iU22 with a 6–15 MHz, 3–11 MHz or 5–12 MHz linear array transducer | CAD-based on artificial intelligence | Histopathology | Qualitative and computed grey scale analysis |
Author(s) | Diagnostic Criteria | SEN (%)–95% CI | SPEC (%)–95% CI | PPV (%)–95% CI | NPV (%)–95% CI | DA (%)–95% CI | AUC–95 CI |
---|---|---|---|---|---|---|---|
Lyshchik et al. [69] | Visual vascularization | 65.2 (49.75–78.65) | 52.5 (36.13–68.49) | * 61.22 (51.71–69.95) | * 56.76 (44.48–68.25) | 58.9 (48.17–69.78) | ND |
Visual <2 cm | 65.5 (45.67–82.06) | 85.7 (57.19–98.22) | * 90.48 (71.94–97.24) | * 54.55 (41.02–67.43) | 72.1 (56.33–84.67) | ||
Normalized VI >0.14 in <2 cm | 72.4 (52.76–87.27) | 100 (76.84–100) | * 100 | * 63.64 (49.25–75.94) | 86.2 (66.60–91.61) | ||
Weighted VI >0.24 in <2 cm | 69 (49.17–84.72) | 100 (76.84–100) | * 100 | * 60.87 (47.48–72.80) | 84.5 (63.96–89.96) | ||
Chen et al. [64] | Qualitative calcification | 48.2 (37.08–59.44) | 89 (83.38–93.26) | 67.8 (56.59–77.27) | 78.2 (74.30–81.60) | 75.8 (70.06–80.90) | ND |
CI threshold at 0.0089 | 63.9 (51.69–73.86) | 80.9 (71.35–87.59) | * 71.43 (61.99–79.31) | * 73.87 (67.58–79.32) | * 72.93 (65.84–79.25) | 0.763 | |
CI threshold at 0.00488 | 80 (69.20–87.96) | 55 (44.74–64.78) | * 57.80 (51.83–63.55) | * 77.78 (68.59–84.87) | * 65.75 (58.34–72.63) | 0.763 | |
Wu et al., 2013 [62] | Mean VI at 37.056 threshold | 84.8 (74.97–91.90) | 40.9 (33.16–48.95) | 41.6 (37.80–45.53) | 84.4 (75.69–90.40) | 55.5 (48.90–61.88) | 0.711 |
Mean VI at 10.330 threshold | 45.6 (34.31–57.17) | 83.7 (76.97–89.03) | 58.06 (47.48–67.95) | 75.6 (71.42–79.29) | 71 (64.80–76.69) | 0.711 | |
Central VI at 32.285 threshold | 83.5 (73.51–90.94) | 41.5 (33.76–49.58) | 41.5 (37.60–45.53) | 83.5 (74.93–89.61) | 55.5 (48.90–61.88) | 0.71 | |
Central VI at 5.453 threshold | 40.5 (29.60–52.15) | 89.3 (83.43–93.65) | 65.3 (52.74–76.05) | 75.1 (71.42–78.51) | 73.1 (67.00–78.63) | 0.71 | |
Overall VI at 42.014 threshold | 78.5 (67.80–86.94) | 40.3 (32.56–48.31) | 39.2 (35.46–43.67) | 78.8 (70.35–85.66) | 52.9 (46.39–59.42) | 0.693 | |
Overall VI at 15.755 threshold | 40.5 (29.60–52.15) | 83 (76.26–88.50) | 53.3 (43.40–64.69) | 73.6 (69.80–77.34) | 68.9 (62.61–74.73) | 0.693 | |
Choi et al., 2015 [72] | 0.64 threshold | 83 (73.19–90.82) | 82.4 (58.09–94.55) | * 94.2 (87.00–97.53) | * 56.7 (43.30–69.13) | 82.8 (73.94–89.67) | 0.83 |
Sultan et al. [73] | Qualitative vascularity | 67.5 (50.45–80.43) | 88.1 (76.70–95.01) | * 80 (65.91–89.22) | * 78.5 (70.15–84.95) | * 79 (69.71–86.51) | ND |
Central vascular fraction area | 90 (77.38–97.34) | 88 (76.70–95.01) | 84 (72.91–91.63) | 92 (83.32–97.02) | 89 (81.17–94.38) | ||
Central flow volume index | 50 (34.19–65.81) | 62 (48.37–74.49) | 48 (37.91–59.88) | 63 (54.38–71.14) | 56 (46.71–66.86) | ||
Wu et al., 2016 [63] | Visual hypoechogenicity | 89.8 (83.98–94.06) | 31.9 (26.20–38.01) | 44.9 (42.46–47.37) | 83.5 (75.47–89.28) | 54 (49.06–58.91) | ND |
Comp. hypoechogenicity (EIN–T) | 79.6 (72.46–85.62) | 52.4 (46.03–58.64) | 50.8 (47.03–54.58) | 80.6 (74.91–85.26) | 62.8 (57.90–67.46) | 0.7 | |
Mark. hypoechogenicity (EIN–M) | 33.1 (25.82–41.07) | 93.3 (89.50–96.05) | 75.4 (64.75–83.59) | 69.3 (66.80–71.69) | 70.3 (65.64–74.69) | 0.77 | |
Baig et al. [66] | Visual grey scale evaluation | 96.3 (81.03–99.91) | 46.4 (35.47–57.65) | 36.6 (31.84–41.67) | 97.5 (84.90–99.63) | 58.6 (48.82–67.83) | ND |
Combined VI at 22% off-set | 70.4 (49.82–86.25 | 71.4 (60.53–80.76) | 44.2 (34.28–54.58 | 88.2 (80.50–93.16) | 71.2 (61.80–79.37) | ND | |
Combined VI + visual GSU | 66.7 (46.04–83.48) | 83.3 (73.62–90.58) | 56.3 (42.65–68.97) | 88.6 (81.90–93.04) | 79.3 (70.55–86.39) | ND | |
Gao et al. [70] | CAD | 96.7 (93.51–98.54) | 48.5 (38.58–58.60) | 81.3 (78.30–84.04) | 86.2 (75.45–92.71) | 82.2 (77.69–86.07) | 0.73 |
Radiologist–KWAK | 96.2 (92.97–98.26) | 75.7 (66.29–83.64) | 90.2 (86.73–92.83) | 89.7 (81.90–94.32) | 90.1 (86.39–93.02) | 0.87 | |
Radiologist–ATA | 95.4 (91.91–97.68 | 78.6 (69.47–86.10) | 91.2 (87.73–93.76) | 88 (80.39–92.97) | 90.4 (86.72–93.26) | 0.83 | |
Radiologist–ACR | 90.0 (85.43–93.46) | 76.7 (67.34–84.46) | 90.0 (86.29–92.73) | 76.7 (68.94–83.00) | 86 (81.83–89.47) | 0.86 | |
Choi et al., 2017 [67] | CAD–all nodules | 90.7 (77.9–97.4) | 74.6 (61.6–85.0) | 72.2 (58.4–83.5) | 91.7 (80.0–97.7) | 81.4 | 0.83 (0.74–0.89) |
Radiologist–all nodules | 88.4 (74.9–96.1) | 94.9 (85.9–98.9) | 92.7 (80.1–98.5) | 91.8 (81.9–97.3) | 92.2 | 0.92 (0.84–0.96) | |
CAD >1 cm nodules | 100 (76.8–100.00) | 71.8 (55.1–85.0) | 56 (34.9–75.6) | 100 (87.7–100) | 79.2 | 0.86 (0.74–0.94) | |
Radiologist >1 cm nodules | 92.9 (66.1–99.8) | 97.4 (86.5–99.9) | 92.9 (66.1–99.8) | 97.4 (86.5–99.9) | 96.2 | 0.95 (0.85–0.99) | |
Gitto et al. [71] | CAD | 21.4 (4.7–50.8) | 81.3 (67.4–91.1) | 25 (9.4–51.6) | 78 (72.3–82.8) | 67.7 | ND |
Radiologist-K-TIRADS | 78.6 (49.2–95.3) | 66.7 (51.6–79.6) | 40.7 (29.8–52.8) | 91.4 (79.3–96.7) | 69.4 | ND | |
Yoo et al. [65] | CAD | 80 (66.28–89.97) | 88.1 (77.82–94.70) | 83.3 (72.00–90.67) | 85.5 (77.09–91.18) | 84.6 (76.78–90.62) | 0.84 (0.76–0.90) |
Radiologist | 84 (70.89–92.83) | 95.5 (87.47–99.07) | 93.3 (82.15–97.71) | 88.9 (80.88–93.80) | 90.6 (83.80–95.21) | 0.90 (0.83–0.95) | |
Radiologist + CAD | 92 (80.77–97.78) | 85.1 (74.26–92.60) | 82.1 (72.08–89.12) | 93.4 (84.70–97.35) | 88 (80.74–93.30) | 0.89 (0.81–0.94) | |
Jeong et al. [68] | Expert Radiologist | 84.1 (69.93–93.36) | 96.4 (87.69–99.56) | 94.9 (82.50–98.64) | 88.5 (79.61–93.84) | 91 (83.60–95.80) | ND |
Expert Radiologist using CAD | 88.6 (75.44–96.21) | 83.9 (71.67–92.38) | 81.3 (70.24–88.84) | 90.4 (80.34–95.58) | 86 (77.63–92.13) | 0.863 | |
User 1 using CAD | 70.5 (54.80–83.24) | 80.4 (67.57–89.77) | 73.8 (61.61–83.19) | 77.6 (68.30–84.76) | 76 (66.43–83.98) | 0.754 | |
User 2 using CAD | 75 (59.66–86.81) | 73.2 (59.70–84.17) | 68.8 (58.01–77.80) | 78.8 (68.57–86.43) | 74 (64.27–82.26) | 0.741 | |
User 3 using CAD | 70.5 (54.80–83.24) | 73.2 (59.70–84.17) | 67.4 (56.28–76.84) | 75 (66.05–83.64) | 72 (62.13–80.52) | 0.718 | |
Reverter et al. [74] | Expert–ATA | 87 (79.75–91.90) * | 91.2 (85.4–94.82) * | 90.5 (82.74–92.70) * | 90.9 (84.39–92.78) * | * 89.00 (84.90–92.31) | 0.88 |
CAD–ATA | 87 (79.75–91.90) * | 68.8 (61.44–76.04) * | 64.5 (64.40–74.42) * | 86.3 (80.28–90.79) * | * 77.00 (71.82–81.64) | 0.72 | |
CAD–EU | 85.2 (78.05–90.71) * | 50.2 (42.43–58.17) * | 50.1 (54.22–62.41) * | 82.6 (72.93–86.47) * | * 66.00 (60.33–71.35) | 0.71 | |
CAD–AACE/AME/ACE | 81.5 (73.89–87.64) | 53.2 (45.42–61.13) * | 51.8 (54.36–63.15) * | 80.8 (70.62–83.75) * | * 66.00 (60.33–71.35) | 0.7 | |
Wang et al. [75] | CAD | 90.5 (86.08–93.88) | 89.9 (82.66–94.85) | 95.2 (91.90–97.22) | 81 (74.19–86.33) | 90.3 (86.73–93.20) | 0.902 (0.866–0.931) |
Radiologist | 93.8 (89.98–96.49) | 78 (69.03–85.35) | 90.4 (86.90–93.10) | 85 (77.46–90.33) | 88.9 (85.12–91.98) | 0.859 (0.818–0.894) |
Author (S) | Patient Selection | Index Test | Reference Standard | Flow and Timing |
---|---|---|---|---|
Lyshchik et al., 2007 [69] | High | Low | Low | Low |
Chen et al., 2011 [64] | Low | Low | Low | Low |
Wu et al., 2013 [62] | Low | Low | Low | Low |
Choi et al., 2015 [72] | High | High | Low | Low |
Sultan et al., 2015 [73] | High | Low | Low | Unclear |
Wu et al., 2016 [63] | Unclear | Low | Low | Low |
Baig et al., 2017 [66] | Low | Low | Low | Low |
Gao et al., 2018 [70] | High | Low | High | Low |
Choi et al., 2017 [67] | Low | Low | Low | Low |
Gitto et al., 2019 [71] | Low | Low | Low | Low |
Yoo et al., 2018 [65] | Low | Low | Low | Low |
Jeong et al., 2019 [68] | Low | Low | Low | Low |
Reverter et al., 2019 [74] | High | Low | Low | Low |
Wang et al., 2019 [75] | High | Unclear | Low | Low |
Author (S) | Patient Selection | Index Test | Reference Standard |
---|---|---|---|
Lyshchik et al., 2007 [69] | Low | Low | Low |
Chen et al., 2011 [64] | Low | Low | Low |
Wu et al., 2013 [62] | Low | Low | Low |
Choi et al., 2015 [72] | High | Unclear | Low |
Sultan et al., 2015 [73] | Low | Low | Low |
Wu et al., 2016 [63] | Low | Low | Low |
Baig et al., 2017 [66] | Low | Low | Low |
Gao et al., 2018 [70] | Low | Low | High |
Choi et al., 2017 [67] | Low | Low | Low |
Gitto et al., 2019 [71] | Low | Low | Low |
Yoo et al., 2018 [65] | Low | Low | Low |
Jeong et al., 2019 [68] | Low | Low | Low |
Reverter et al., 2019 [74] | Low | Low | Low |
Wang et al., 2019 [75] | Low | Low | Low |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chambara, N.; Ying, M. The Diagnostic Efficiency of Ultrasound Computer–Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis. Cancers 2019, 11, 1759. https://doi.org/10.3390/cancers11111759
Chambara N, Ying M. The Diagnostic Efficiency of Ultrasound Computer–Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis. Cancers. 2019; 11(11):1759. https://doi.org/10.3390/cancers11111759
Chicago/Turabian StyleChambara, Nonhlanhla, and Michael Ying. 2019. "The Diagnostic Efficiency of Ultrasound Computer–Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis" Cancers 11, no. 11: 1759. https://doi.org/10.3390/cancers11111759