Diagnostic Performance of KLCA-NCC 2018 Criteria for Hepatocellular Carcinoma Using Magnetic Resonance Imaging: A Systematic Review and Meta-Analysis

Several imaging-based systems have been proposed for the diagnosis of hepatocellular carcinoma (HCC) using magnetic resonance imaging (MRI), reflecting geographical differences in the clinical environment for HCC. We conducted a systematic review and meta-analysis to determine the performance of the Korean Liver Cancer Association-National Cancer Center (KLCA-NCC) 2018 criteria for the MRI diagnosis of HCC. Original studies reporting the performance of the KLCA-NCC 2018 criteria for the diagnosis of HCC using MRI were identified in MEDLINE and EMBASE until 29 March 2021. The meta-analytic pooled sensitivity and specificity of the KLCA-NCC 2018 criteria for diagnosing HCC were calculated using a bivariate random-effects model. A meta-regression analysis was performed to explore study heterogeneity further. Eight studies involving 1690 HCCs reported the accuracy of the KLCA-NCC 2018 imaging criteria. The pooled sensitivity and specificity of the definite HCC criteria for diagnosing HCC were 81% (95% confidence interval, 76–85%; I2 = 86%) and 90% (86–93%; I2 = 23%), respectively. For five available studies, the pooled sensitivity and specificity of the definite HCC criteria for diagnosing HCCs smaller than 20 mm were 80% (72–86%; I2 = 76%) and 91% (86–94%; I2 = 0%), respectively. A considerable threshold effect with a correlation coefficient of 0.667 was observed. The results of the meta-regression analysis revealed that the accuracy of the definite HCC criteria differed significantly depending on the type of MRI contrast agent (p = 0.01). In conclusion, the KLCA-NCC 2018 criteria had good overall diagnostic performance in diagnosing HCC. Substantial study heterogeneity was observed for sensitivity, which was significantly influenced by the type of contrast agent and by a threshold effect.


Introduction
Hepatocellular carcinoma (HCC) is the most common primary liver cancer and the third most frequent cause of cancer-related deaths [1,2]. HCC is the only malignancy that can be noninvasively diagnosed in at-risk patients by imaging, including multiphase computed tomography, contrast-enhanced ultrasonography, and magnetic resonance imaging (MRI), without pathological confirmation [3,4]. Therefore, accurate and consistent imagingbased diagnosis is of the utmost importance for the management of HCC. Worldwide, all imaging-based systems use a combination of arterial-phase hyperenhancement plus washout for the diagnosis of HCC [3][4][5]. However, geographical differences in the clinical environment and treatment practice make it challenging to establish universal guidelines for the diagnosis of HCC [3][4][5]. Unlike Western countries where liver transplantation is considered the only curative treatment, resection and loco-regional therapy are common curative options in Asia.  [5][6][7][8]. Unlike prior versions, the Korean Liver Cancer Association (KLCA, formerly KLCSG)-NCC (KLCA-NCC) 2018 practice guidelines have been revised into a nonbinary system for the imaging diagnosis of HCC [5]. The KLCA-NCC 2018 guidelines categorize each hepatic lesion according to its likelihood of HCC, i.e., indeterminate nodules, probable HCC, and definite HCC [5]. A diagnosis of the definite HCC can be made when hepatic lesions ≥1 cm show arterial-phase hyperenhancement with delayed washout after the exclusion of benign lesions showing marked T2 hyperintensity or non-HCC malignancies showing a targetoid appearance [5]. In particular, washout is possible not only during the portal phase but also during the transitional phase and hepatobiliary phase (HBP) on MRI with a hepatobiliary contrast agent [5]. The KLCA-NCC 2018 criteria were intended to improve sensitivity in the detection of early-stage HCC, which is eligible for curative treatments such as surgical resection and local ablation.
Several studies, most of which were single-center studies, reported conflicting results regarding the performance of KLCA-NCC 2018 for the diagnosis of HCC [9][10][11][12][13]. Given the increased attention to and use of the KLCA-NCC 2018 criteria, it is time to assess clearly the performance of KLCA-NCC 2018 in the diagnosis of HCC and to understand the causes of heterogeneity between studies. Therefore, we conducted a systematic review and meta-analysis to assess the performance of the KLCA-NCC 2018 criteria for diagnosing HCC with MRI and to determine the factors associated with study heterogeneity.

Materials and Methods
This meta-analysis was performed in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [14] and was registered in PROS-PERO (ID: CRD42021269626). The following literature search, study selection, data extraction, and study quality assessments were independently conducted by two reviewers (each having ≥10 years of experience in liver MRI), and any disagreements were resolved via consensus.

Literature Search Strategy
A literature search of the MEDLINE and EMBASE databases was conducted to identify studies investigating the diagnostic performance of the KLCA-NCC 2018 criteria for the dichotomous diagnosis of HCC based on dynamic contrast-enhanced MRI. The search queries included "Liver", "Hepatocellular carcinoma", "Korean Liver Cancer Association-National Cancer Center", and "KLCA-NCC". Supplementary Table S1 lists the search terms in detail. The beginning date for the search was set to 1 January 2018 to focus on original articles using the 2018 version of the KLCA-NCC. The literature search was updated until 29 March 2021. A manual evaluation of the searched articles was performed to narrow down the number of relevant articles. The search was limited to original studies on human subjects written in English. In order to expand the search, the bibliographies of articles were screened for other potentially eligible articles.

Inclusion and Exclusion Criteria
After duplicate articles were removed, the articles were reviewed regarding eligibility: (i) Population, patients at risk of HCC with focal hepatic lesions (≥1 cm); (ii) Index test, dynamic contrast-enhanced liver MRI; (iii) Reference standard, pathological diagnosis or imaging follow-up; (iv) Outcomes, sensitivity and specificity of definite HCC diagnosis according to the KLCA-NCC 2018 criteria; and (v) Study design, not limited. The exclusion criteria were as follows: (i) inclusion of a small number of patients (<10); (ii) animal studies, case reports, review articles, editorials, or scientific abstracts/conference proceedings; (iii) studies that were not within the field of interest of this study; and (iv) studies without sufficient details to construct a diagnostic 2-by-2 table of the imaging results and reference Diagnostics 2021, 11, 1763 3 of 12 standard findings. Articles were first screened by titles and abstracts, and fully reviewed after the first screening.

Data Extraction and Quality Assessment
The following data were extracted: (i) study characteristics including authors, year of publication, and study design (prospective vs. retrospective); (ii) subject characteristics including sample size, age, sex, underlying liver disease, total number of hepatic lesions, and number of HCCs; (iii) MRI characteristics including indications for MRI, MRI scanner field strength, and type of contrast agent; (iv) image interpretation method (multiple reviewers with consensus vs. multiple independent reviewers); (v) reference standards for HCC and non-HCC; (vi) interobserver agreement (κ) for the categorization of hepatic lesions according to the KLCA-NCC 2018 criteria; and (vii) the accuracy of KLCA-NCC 2018 imaging criteria for the dichotomous diagnosis of HCC. The exact numbers of true positives, true negatives, false positives, and false negatives among hepatic lesions were extracted to determine the diagnostic accuracy of the criteria. If not distinctly mentioned, data were manually retrieved from tables and figures. If an article did not contain sufficient data, we contacted the corresponding authors by email to request additional information or clarification.
The Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) criteria [15] were used to assess the quality of the selected studies. The QUADAS-2 tool assesses study quality according to four different domains (patient selection, index test, reference standard, and flow and timing).

Data Synthesis
The meta-analytic pooled sensitivity and specificity were calculated using a bivariate random-effects and hierarchical summary receiver operating characteristic (HSROC) model. Heterogeneity between studies for sensitivity and specificity was assessed using the Higgins I 2 statistic (I 2 > 50%: substantial heterogeneity). The presence of a threshold effect was analyzed by visual assessment of the coupled forest plots of sensitivity and specificity as well as by calculating the Spearman correlation coefficient between the sensitivity and the false-positive rate (i.e., 1-specificity) [16]. A correlation coefficient > 0.6 was considered to represent a considerable threshold effect [16]. Deeks' funnel plot and Deeks' asymmetry test were used to assess the presence of publication bias.
A subgroup analysis was performed on studies reporting diagnostic performance for HCCs less than 20 mm in size and those reporting data on the "probable HCC" category in the KLCA-NCC 2018 imaging criteria. Additional subgroup analyses were performed for intra-individual comparative studies between the latest updated international guidelines. The pooled sensitivity and specificity of the KLCA-NCC 2018 criteria were compared to those of the Liver Imaging Reporting and Data System (LI-RADS) v2018, European Association for the Study of the Liver (EASL), and Asian Pacific Association for the Study of the Liver (APASL) guidelines [4,17,18] using joint-model bivariate meta-regression.
In order to investigate the causes of study heterogeneity further, a meta-regression analysis was performed. The following covariates were considered: (i) study design (prospective vs. retrospective); (ii) number of patients (<200 vs. ≥200); (iii) MRI scanner field strength (3.0 T only vs. 1.5 T or 3.0 T); (iv) MRI contrast agent (hepatobiliary contrast agent (HBA) only vs. extracellular contrast agent (ECA) or HBA); (v) image interpretation method (multiple independent reviewers vs. multiple reviewers with consensus); (vi) reference standard for HCC (pathology only vs. pathology or imaging follow-up); and (vii) reference standard for non-HCC (pathology only vs. pathology or imaging follow-up).
Using the κ and 95% confidence interval (CI) reported in the individual studies, the pooled κ with a 95% CI for the categorization of hepatic lesions according to the KLCA-NCC 2018 criteria was calculated using the DerSimonian-Laird random-effects model. κ was categorized according to the standards of Landis and Koch [19]. Stata version 16.0 (StataCorp LP, College Station, TX, USA) and R version 3.3.2 (The R Foundation for Statistical Computing, Vienna, Austria) were used for the analysis, with p < 0.05 considered statistically significant.

Literature Search
A total of 99 articles were screened after removing duplicates. Of these, 75 articles were excluded based on their titles and abstracts. Seventeen additional articles were excluded during a full-text review ( Figure 1). A search of the bibliographies of the remaining articles yielded one additional eligible article. Ultimately, a total of eight eligible articles reporting the diagnostic performance of the KLCA-NCC 2018 criteria for the diagnosis of HCC were selected [9][10][11][12][13][20][21][22].
A total of 99 articles were screened after removing duplicates. Of these, 75 articles were excluded based on their titles and abstracts. Seventeen additional articles were excluded during a full-text review ( Figure 1). A search of the bibliographies of the remaining articles yielded one additional eligible article. Ultimately, a total of eight eligible articles reporting the diagnostic performance of the KLCA-NCC 2018 criteria for the diagnosis of HCC were selected [9][10][11][12][13][20][21][22].

Quality Assessment
The quality of the included articles is summarized in Supplementary Figure S1. In the flow and timing domain, one study had a high risk of bias due to an inappropriate interval between the index test and the reference standard (i.e., within 180 days) and did not consistently use the same reference standard [11]. In addition, three studies showed an unclear risk of bias because the interval between the index test and reference standard was not specified [12,13,22]. In the patient selection domain, one study showed a high risk of bias because it enrolled only patients who underwent liver transplantation [10]. In the reference standard domain, two studies left it unclear whether the reference standard results were interpreted without knowledge of the index test results [9,22].

Meta-Regression Analysis
The results of the meta-regression analysis for the diagnostic performance of KLCA-NCC 2018 are summarized in Table 3. Among the eight included covariates, the type of MRI contrast agent (p = 0.01) was the only factor that significantly influenced study heterogeneity. Studies using only HBA showed significantly higher sensitivity than those using either ECA or HBA (84% vs. 72%) but showed significantly lower specificity (87% vs. 95%). The other covariates were not significantly associated with study heterogeneity.

Discussion
Our meta-analysis showed that the pooled sensitivity and specificity of the KLCA-NCC 2018 imaging criteria for definite HCC were 81% (95% CI, 76-85%) and 90% (95% CI, 86-93%), respectively. For HCCs smaller than 20 mm, the performance of these standards was comparable to its performance for all HCCs. For the definite/probable HCC categories, the meta-analytic summary sensitivity and specificity were 87% (95% CI, 81-92%) and 87% (95% CI, 81-93%), respectively. Substantial study heterogeneity was noted in sensitivity, with the type of MRI contrast agent and the threshold effect being significantly associated with study heterogeneity.
This study demonstrated that the definite HCC criteria of KLCA-NCC 2018 had good overall diagnostic performance for the MRI diagnosis of HCC, with substantial to almost perfect interobserver agreement. In addition, the subgroup analyses revealed that the pooled sensitivity of the definite HCC criteria of KLCA-NCC 2018 was significantly higher than that of the LR-5 or EASL guideline, consistent with recent comparative studies [9][10][11][12][13]20,22]. In line with previous studies [23,24], the high sensitivity of KLCA-NCC 2018 is largely attributable to the extended washout time, which extends to the transitional phase and the HBP. Since HBP signal intensity alterations precede typical vascular profile changes during hepatocellular carcinogenesis [25], more cases of early HCC can be diagnosed as definite HCC when an extended washout is used. The result that the APASL guideline had the highest pooled sensitivity may also be attributable to the application of HBP hypointensity as an alternative to the washout appearance. These higher sensitivities are particularly valuable in Asian countries, where HCC is highly prevalent, and surgical resection or image-guided ablation is preferred for HCC treatment [26,27]. However, the pooled specificity of the definite HCC criteria of KLCA-NCC 2018 was significantly lower than that of the LR-5 or EASL guideline. In the KLCA-NCC 2018 criteria, the specificity inevitably decreased as the sensitivity increased, which is a rather expected result stemming from the fundamental nature of diagnostic test accuracy. However, the pooled specificity of the KLCA-NCC 2018 guideline was significantly higher than that of the APASL guideline as a result of applying size criteria (i.e., ≥1 cm) and exclusion criteria for HCC, such as targetoid appearance, to exclude common causes of a false-positive diagnosis, including non-HCC malignancy. Consequently, unnecessary treatment or biopsy can be avoided through the application of the KLCA-NCC 2018 rather than the APASL guideline. Overall, we believe that the KLCA-NCC 2018 achieved an appropriate balance between sensitivity-oriented guideline such as the APASL guideline and specificity-oriented guidelines such as the LI-RADS or EASL guidelines and is more optimized for Asian countries than other international guidelines.
In our meta-regression analyses, the type of contrast agent was the only significant factor affecting study heterogeneity. The results revealed that the use of the HBA for MRI showed significantly better sensitivity for definite HCC than the mixed use of ECAs and HBAs, but it also showed lower specificity. The superior sensitivity of HBA-MRI may be attributable to the extended definition of washout, but this high sensitivity comes at the cost of specificity [28]. Although most Asian countries, to improve sensitivity for early detection of HCC, tend to use HBA-MRI more than ECA-MRI for the evaluation of HCC in clinical practice, consideration should be given for an improvement of the low sensitivity of ECA-MRI. As with LI-RADS, it can be optional to adopt additional major imaging features of definite HCC, such as an enhancing capsule and threshold growth. In addition, we noted a significant positive correlation between the sensitivity and the false-positive rate (correlation coefficient = 0.667). This result probably indicates that the study heterogeneity was partly due to the threshold effect that occurs when different thresholds or cutoff values are used to determine a positive test result [16]. For example, the included studies applied different washout criteria depending on the type of contrast agent used, and there might be some uncertainty about the definition of ancillary imaging features such as targetoid appearance.
This study has several limitations. First, the number of included studies was small (n = 8), as the KLCA-NCC 2018 criteria have only recently been introduced. Second, substantial study heterogeneity was noted regarding sensitivity, which could preclude the creation of robust meta-analytic estimates for diagnostic accuracy. To minimize this limita-tion, we investigated its causes and found that the type of contrast agent and the threshold effect significantly influenced study heterogeneity. Third, all included studies originated from South Korea, an area where hepatitis B is endemic, which limits the generalization of our results to other geographic populations with different etiologies of chronic liver disease. However, this geographic homogeneity brought somewhat homogeneous patient cohorts, which may enhance the quality of the evidence provided by this meta-analysis. Given that the KLCA-NCC criteria are focused on the Asian population in an area where hepatitis B is endemic, our meta-analysis represents the currently available evidence regarding its performance for the diagnosis of HCC.

Conclusions
In conclusion, the KLCA-NCC 2018 criteria had good overall diagnostic performance with balanced sensitivity and specificity in diagnosing HCC using MRI, even for HCCs smaller than 20 mm. In comparison with the definite HCC category alone, the definite/probable HCC category showed an increase in pooled sensitivity and a decrease in specificity. Substantial study heterogeneity was noted for sensitivity and was significantly associated with the threshold effect and the type of MRI contrast agent. In addition, studies that used HBAs showed significantly higher sensitivity but lower specificity than those that used ECAs to diagnose HCCs.