Performance of Radiologists in Characterizing and Diagnosing Hepatic Lesions Using Dynamic Contrast-Enhanced CT With and Without Artificial Intelligence

Abstract

Background: To investigate the performance of radiologists in characterizing and diagnosing hepatic lesions with and without the assistance of deep learning-based artificial intelligence (AI). Methods: This retrospective study included 83 nodules/masses from 69 patients who underwent dynamic contrast-enhanced CT of the liver. Image assessments were conducted by 20 radiologists. grouped according to their level of experience (10 senior and 10 junior). Each radiologist determined the probability of eight characteristics based on enhancement patterns and the diagnosis with and without AI attached to the SYNAPSE SAI viewer (FUJIFILM Corporation, Minato-ku, Japan). The reference standard for comparison was established as follows: final diagnoses were based on pathology for 39 lesions and expert imaging consensus for the remainder, while image characteristics for all lesions were determined by expert imaging consensus. Areas under the receiver operating characteristic curves (AUCs) were analyzed using the multireader multicase method. Results: Using AI significantly improved the overall AUCs for both the characterization and the diagnosis of liver lesions. Improvement was suggested for specific items, including the characterization of enhancement, nonperipheral washout, and delayed enhancement, and the diagnosis of hepatocellular carcinoma. The utilization of AI system also suggested potential improvements in the AUCs for image characterization in both the senior and junior groups. Conclusions: Using AI improved the radiologists’ performance in characterizing and diagnosing hepatic lesions. In terms of their capacity to assess imaging characteristics, improvements were observed regardless of their level of experience.

1. Introduction

Liver cancer occurs worldwide and has a poor prognosis. According to GLOBOCAN 2022 data, liver cancer was the sixth most diagnosed type of cancer and the third leading cause of cancer death in 2022 [1]. Hepatocellular carcinoma (HCC) accounts for up to 90% of primary liver cancers and has a five-year survival rate of <20%. The prognosis is even poorer when HCC is diagnosed at an advanced stage [2,3,4,5]. Therefore, detailed characterization and early diagnosis of liver lesions are important for appropriate patient management and optimal treatment selection [6,7,8,9].

Radiologists contribute to the early diagnosis of HCC and malignant tumors by evaluating images of hepatic lesions and providing accurate information, which is combined with pathological evidence. Dynamic contrast-enhanced CT is one of the most widely used methods for imaging hepatic lesions because of its noninvasive nature [10,11,12,13], and radiologists evaluate enhancement patterns of lesions using standardized imaging guidelines, such as the LI-RADS 2018 [14,15,16,17,18].

Recent research and developments in the use of artificial intelligence (AI) in radiology indicate that radiologists who use AI will be able to more precisely evaluate images of liver lesions than those who do not, and this is expected to improve the clinical care and prognosis of patients [19,20,21,22,23]. Recently, deep learning was applied to develop a system that automatically evaluates the imaging characteristics of hepatic lesions [24,25]. The AI system reportedly simultaneously analyzes multi-phase image data and recognizes imaging features important for diagnosis with high accuracy. Using such AI systems is expected to improve the daily workflow of radiologists and reduce the occurrence of medical errors. There is still insufficient research on how AI systems affect a radiologist’s performance when interpreting images of liver lesions. Thus, there is an urgent need to determine the effects that utilizing AI has on the performance of radiologists so that its clinical benefits and applicability can be verified.

In this study, we investigated how the use of an AI system affected the performance of radiologists in characterizing and diagnosing hepatic lesions. The primary aim was to compare the performance of radiologists with and without AI assistance. The secondary objective was to examine differences in the effectiveness of using an AI system based on the experience level of radiologists.

2. Materials and Methods

2.1. Study Population and CT Examinations

The flowchart for the patient enrollment process is shown in Figure 1. There were 385 patients who underwent dynamic contrast-enhanced CT of the liver at the University of Osaka Hospital between July 2020 and May 2021. CT image data from the arterial, portal, and equilibrium phases were acquired using either a 64-channel (Discovery CT 750 HD; GE Healthcare, Chicago, IL, USA), a 256-channel (Revolution CT; GE Healthcare), or a 320-channel (Aquilion ONE, Aquilion Precision; Canon Medical Systems, Otawara, Japan) CT scanner. Images were reconstructed using the following iterative reconstruction algorithms: Adaptive Statistical Iterative Reconstruction (ASiR) for Discovery CT 750 HD, Adaptive Statistical Iterative Reconstruction-V (ASiR-V) for Revolution CT, and Adaptive Iterative Dose Reduction Using 3-dimensional Processing (AIDR 3D) for Aquilion Precision and Aquilion ONE GENESIS edition. From the set of identified lesions (up to two of the largest lesions per patient), a subset of 83 nodules/masses in 69 patients that met the following inclusion criteria was selected: (a) lesion(s) identified during the earliest examination in cases of patients who underwent multiple examinations; (b) no therapeutic intervention for the lesion(s) before the examination; (c) complete CT data from all phases available; (d) lesion(s) not considered indicative of liver cysts; (e) lesion size ≥ 10 mm; (f) no significant imaging artifacts.

2.2. AI System and Feature Definition for Hepatic Lesion Evaluation

We employed a deep learning-based system for evaluating hepatic lesions attached to a SYNAPSE SAI viewer (FUJIFILM Corporation, Minato-ku, Japan), corresponding to the “Averaging model” in the original paper [24]. This system was trained on a dataset consisting of 3318 tumors with labeled characteristics. The core architecture utilizes Convolutional Neural Networks (CNNs) and is specifically designed to handle a variable number of input images from two- or three-phase Dynamic contrast-enhanced CT. We ensured data independence by confirming that the AI system’s training data were derived from a historical cohort preceding the enrollment period of our study population, with no overlap in patient identity between the two datasets. When any liver lesion was selected in the dynamic contrast-enhanced CT images, the system was activated to analyze three-dimensional data of the lesion across all phases, which produced characteristic analyses (Figure 2). The system was set to output eight characteristics; enhancement (hypervascular or hypovascular), margin (circumscribed or noncircumscribed), enhancing capsule appearance (+ or −), nonrim arterial phase hyperenhancement (APHE) (+ or −), nonperipheral washout (+ or −), expansion and coalescence of enhancing areas (+ or −), delayed enhancement (+ or −), and peripheral enhancement (+ or −), where “+” indicates present and “−” indicates absent. The operational definitions of these features were established based on the LI-RADS 2018 criteria [14]. A detailed, point-by-point summary of these operational definitions is provided in Supplementary Table S1.

2.3. Generation of the Reference Standard

The dynamic contrast-enhanced CT images of the 83 lesions were transferred to a picture archiving and communication system. The slices used for the image assessments were limited to a level that encompassed the entire liver; slices both cranial and caudal to the liver were excluded. Bounding boxes were pre-annotated for the target lesions. The order of the lesions to be evaluated was randomized. The observers were blinded to the patients’ clinical backgrounds. Adjustments to the window level and width were permitted using the SAI viewer. Color monitors with a 3-megapixel resolution and a screen size of 21.2 inches were used for image assessments.

The characteristics and diagnostic reference standard for each lesion were determined by two board-certified abdominal radiologists: one with 20 years of diagnostic experience and one with 15 years of diagnostic experience. These abdominal imaging experts independently conducted their evaluations without AI assistance. Disagreements were resolved by a consensus agreement. The lesions were categorized into four diagnostic groups, following the classification system of Yasaka et al.: Category A contained classic HCCs, Category B contained malignant liver tumors other than classic and early HCCs, Category C contained indeterminate lesions (including early HCCs and dysplastic nodules), and Category D contained benign lesions (including hemangiomas and perfusion alteration) [26]. In this study, 39 lesions were diagnosed pathologically using surgical or biopsy specimens. The diagnostic reference standard for these lesions was determined based on pathological evidence, with lesions lacking histopathological confirmation categorized according to expert imaging readings. Specifically, the histopathology reports provided detailed information essential for categorization, including the specific type of lesion (e.g., cholangiocellular carcinoma, liver metastasis) and, crucially for HCCs, the differentiation grade (e.g., moderately and poorly differentiated). Category A included pathologically confirmed moderately and poorly differentiated HCC. Category B included pathologically confirmed cholangiocellular carcinoma, liver metastasis, and neuroendocrine carcinoma. All other pathologically confirmed entities were defined as the reference standard for Category C. For Category D, the reference standard was defined by a conclusive imaging evaluation by an expert, primarily including benign lesions such as hemangiomas.

2.4. Image Assessments Conducted with and Without the Use of AI

To investigate the impact of using the AI system on image evaluation, radiological evaluations were conducted by 20 radiologists. They were placed into two groups (n = 10 per group) according to their level of experience. Those in the senior group had >5 years of experience, and those in the junior group had ≤5 years of experience. To standardize the reading criteria, the participants were presented with image examples taken from the LI-RADS CT/MRI manual [14] and received training on the definition of each characteristic feature before the reading experiment.

The readers evaluated the eight characteristics on a continuous scale from 0 to 1 based on their confidence level, following the approach described by Wataya et al. [27]. Scores of >0.5 were interpreted as positive, scores of <0.5 were interpreted as negative, and scores of 0.5 were interpreted as indeterminate. After assessing the characteristics of a lesion, the readers placed it within one of the four aforementioned diagnostic categories.

The image review process comprised two sessions: one without AI, followed by one with AI. The design is based on Wataya et al. [27], who evaluated lung-nodule AI, with the assumption that the AI system is used as a second reader computer-assisted diagnosis. All participating radiologists received dedicated training on how to properly interpret the AI results and the output interface prior to the reading sessions. In the first session, the readers evaluated the images without the AI system. In the second session, they reviewed the images again, this time with the AI assistance. The AI output was presented to the readers as a binary assessment (e.g., present/absent) for each image characteristic. To minimize intra-observer (inter-session) variability, the readers could refer to their initial assessment in the second session. To minimize the potential for carry-over effects and recall bias between sessions, the two reading sessions were separated by a washout period of approximately three months.

2.5. Statistical Analysis

The performance of the readers in characterizing and diagnosing the lesions was assessed using receiver operating characteristic (ROC) curve analysis. The primary outcome was to compare the readers’ average area under the ROC curve (AUC) in image assessments with and without the AI assistance. The primary null hypothesis is that “The average Area Under the ROC Curve (AUC) for characterizing and diagnosing hepatic lesions among all radiologists does not significantly differ between assessments without AI assistance and with AI assistance.” Two primary endpoints were defined: one summarizing all characteristics and one summarizing all diagnoses. The Obuchowski–Rockette method was employed to analyze the multireader multicase data. The ROC analysis and AUC calculations were performed using the continuous probability scores (ranging from 0 to 1). The ROC curves were plotted and the AUCs, sensitivities, specificities and 95% confidence intervals (CIs) calculated using the MRMCaov library 0.3.0 (https://github.com/brian-j-smith/MRMCaov (accessed on 1 June 2025)) and R 4.1.2 (https://www.r-project.org (accessed on 1 June 2025)). Statistical significance was indicated by p values < 0.05 [28]. Multiplicity correction was applied to these two primary endpoints using the Bonferroni method, setting the significance level at α = 0.05/2 = 0.025. All individual item analyses other than these primary endpoints are treated as exploratory analyses. In addition, the calculation of specific binary classification metrics, including accuracy, sensitivity, specificity, and F1 score, was performed for each imaging feature. The classification threshold for determining these characteristics was uniformly set at 0.5. This threshold was solely applied for the calculation of these binary classification metrics and for descriptive purposes.

Inter-observer agreement for each characteristic was analyzed using the kappa statistic and intraclass correlation coefficient (ICC). The kappa statistic was employed to assess the agreement among reference standard annotators because of the discrete nature of their output values (0 or 1). The ICC was used to calculate the agreement among the readers in the senior group and the junior group because their outputs ranged from 0 to 1. The inter-rater agreements and their 95% CIs and p values were computed by the bootstrap method using Python 3.8.5 (https://www.python.org (accessed on 1 April 2025)) and Pingouin 0.5.2 (https://pingouin-stats.org (accessed on 1 April 2025)) [29,30]. In this study, values of 0.00–0.20 indicated slight agreement, values of 0.21–0.40 indicated fair agreement, values of 0.41–0.60 indicated moderate agreement, values of 0.61–0.80 indicated substantial agreement, and values of 0.81–1.00 indicated almost perfect agreement, following the guidelines of Landis and Koch [31].

To retrospectively confirm the statistical robustness of our design, a post hoc power analysis was performed. The anticipated effect size was set ⊿AUC = 0.05 based on clinical relevance and typical effect sizes observed in a similar study [27]. Calculations were based on the Obuchowski–Rockette method to estimate the standard errors of the difference in AUC. The Hillis power formula was subsequently used to calculate the statistical power (α = 0.05, two-sided).

3. Results

3.1. Clinical Characteristics, Reference Standard Distribution, and AI System Performance

As shown in

Table 1. Clinical characteristics of the patients and hepatic nodules/masses included in the study. Data in parentheses are percentages. Mean data include ± standard deviation.

Patient Characteristics
Sex
Male	47 (68.1)
Female	22 (31.9)
Mean age (years)	63 ± 16
Cirrhosis	8 (11.6)
Hepatitis B Virus	7 (10.1)
Hepatitis C Virus	3 (4.3)
Hepatic steatosis	13 (18.8)
Nodule/mass characteristics
Size (mm)	39.9 ± 28.1
Pathological diagnosis
Hepatocellular carcinoma	19
Well differentiated type	4
Moderately differentiated type	13
Poorly differentiated type	2
Cholangiocellular carcinoma	8
Liver metastasis	3
Neuroendocrine carcinoma	3
Focal nodular hyperplasia	1
Hepatocellular adenoma	1
Epithelioid hemangioendothelioma	1
No malignancy	3

, among the 69 patients included in this study, 47 were male, and the mean age was 63 years ± 16 (SD). Underlying liver conditions in the cohort included hepatic steatosis (n = 13, 18.8%), cirrhosis (n = 8, 11.6%), Hepatitis B Virus infection (n = 7, 10.1%), and Hepatitis C Virus infection (n = 3, 4.3%). The characteristics of the 83 hepatic nodules/masses examined in this study are presented in Table 1. The average lesion size was 39.9 mm ± 28.1. The pathological diagnoses of the lesions were predominantly classified as HCC and cholangiocellular carcinoma (CCC). There were also cases of liver metastases, neuroendocrine carcinoma, focal nodular hyperplasia, hepatocellular adenoma, and epithelioid hemangioendothelioma. Three lesions presented no histopathological evidence of malignancy.

Table 2. Reference standard for the characteristics and diagnoses of the 83 hepatic nodules/masses.

Imaging Characteristics	No. of Positive *	No. of Negative †
Enhancement	75	8
Margin	57	26
Enhancing capsule appearance	18	65
Nonrim arterial phase hyperenhancement	38	45
Nonperipheral washout	23	60
Expansion and coalescence of enhancing areas	24	59
Delayed enhancement	29	54
Peripheral enhancement	39	44
Diagnosis		With pathological evidence
Category A (classic HCCs)	25	15
Category B (malignant tumors)	26	14
Category C (indeterminate lesions)	15	10
Category D (benign lesions)	17	0

* Hypervascular for enhancement and circumscribed for margin. † Hypovascular for enhancement and noncircumscribed for margin.

shows the reference standard for the characteristics and diagnoses of the lesions. Features strongly associated with HCC, such as enhancing capsule appearance, nonrim APHE, and nonperipheral washout, were observed in 18, 38, and 23 lesions, respectively. Expansion and coalescence of enhancing areas, delayed enhancement, and peripheral enhancement, which are related to hemangioma, liver metastases, and CCC, were identified in 24, 29, and 39 lesions, respectively. In terms of the distribution of the diagnoses, 25 lesions were classified as Category A, 26 as Category B, 15 as Category C, and 17 as Category D.

3.2. Performance of Radiologists in Characterizing and Diagnosing Hepatic Lesions with and Without AI

Figure 3 shows the ROC curves and AUCs that were obtained when the readers evaluated the characteristics of the 83 nodules/masses and attributed diagnoses. The analysis of the two primary endpoints showed that, when the readers used the AI system, statistically significant improvements were observed in the overall AUCs for both the characterization (0.86 [95% CI 0.84, 0.89] without AI vs. 0.88 [0.85, 0.91] with AI; p = 0.006) and the diagnosis (0.80 [0.76, 0.84] without AI vs. 0.81 [0.77, 0.85] with AI; p = 0.021) of liver lesions. Numerical improvements were suggested in the evaluation of specific features; enhancement (0.90 [0.86, 0.93] without AI vs. 0.92 [0.88, 0.96] with AI; p = 0.018), nonperipheral washout (0.89 [0.83, 0.94] without AI vs. 0.91 [0.86, 0.96] with AI; p = 0.031), and delayed enhancement (0.83 [0.77, 0.89] without AI vs. 0.87 [0.81, 0.93] with AI; p = 0.005). In addition, the use of the AI system was associated with a trend toward improved diagnostic performance of the readers in terms of their capacity to classify Category A (classic HCCs) lesions (AUC 0.81 [0.75, 0.88] without AI vs. 0.83 [0.77, 0.89] with AI; p = 0.043). This improvement potentially translated to enhanced sensitivity (when specificity was fixed at 0.75: 0.76 [0.67, 0.85] without AI vs. 0.80 [0.71, 0.89] with AI). In addition to the ROC analysis, performance for evaluation of each imaging feature was assessed using binary classification metrics. Consistent with the visual differences observed in the ROC curves for delayed enhancement (Figure 3), the use of AI assistance resulted in a notable increase in both accuracy (0.80 without AI vs. 0.85 with AI) and specificity (0.70 without AI vs. 0.79 with AI) for evaluating this feature. Detailed results for all imaging characteristics are presented in Supplementary Table S2.

Figure 4 shows an example of a case in which the diagnostic performance of two readers was improved using the AI system. The reference standard for the diagnosis of this lesion was defined as Category A based on histopathological evidence from surgical specimens. The AI system correctly described all the lesion’s imaging features the same as the reference standard. In the first reading session (without the AI), reader A (who had 5 years of experience) determined that there was an absence of nonperipheral washout (probability = 0.11) and classified the lesion as Category B. In the second reading session (with the AI), reader A changed their interpretation, determining that washout was present (probability = 0.89) and thus revised the diagnosis to Category A. Similarly, reader B (who had 5 years of experience) initially determined that there was an absence of enhancing capsule appearance (probability = 0.19) and diagnosed the lesion as Category B. In the following reading session (with the AI), reader B stated that enhancing capsule appearance was present (probability = 0.79) and revised the diagnosis to Category A accordingly.

3.3. Comparison of Radiologists with Different Levels of Experience

Table 3. AUCs for the junior and senior groups of radiologists for their performances in characterizing the 83 nodules/masses. These findings are presented for exploratory purposes only, as multiplicity correction was not applied to these specific analyses. Data in square brackets are 95% confidence intervals.

	AUC for the Junior Group			AUC for the Senior Group
	Without AI	With AI	p Value	Without AI	With AI	p Value
Enhancement	0.89 [0.84, 0.93]	0.91 [0.85, 0.97]	0.121	0.91 [0.86, 0.96]	0.93 [0.90, 0.97]	0.043
Margin	0.79 [0.71, 0.87]	0.80 [0.71, 0.88]	0.778	0.83 [0.76, 0.89]	0.83 [0.77, 0.90]	0.583
Enhancing capsule appearance	0.76 [0.65, 0.87]	0.74 [0.63, 0.86]	0.461	0.81 [0.72, 0.89]	0.79 [0.69, 0.88]	0.233
Nonrim arterial phase hyperenhancement	0.87 [0.80, 0.93]	0.88 [0.82, 0.94]	0.345	0.91 [0.86, 0.96]	0.90 [0.84, 0.95]	0.352
Nonperipheral washout	0.88 [0.82, 0.94]	0.91 [0.86, 0.95]	0.136	0.89 [0.83, 0.95]	0.91 [0.85, 0.97]	0.149
Expansion and coalescence of enhancing areas	0.85 [0.78, 0.92]	0.86 [0.78, 0.94]	0.547	0.82 [0.75, 0.89]	0.86 [0.79, 0.93]	0.014
Delayed enhancement	0.82 [0.75, 0.89]	0.86 [0.79, 0.93]	0.043	0.84 [0.77, 0.92]	0.88 [0.81, 0.95]	0.019
Peripheral enhancement	0.79 [0.72, 0.87]	0.80 [0.72, 0.88]	0.758	0.87 [0.82, 0.92]	0.88 [0.83, 0.93]	0.274

shows the AUCs for the junior and senior groups of radiologists for their performances in characterizing the 83 nodules/masses. In the junior group, a numerical trend toward improvement in the AUC was suggested for the evaluation of delayed enhancement (0.82 [0.75, 0.89] without AI vs. 0.86 [0.79, 0.93] with AI; p = 0.043). In the senior group, numerical trends toward improvement in the AUCs were suggested for the evaluation of enhancement (0.91 [0.86, 0.96] without AI vs. 0.93 [0.90, 0.97] with AI; p = 0.043), expansion and coalescence of enhancing areas (0.82 [0.75, 0.89] without AI vs. 0.86 [0.79, 0.93] with AI; p = 0.014), and delayed enhancement (0.84 [0.77, 0.92] without AI vs. 0.88 [0.81, 0.95] with AI; p = 0.019).

3.4. Inter-Observer Agreement for the Characterization of Hepatic Lesions with and Without AI

Table 4. Inter-observer agreement: the kappa statistics of the board-certified abdominal radiologists (reference standard annotators) and the ICCs of the senior and junior radiologists for the characteristics of the 83 nodules/masses. The 95% confidence intervals and p values for all reported Kappa statistics and ICCs were calculated using the bootstrap method. Data in square brackets are 95% confidence intervals. All p values were less than 0.001. No multiplicity correction was applied to these p values; they should be interpreted as suggestive only.

	Kappa Statistics for Two Abdominal Radiologists (Reference Standard Annotators)	ICCs of the 10 Senior Radiologists		ICCs of the 10 Junior Radiologists
		Without AI	With AI	Without AI	With AI
Enhancement	0.47 [0.27, 0.65]	0.63 [0.55, 0.70]	0.63 [0.55, 0.70]	0.60 [0.52, 0.66]	0.63 [0.56, 0.70]
Margin	0.51 [0.37, 0.65]	0.39 [0.31, 0.47]	0.45 [0.35, 0.53]	0.48 [0.36, 0.56]	0.47 [0.35, 0.56]
Enhancing capsule appearance	0.73 [0.58, 0.86]	0.54 [0.43, 0.63]	0.58 [0.48, 0.66]	0.39 [0.29, 0.47]	0.47 [0.37, 0.54]
Nonrim arterial phase hyperenhancement	0.81 [0.71, 0.91]	0.70 [0.64, 0.76]	0.71 [0.65, 0.77]	0.58 [0.52, 0.64]	0.61 [0.54, 0.67]
Nonperipheral washout	0.55 [0.41, 0.69]	0.71 [0.65, 0.77]	0.77 [0.71, 0.82]	0.63 [0.55, 0.69]	0.67 [0.61, 0.72]
Expansion and coalescence of enhancing areas	0.72 [0.59, 0.85]	0.52 [0.42, 0.61]	0.57 [0.47, 0.65]	0.54 [0.44, 0.62]	0.57 [0.48, 0.65]
Delayed enhancement	0.60 [0.48, 0.73]	0.53 [0.46, 0.60]	0.64 [0.56, 0.70]	0.48 [0.41, 0.54]	0.62 [0.54, 0.69]
Peripheral enhancement	0.63 [0.51, 0.76]	0.58 [0.51, 0.64]	0.63 [0.56, 0.69]	0.42 [0.36, 0.49]	0.49 [0.42, 0.55]

shows the kappa statistics of the board-certified abdominal radiologists (reference standard annotators) and the ICCs of the senior and junior radiologists for the characterization of the 83 nodules/masses. The inter-observer agreement among the reference standard annotators was moderate for enhancement (0.47 [0.27, 0.65]), margin (0.51 [0.37, 0.65]), nonperipheral washout (0.55 [0.41, 0.69]), and delayed enhancement (0.60 [0.48, 0.73]). Almost perfect agreement was indicated for nonrim APHE (0.81 [0.71, 0.91]). There was potential improvement in all the ICCs for all the lesion features when the AI system was utilized, except for margin in the junior group. In the senior group, the ICC potentially improved from fair to moderate agreement for margin (0.39 [0.31, 0.47] without AI vs. 0.45 [0.35, 0.53] with AI) and enhancing capsule appearance (0.54 [0.43, 0.63] without AI vs. 0.58 [0.48, 0.66] with AI), and from moderate to substantial agreement for delayed enhancement (0.53 [0.46, 0.60] without AI vs. 0.64 [0.56, 0.70] with AI) and peripheral enhancement (0.58 [0.51, 0.64] without AI vs. 0.63 [0.56, 0.69] with AI). In the junior group, the ICCs potentially improved from fair to moderate agreement for enhancing capsule appearance (0.39 [0.29, 0.47] without AI vs. 0.47 [0.37, 0.54] with AI), and from moderate to substantial agreement in nonrim APHE (0.58 [0.52, 0.64] without AI vs. 0.61 [0.54, 0.67] with AI) and delayed enhancement (0.48 [0.41, 0.54] without AI vs. 0.62 [0.42, 0.55] with AI).

3.5. Post Hoc Power Analysis Results

The post hoc power analysis estimated the standard errors for the difference in AUC to be 0.00478 for image characteristics and 0.00447 for imaging diagnosis. Using the target effect size ⊿AUC = 0.05, the calculated statistical power for both qualitative features and imaging diagnosis was nearly 1.00. This finding retrospectively validates that our study design (83 cases and 20 readers) provided exceptionally robust statistical power to detect a clinically meaningful difference in AUC of 0.05.

4. Discussion

In this study, we evaluated the effect of using an AI system on the performance of radiologists in characterizing and diagnosing hepatic lesions. The use of the AI system improved the AUCs for the characterization as well as for the diagnosis. The use of the AI system potentially improved the AUCs for the characterization of enhancement, nonperipheral washout, and delayed enhancement, as well as for the diagnosis of HCC. Because the multiplicity adjustment was applied only to the two comprehensive primary endpoints, results concerning individual features and diagnosis should be interpreted as exploratory findings. In addition, it was also suggested that the use of the AI system improved the AUCs for the assessment of imaging features in groups of both senior and junior radiologists, as well as the associated ICCs.

The characteristics that were better detected when the radiologists utilized the AI system (i.e., enhancement, nonperipheral washout, and delayed enhancement) are related to the contrast enhancement of hepatic lesions. These features are evaluated relative to the enhancement of the background liver parenchyma, and assessing these features may require radiologists to make subjective judgments. For example, in cirrhotic livers, the contrast enhancement of the background can be heterogeneous, and the baseline CT number changes depending on where the region of interest is placed. In fact, these features tended to have relatively low inter-observer agreement scores, even among the experienced experts who determined the reference standard (Table 4). The assessment of these characteristics was likely affected by the output of the AI system.

Detailed analysis of imaging characteristics leads to accurate imaging diagnosis and provides evidence for clinical decision-making about invasive examinations and treatments and other critical issues. In fact, LI-RADS-based characterization of hepatic lesions during dynamic contrast-enhanced examinations has been reported to result in the correct diagnosis of classic HCC [14,17,32,33,34]. Our results showed that using an AI system led to an improvement in the characterization of hepatic lesions and the potential for a subsequent increase in the performance of diagnosing classic HCC. As shown in Figure 4, the AI output of “nonperipheral washout (+)” and “enhancing capsule appearance (+)” indicated that a lesion was HCC-like, and this provided the radiologists with increased confidence in their imaging diagnosis of classic HCC.

The strict categorization adopted in our analysis, distinguishing classic HCCs (Category A) from early HCCs (included in Category C), is supported by clinical considerations. These two entities often differ significantly in imaging features, biological behavior, malignancy grade, and, crucially, clinical decision-making, particularly concerning therapeutic strategies. Classic HCCs typically demonstrate hallmark features (nonrim APHE and nonperipheral washout) that fulfill international criteria (e.g., LI-RADS) for non-invasive definitive diagnosis. Given their tendency toward higher malignancy and faster progression, their confirmation often necessitates prompt therapeutic intervention. Conversely, early HCCs are often challenging to distinguish radiologically from indeterminate lesions, such as dysplastic nodules, frequently requiring biopsy for definitive diagnosis. This ambiguity dictates a more cautious approach to management, including surveillance or delayed therapeutic planning. The observed improvement in diagnostic performance for classic HCC with AI integration, as suggested by our results, is therefore clinically significant. By enhancing the certainty of classic HCC diagnosis, imaging evaluation with AI assistance is anticipated to contribute to more rapid clinical decision-making and potentially improve patient outcomes.

Several studies have been conducted to investigate the influence of AI on the performance of radiologists based on their level of experience. It has been reported that using AI systems improves the ability of less experienced radiologists to characterize and diagnose pulmonary nodules but not that of more experienced radiologists to a significant extent [26,35,36]. In contrast, it has been reported that utilizing AI systems improves the ability of radiologists to assess hepatic lesions regardless of their years of experience. Ying et al. demonstrated that for both senior and junior radiologists, using an AI system increased the accuracy of classifying focal liver lesions as benign or malignant using contrast-enhanced CT scans [37]. Given that it can be difficult to evaluate slight differences in enhancement between a hepatic lesion and background hepatic parenchyma, the use of AI will help many radiologists with a wide range of experience levels in terms of assessing contrast enhancement patterns, which is important in the diagnosis of hepatic lesions (as opposed to assessing morphological characteristics in the diagnosis of pulmonary nodules).

AI-assisted image evaluation results in not only improved performance by radiologists but also greater inter-reader agreement [27]. The interpretation of image characteristics of hepatic lesions is often subject to inter-observer variability, and diagnostic difficulties abound in hepatology [38]. To reduce such difficulties, initiatives have been devised, such as the development of imaging guidelines (e.g., the LI-RADS); however, further improvement of the level of inter-reader agreement is required to ensure accurate diagnoses and appropriate clinical decisions are made, and to improve patient prognosis [39]. In this study, it was suggested that using an AI system can boost the inter-reader agreement associated with image interpretation. We propose that the AI system can impact on the workflow in two keyways: First, in terms of efficiency: By improving the performance of junior radiologists in subjective feature assessment, the system helps minimize the experience gap. Second, regarding quality: By improving the performance of radiologists for evaluation of the overall characterization and diagnosis, the system enables radiologists to provide more consistent, standardized, reliable, and beneficial imaging information for clinical practice. These workflow improvements are crucial for facilitating the standardized assessment of malignancy risk and management urgency for liver lesions among diagnostic radiologists regardless of levels of experience, and across different clinical departments and institutions. This standardization is expected to contribute to correct clinical decision-making and improve patient management consistency.

There were some limitations in our study. First, we used data from a single institution. Hence, there is a risk that the disease epidemiology noted in this study may be specific to the hospital where the study was conducted. Therefore, in the future, we plan to utilize multi-center data. Second, not all the lesions included in our reference standard data had associated with histopathological evidence. In this study, the reference standard for the lesions without a pathological diagnosis was determined based on imaging diagnosis to avoid the exclusion of many benign lesions and to ensure the disease epidemiology in the study reflected that in daily clinical practice as closely as possible. Future research should include larger study populations so that included cases can be evaluated solely based on pathological evidence. Third, our analysis has potential selection bias related to lesion size, as we restricted our inclusion to the two largest lesions per patient. This should be considered when generalizing our findings, particularly regarding smaller hepatic lesions. Finally, the use of different CT scanners and varying iterative reconstruction algorithms (ASiR, ASiR-V, and AIDR 3D) is a technical limitation that may have introduced subtle visual variability in image quality, potentially affecting the subjective assessment of subtle imaging features.

5. Conclusions

Our findings demonstrate the benefits associated with AI-assisted image evaluation for the characterization and diagnosis of hepatic lesions. We have shown that utilizing an AI system is helpful for radiologists, regardless of their level of experience, as it increases their ability to assess imaging features uniformly and accurately. Using an AI system is expected to contribute to improved clinical decision-making, which may ultimately improve patient outcomes.