Skip to Content
MDPIMDPI

Advanced

Journal of ImagingJournal of Imaging
PDF
  • Article
  • Open Access

21 September 2026

19 Pages

Large Language Models Meet Gynecologic Ultrasound: Advancing the Characterization of ADNEXal Masses

,
,
,
,
,
,
,
and
1
Department of Interdisciplinary Medicine (DIM), University of Bari “Aldo Moro”, 70124 Bari, Italy
2
Gynecologic Oncology Unit, IRCCS Istituto Tumori “Giovanni Paolo II”, 70124 Bari, Italy
3
Translational Biomedicine and Neuroscience Department (DiBraIn), University of Bari “Aldo Moro”, Polyclinic of Bari, 70124 Bari, Italy
4
SSD Radiologia Senologia, IRCCS Istituto Tumori “Giovanni Paolo II”, 70124 Bari, Italy
J. Imaging2026, 12(9), 462;https://doi.org/10.3390/jimaging12090462
This article belongs to the Special Issue Artificial Intelligence and Deep Learning Models for Human Diseases Diagnosis and Prediction
Version Notes
Order Reprints

Abstract

Ovarian cancer (OC) is the second most common gynecological malignancy and remains one of the leading causes of gynecological cancer-related mortality worldwide. A major clinical challenge is the lack of an accurate and widely applicable strategy for identifying patients at high risk of malignancy at an early stage. In this context, artificial intelligence (AI) has emerged as a promising tool to improve diagnostic performance. Among AI technologies, large language models (LLMs) have recently shown considerable potential in healthcare applications. In this study, we evaluated the diagnostic performance of ChatGPT (GPT-5) in classifying 300 adnexal masses as benign or malignant and compared its performance with that of the IOTA Simple Rules, the ADNEX model, and expert subjective assessment. We also assessed ChatGPT’s ability to predict the most likely histological diagnosis for each lesion. All adnexal masses were described using the International Ovarian Tumor Analysis (IOTA) terminology, and histopathological examination served as the reference standard. Our findings showed that expert subjective assessment achieved the highest overall diagnostic performance for both benign/malignant classification (accuracy 87.3%; 95% CI, 83.0–90.9%) and prediction of the presumed histological diagnosis. ChatGPT A and ChatGPT B reached a sensitivity of 72.3% and 73.5%, a specificity of 74.5% and 75.9%, a positive predictive value of 75.2% and 76.5%, and a negative predictive value of 71.5% and 72.8%, respectively (inconclusive responses counted as misclassifications), with an overall accuracy of 73.3% and 74.7%. After adequate validation, large language models might complement existing decision-support tools for less experienced examiners, without replacing expert evaluation. Their ease of use and reliance on standardized ultrasound descriptors make them accessible to ultrasonographers with varying levels of expertise.

1. Introduction

Ovarian cancer (OC) is the fifth most common oncological disease and the fourth leading cause of cancer death in the United States [1], with a 5-year survival rate of approximately 45% [2]. Because early detection significantly reduces mortality, accurately determining whether a pelvic mass is benign or malignant is critical.
Expert sonographers are frequently able to determine the nature of an adnexal mass during the pre-surgical workup with an accuracy of 92%, compared with 82–87% for less experienced examiners [3].
In order to make this assessment objective, reproducible, and universally shared, the definition of a universal language has become necessary. To standardize this assessment universally, the IOTA (International Ovarian Tumor Analysis) group [4] introduced objective ultrasound models, such as the Simple Rules [5] and the ADNEX model (Assessment of Different NEoplasias in the adneXa) [6] (Table 1 and Table 2).
Table 1. IOTA Simple Rules.
Table 2. IOTA ADNEX model (Assessment of Different NEoplasias in the adneXa).
However, the subjective evaluation made by an experienced sonographer has consistently been confirmed as an accurate method to discriminate between benign and malignant lesions [3] with better performance compared with the other ultrasound methods [7].
Since expert sonographers are not always available, AI has emerged as a prominent source of healthcare innovation. Although gynecology relies heavily on imaging, AI implementation in this field is still in its early stages compared to other medical specialties [8].
Multiple studies have demonstrated its validity in the fields of screening [9,10], triage [11,12], diagnosis [13,14], drug development [15,16], treatment [17,18], monitoring [19], and image interpretation [20,21].
In 1999, Timmerman et al. showed that an artificial neural network can be trained to discriminate between a suspected benign or malignant nature of an adnexal mass using some simple ultrasound criteria and some clinical information (e.g., menopausal state and serum level of CA 125) [22].
Acharya et al. developed a machine learning model to automatically discriminate between the diagnosis of benignity and malignancy based on a dataset of 1000 benign and 1000 malignant ultrasound images. This model achieved an accuracy of 99.9%, a sensitivity of 100%, and a specificity of 99.8% [23].
A deep learning model based on a convolutional neural network (CNN) was proposed in the study by Aramendia-Vidaurreta et al. to obtain an automatic classification of adnexal masses, combining the ultrasound characteristics and the age of the patients. This model revealed an overall accuracy of 98.8% with a sensitivity of 98.5% and a specificity of 98.9% [24].
Currently, innovation is driven by large language models (LLMs). These deep learning systems are trained on massive datasets, allowing them to process realistic text and mimic human reasoning and communication. The most widespread variant today is ChatGPT (developed by OpenAI).
Given the exponential growth of AI and the integration of ChatGPT into daily life, evaluating its reliability in healthcare has become an urgent priority.
To date, however, no study has systematically evaluated a state-of-the-art LLM against the validated IOTA instruments (Simple Rules and ADNEX model) and expert subjective assessment on the same consecutive cohort of surgically confirmed adnexal masses described with standardized IOTA terminology. Prior work on artificial intelligence for adnexal masses has focused on image-based approaches, such as neural networks and radiomics [22,23,24], whereas the ability of LLMs to reason over standardized ultrasound descriptors remains essentially unexplored. This is the research gap addressed by the present study.
The aim of our study is to evaluate what could be the prospects of the use of ChatGPT in ultrasound evaluation of adnexal masses, specifically exploring potential and limits in predicting the histological diagnosis. Therefore, this study aims to offer a point of reflection on the new application perspectives of AI such as ChatGPT in the healthcare field and on the possibility that its introduction could facilitate medical work or, conversely, hinder it.

2. Materials and Methods

All patients were enrolled at the Gynecology Oncology Unit of the “Giovanni Paolo II” I.R.C.C.S. Cancer Institute in Bari and of the Hospital Polyclinic in Bari between August 2022 and August 2025. All the ovarian masses were analyzed by ultrasound examination according to the IOTA criteria. Clinical and histopathological data were collected from clinical charts.
Ultrasound examinations were performed by one expert operator at each participating center, each having at least 15 years of experience in gynecologic ultrasound.
Transabdominal examination was performed with a 3.5–5.0 MHz convex transducer, and transvaginal examination with a 9.0–5.0 MHz broadband transducer (Voluson Expert 22, GE HealthCare, Chicago, IL, USA). The main stages of the study are briefly summarized in Figure 1.
Figure 1. Five-step summary process of the study.
The study was conducted according to the guidelines of the Declaration of Helsinki and the protocol was approved by the Institutional Review Board of “Giovanni Paolo II” I.R.C.C.S. Cancer Institute, Bari, Italy. All patients provided informed consent to use the data for scientific purposes.
We collected 339 ultrasound examinations of adnexal masses, which were classified according to their morphology (Figure 2), echogenicity (Figure 3), and vascularization (Figure 4).
Figure 2. Ultrasound images showing adnexal mass morphologies, according to IOTA terms and definitions: unilocular (A), unilocular solid (B), multilocular (C), multilocular solid (D), solid (E,F).
Figure 3. Ultrasound images showing adnexal mass echogenicity, according to IOTA term and definitions: anechoic (A), ground glass (B), low level (C), hemorrhagic (D), mixed (E), not applicable/solid mass (F).
Figure 4. Ultrasound images showing adnexal mass vascularization, according to IOTA terms and definitions: color score 1 when no blood flow is detected (A), color score 2 when minimal flow is detected (B), color score 3 when moderate flow is detected (C), color score 4 when intense flow is detected (D).
In the case of bilateral adnexal masses, the mass with the more complex ultrasound morphology was considered in the analysis. If both masses had similar ultrasound morphology, the larger mass or the one more easily accessible by ultrasound was included.
Inclusion criteria were:
-
Suspected adnexal disease for which the patient was referred to centers adherent to the study;
-
Execution of a pre-surgical pelvic ultrasound examination;
-
Pre-surgical check of serum CA125;
-
Patients selected for surgery.
We excluded 39 patients because of missing CA125. All the remaining 300 masses were evaluated by an expert ultrasound examiner who described them using IOTA terminology and assigned the presumptive diagnosis of benignity and malignancy using the Simple Rules, the ADNEX model, and subjective assessment.
The approach to assess the pelvis was, at first, the transvaginal ultrasound, while the transabdominal ultrasound was used just for the masses which cannot be adequately evaluated for dimension, localization, or for patients who were virgo.
Our reference was the histological examination which identified 48.33% benign masses and 51.67% malignant masses.
The OpenAI ChatGPT system (GPT-5) was used to classify adnexal masses as benign or malignant based on standardized ultrasound descriptors.
All cases were submitted through the ChatGPT web interface via a plus subscription, using ChatGPT (GPT-5; model introduced in ChatGPT on 7 August 2025 OpenAI, San Francisco, CA, USA), between 2 September 2025 and 13 October 2025. Web browsing, memory, and all additional tools were disabled. No system instructions other than those reported in Supplementary Material were provided. Sampling parameters (e.g., temperature, top_p) could not be customized in the web interface and were kept at default settings.
Each case was submitted once in a newly opened, independent conversation, so that no information could carry over between cases. The prompt required a predefined response format, and responses were read and coded by G.S. and F.A.; a response was coded as inconclusive when it did not explicitly categorize the lesion as benign or malignant. The complete prompts, including the task instructions, the exact wording and order of the input variables, and the required output format, are reported verbatim in Supplementary Material.
Only fully de-identified, structured ultrasound descriptors and serum CA125 values were entered into the platform; no images, personal identifiers, dates, or free-text clinical notes were transmitted. The information submitted could not, alone or in combination, allow patient re-identification, in compliance with the General Data Protection Regulation (GDPR).
The main analysis focused on different combinations of parameters (Table 3; Figure 5).
Table 3. Summary of the ChatGPT prompting configurations (A–E) used to evaluate the benignity or malignancy of an adnexal mass based on combinations of ultrasound, laboratory, and clinical data.
Figure 5. Predefined and additional ultrasound parameters selected to describe the adnexal masses, according to IOTA terminology. (A–C) show examples of predefined parameters, respectively morphology ((A), e.g., multilocular cyst), size (B), and vascularization of the lesion (C), used in all of the ChatGPT configurations proposed. (D) shows the presence of ascites, an additional clinical and ultrasound parameter considered in ChatGPT C configuration. (E) represents an example of acoustic shadows, an additional ultrasound parameter considered in ChatGPT D configuration. (F) represents a vascularized papilla, an additional ultrasound parameter considered in ChatGPT configuration E.
At first, the accuracy of the ChatGPT prompting configurations in the diagnosis of benignity/malignancy was tested against the Simple Rules, the ADNEX model, and subjective assessment. Questions were asked in English describing the adnexal formations according to IOTA terminology and always using the same sequence: morphology, diameter, color score (ChatGPT A) and, then, the additional characterizing element for configurations ChatGPT B, C, D, and E (Table 3). The statement “is a unilocular solid ovarian cyst of 14 mm with a color score 2 and a vascularized papilla more likely benign or malignant?” is an example of the question scheme we followed.
Afterwards, the accuracy of ChatGPT in the determination of the presumed histological diagnosis was tested through standardized questions. The following information was provided: morphology, size, color score, and serum level of presurgical CA125. According to the IOTA Consensus Statement, we chose to add the information of the cystic content for unilocular and unilocular solid morphologies. For multilocular and multilocular solid morphologies, we also added the cystic content and the number of loci. The statement “which is the histological presumed diagnosis for a unilocular solid ovarian cyst with the following features: diameter 199 mm, color score 2, anechoic content, CA125 of 31?” is an example of the question scheme we followed.
The result of the histological examination of the lesion after its surgical removal was used as the reference standard in all phases of the study.
Agreement between the different diagnostic methods was evaluated using Cochran’s Q test for comparing more than two percentages from non-independent samples. If the null hypothesis of equality was rejected, Sheskin’s test for comparing two paired proportions was applied [25].
The accuracy assessment was carried out with particular reference to malignant cases, defining true positives (TP), false negatives (FN), true negatives (TN) and false positives (FP) (Table 4). For each diagnostic method, sensitivity, specificity, positive and negative predictive values, and overall accuracy were determined, along with their respective confidence intervals using the Clopper–Pearson method. Inconclusive ChatGPT responses and cases in which the Simple Rules were not applicable were not treated as test-negative results: in the primary analysis, they were counted as misclassifications, i.e., as false negatives when the lesion was malignant and as false positives when it was benign at histology (intention-to-diagnose approach), and a sensitivity analysis restricted to conclusive classifications was also performed. Differences in sensitivity, specificity, and predictive values between methods were described by comparing point estimates and their 95% confidence intervals.
Table 4. Accuracy assessment: definition of true positive, false positive, false negative, and true negative used in the analysis. Histological examination is the reference standard.

3. Results

The study population consisted of 300 patients, with a median age of 52 years. At the time of the assessment, 162 patients were in menopause, 106 were of fertile age, and for the remaining 32 patients, these data were unavailable. Ascites was present in 41/300 cases.
Table 5 summarizes the characteristics of the examined adnexal formations.
Table 5. Main ultrasound features of the adnexal masses examined (n = 300). Values are numbers of cases, with percentages of the total population in parentheses.
The outcome of the histological examination reported a total of benign formations of 48.33% and a total of malignant formations of 51.67%.
ChatGPT A and ChatGPT B were unable to determine whether the lesion was benign or malignant in 9% (27/300) and 4.33% (13/300) of cases, respectively; to reflect the clinical utility of the configuration, inconclusive responses and cases in which Simple Rules were not applicable were categorized as diagnostic failures (i.e., incorrect outputs), as the configuration failed to recognize the nature of the lesion.
The configurations ChatGPT A and B which were assessed for the total study population agree with the histological examination (Figure 6).
Figure 6. Benign and malignant classifications and pairwise agreement among diagnostic methods applied to the whole cohort (n = 300): histology, subjective assessment, ADNEX, Simple Rules and ChatGPT configurations A and B (configurations C and D, evaluated in morphological subgroups, are shown in Figure 7). (Left): Pairwise agreement matrix; cells below the diagonal report Cohen’s κ for the benign classification (blue scale) and cells above the diagonal report Cohen’s κ for the malignant classification (orange scale), with darker shading indicating greater agreement. (Right): Percentage of masses classified as benign (blue) or malignant (orange) by each method; grey segments correspond to masses without a benign or malignant classification. Asterisks indicate statistically significant pairwise differences in diagnostic proportions following Cochran’s Q test and post hoc comparison of paired proportions (minimum significant difference method). Histopathological examination was used as the reference standard.
The test to compare proportion of diagnosis among different methods showed a statistically significant difference for both benign (Figure 6, Q = 108.04, p < 0.001) and malignant (Figure 6, Q = 263.25, p < 0.001). The post hoc comparison between pairs was performed through the Sheskin method, which determined the minimum significant difference (MSD) between two paired proportions with a p-value lower than 0.05, if the Cochrane test was significant. For malignant, MSD was 10.1%, while for benign, the MSD was 9.5%.
In a descriptive evaluation of the subpopulations (prompting configurations ChatGPT C, ChatGPT D, and ChatGPT E), performance was observed when additional parameters were considered beyond the predefined ones (morphology, size, and vascularization). Within its respective subpopulation, ChatGPT C aligned with the histological diagnosis for both malignant and benign lesions. Conversely, ChatGPT D aligned primarily with benign diagnoses, while ChatGPT E yielded a high proportion of inconclusive outputs (Figure 7 for configurations C and D). Since configurations C, D, and E were evaluated in different eligible subpopulations, these comparisons are strictly descriptive and exploratory: a formal assessment of the incremental value of each additional variable would require comparing the baseline and expanded prompts within the same cases.
Figure 7. Benign and malignant classifications and pairwise agreement among diagnostic methods within the eligible morphological subgroups. (A) Solid and multilocular solid masses (n = 135), including ChatGPT configuration C (additional information on ascites). (B) Solid and unilocular solid masses (n = 253), including ChatGPT configuration D (additional information on acoustic shadows). In each panel, the matrix on the left reports pairwise Cohen’s κ coefficients for the benign classification below the diagonal (blue scale) and for the malignant classification above the diagonal (orange scale), with darker shading indicating greater agreement; the bars on the right show the percentage of masses classified as benign (blue) or malignant (orange) by each method, gray segments corresponding to masses without a benign or malignant classification. Given the different eligible subpopulations, comparisons within subgroups are descriptive. Histopathological examination was used as the reference standard.
Comparing the accuracy of the methods examined about the presumed diagnosis benignity vs. malignancy, the subjective assessment results were the most performing overall.
In our study (Table 6), the subjective assessment reported the highest accuracy (87.3%; 95% CI 83.0–90.9%), while the Simple Rules reported the lowest (66.7%; 95% CI 61.0–72.0%).
Table 6. Accuracy of the methods examined (subjective assessment, Simple Rules, ADNEX, ChatGPT A, and ChatGPT B) concerning the presumed diagnosis benignity vs. malignancy referring to the total population. The subjective assessment results were the best performing overall.
Analyzing the agreement between the methods, it could be noticed that ADNEX had the highest percentage of malignant diagnosis (225/300 cases, 75%) (Table 7A), while the histological examination reported a percentage of 51.67% cases (155/300 cases). That result could be explained, in our examination, as referable to the nature of the method, which uses a cut-off of 10% to define an adnexal mass as a higher risk of malignancy, with subsequent surgical management. Therefore, in our study, we considered all the masses with an ADNEX score >10% as malignant, but, of course, not all of them were determined to be malignant at the histological diagnosis. This could be the reason why ADNEX overestimated malignant masses in our study population with considerable disagreement compared with other methods. Compared to ADNEX, ChatGPT A and B show a lower number of false positives but higher false negatives, resulting in lower sensitivity but greater specificity.
Table 7. Diagnostic performance of subjective assessment, Simple Rules, ADNEX, ChatGPT A, and ChatGPT B in the total population of 300 patients, with histopathology as the reference standard. (A) reports true positives, false positives, false negatives, and true negatives; inconclusive ChatGPT responses and cases in which the Simple Rules were not applicable are reported separately and were not counted as test-negative. (B) reports sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) as % (n/N) with 95% confidence intervals.
On the other hand, subjective assessment seems to slightly overestimate the percentage of malignant masses while remaining homogeneous with the histological result. In this regard, the use of an additional method such as ChatGPT to support subjective diagnosis could be of clinical utility.
After analyzing the agreement between methods, we calculated the positive predictive value (PPV), the negative predictive value (NPV), the sensitivity (SE), and the specificity (SP), with their 95% confidence intervals (Table 7B). Since the identification of tumor pathology is of greater clinical use, the focus of this analysis was on the malignant diagnosis.
Subjective assessment showed a higher sensitivity than ChatGPT A (93.5% vs. 72.3%) and ChatGPT B (93.5% vs. 73.5%), with non-overlapping 95% confidence intervals (Table 7B). Configurations ChatGPT A and B showed similar sensitivity (72.3% vs. 73.5%) and specificity (74.5% vs. 75.9%), with largely overlapping confidence intervals.
Regarding predictive values, the main difference between subjective assessment and ADNEX concerned the positive predictive value (83.8% vs. 67.6%, with non-overlapping 95% confidence intervals), which confirms our previous observation related to the ADNEX overestimation of malignancy.
ChatGPT A and B were also similar in terms of positive predictive value (75.2% vs. 76.5%) and negative predictive value (71.5% vs. 72.8%).
The similarity between sensitivity, specificity, and positive and negative predictive values between the two proposed configurations ChatGPT A and B could be due to the similarity between the two methods. In fact, ChatGPT B includes the same ultrasound parameters (morphology, size, and color score) plus the laboratory data (serum level of CA125).
Finally, both predictive values of subjective assessment (PPV 83.8%, NPV 92.1%) were higher than those of ChatGPT A and B, the difference being most evident for the negative predictive value.
The aim of the second phase of the study is to evaluate the feasibility of the use of LLM instead of an expert evaluation in the definition of the presumed histological diagnosis.
The histological reports with the greatest statistical and clinical significance are cystadenoma, cystadenofibroma, teratoma, endometrioma, fibroid, borderline tumor, and invasive carcinoma.
Compared with the histological report, the numbers of cases assigned by ChatGPT and by subjective assessment were similar for teratoma and endometrioma, whereas ChatGPT deviated more markedly than subjective assessment from the histological distribution for cystadenoma, cystadenofibroma, fibroid, and borderline tumor (Table 8 and Figure 8).
Table 8. Comparison between methods (subjective assessment vs. ChatGPT) in the detection of the presumed histological diagnosis. The histological report is the reference standard. Values are numbers of cases, with percentages of the total population in parentheses.
Figure 8. Distribution of the presumed histological diagnoses assigned by ChatGPT and by subjective assessment compared with the histopathological examination (n = 300). Bars show the percentage of masses assigned to each diagnostic category; labels report the number of cases with the corresponding percentage (data from Table 8).
As can also be seen in Figure 8, ChatGPT seems to overestimate some diagnoses (cystadenoma, borderline tumors, mature teratoma) and underestimate others (cystadenofibroma, invasive carcinomas or metastatic ones, fibroids).
Looking at our results, it is possible to hypothesize that they are due to a limited ability of the system to interpret and contextualize some ultrasound features.
For instance, teratomas and fibroids are both characterized by the ultrasound appearance of acoustic shadows. The discrepancy in the results obtained could be linked to the incorrect interpretation and contextualization of this ultrasound feature, resulting in incorrectly classifying many fibroids as teratomas.
Likewise, cystadenoma and cystadenofibroma could be similar in their ultrasound appearance. Therefore, we could hypothesize that a large proportion of unidentified cystadenofibromas have been misclassified as cystadenomas. Furthermore, misdiagnosed invasive carcinomas may have been classified as borderline cancer. Lastly, ChatGPT does not express an opinion in 3.67% of cases.

4. Discussion

OC ranks as the seventh most diagnosed cancer among women worldwide and the second most common gynecological malignancy [26]. Its high lethality is primarily driven by a rapid peritoneal spread and the absence of effective screening programs capable of detecting the disease at an early stage [27]. Identifying accurate tools for early diagnosis and prognosis remains an unmet clinical need [28]. In this scenario, scientific research is increasingly exploring the applications of AI and LLMs, such as ChatGPT, within the gynecological field [29,30,31,32,33,34].
In 2021, the World Health Organization (WHO) published the “Ethics and Governance of Artificial Intelligence for Health” report, establishing a core principle: medical decisions should be made by humans, supported but never replaced by AI. Indeed, the misuse of AI entails significant ethical risks, profoundly impacting:
  • Privacy and Cybersecurity: The vulnerability of sensitive patient data.
  • Informed Consent and Transparency: The lack of clear protocols regarding how AI is utilized during the diagnostic–therapeutic process.
  • Skill Degradation: The risk of a progressive decline in the clinical skills of healthcare professionals due to over-reliance.
  • Patient-Guided Misuse: The danger of patients seeking care outside the health system without proper medical surveillance [35].
LLMs have a unique ability to mimic human language. That may result in false, inaccurate, or incomplete statements with a potential negative consequence on patients’ health. Moreover, LLMs could be trained on poor quality or biased data [36].
Despite the growing literature on clinical LLMs, current reviews typically address wide-ranging clinical use cases at the expense of a dedicated focus on disease diagnosis. This has created a critical research gap, as the specific methodologies, strengths, and challenges of deploying LLMs for diagnostic tasks are rarely analyzed in depth.
Existing specialty-specific reviews, focusing on areas such as digestive or infectious diseases [37,38], fail to provide a broader, cross-specialty analysis that incorporates diverse data types, LLM techniques, and clinical diagnostic workflows.
Currently, there are still few studies concerning the application of LLM in the field of gynecology.
A 2024 Italian study revealed that official national guidelines (AIOM) scored significantly higher in clarity, consistency, and completeness compared to answers provided by ChatGPT regarding OC [39].
On the other hand, the study by Cascella et al. highlighted that, despite LLMs’ impressive capabilities, they could obtain poor performance in real settings such as the medical field, where complex high-level thinking is needed [40].
Conversely, ChatGPT demonstrated high concordance with human evaluators when tested in triaging and recommending management interventions for fictitious obstetric and gynecologic emergencies [41].
The study by Ługowski et al. evaluated ChatGPT’s clinical knowledge by assessing its accuracy in answering questions from the Polish Specialty Certificate Examination (SCE) in Obstetrics and Gynecology [42]. They demonstrated that even though ChatGPT is capable of answering very difficult questions, its accuracy is better when solving less complex problems; therefore, the newer version should aid physicians in their daily practice. Moreover, it should be used in English to provide the best accuracy of answers.
To mitigate the risks of misdiagnosis, it is essential to implement systematic processes, ensuring the accuracy of training datasets, and to establish clear procedures for handling situations where the AI’s output conflicts with clinical judgment.
With this study, we aimed to expand the application of LLMs in the field of gynecology, placing a stronger emphasis on diagnostic models. Specifically, we evaluated the diagnostic potential of ChatGPT in the ultrasound assessment of 300 adnexal masses. Although the proposed AI configurations showed a lower overall accuracy than subjective expert assessment, the results are encouraging. ChatGPT offers clear advantages, including easy accessibility and educational utility due to its highly detailed explanations.
However, the model has a critical structural limitation: it cannot interpret ultrasound images directly. It relies entirely on a human operator to provide correct data. In this study, the descriptions were inputted by an expert operator using standardized IOTA terminology. This demonstrates that the accuracy of the AI is strictly dependent on the expertise of the physician, confirming that AI should support, rather than replace, medical judgment.
The lower diagnostic performance observed for ChatGPT is likely explained by its inability to discriminate and appropriately contextualize individual ultrasound features. Interestingly, incorporating additional input parameters, including serum CA-125 levels, vascularized papillary projections, acoustic shadows, and ascites, did not appreciably improve diagnostic accuracy. This finding is not unexpected, as none of these variables is pathognomonic for a specific diagnosis. For example, CA-125 may be elevated in benign conditions such as endometriosis; acoustic shadows may result from fibrous septa, calcifications, or solid tissue, which can be observed in both benign and malignant lesions; vascularized papillary projections are suggestive, but not exclusive, of borderline ovarian tumors; and ascites may occur in non-neoplastic conditions, whereas some patients with ovarian cancer present without ascites. Therefore, these variables require clinical interpretation rather than isolated pattern recognition.
Our text-based approach should also be considered against state-of-the-art artificial intelligence systems that analyze ultrasound images directly. Deep learning models trained on pelvic ultrasound images have achieved expert-level discrimination of ovarian cancer in large multicenter series [43], and a recent systematic review confirmed the rapid growth and overall promising accuracy of image-based artificial intelligence in gynecologic oncology [44]. Radiomics-based machine learning has likewise shown good diagnostic performance for adnexal masses [45]. Compared with these systems, an LLM working on standardized IOTA descriptors currently offers lower discriminative performance, but it requires no imaging pipeline, is immediately available in clinical settings, and leverages the same standardized lexicon already used in routine reporting. The two approaches are therefore complementary, and multimodal systems combining direct image analysis with structured descriptors represent a natural next step.
The application of LLMs in the clinical management of adnexal masses and ovarian cancer is supported by the recent literature across both pre-diagnostic screening and ultrasound stratification. LLM-based natural language processing has proven effective in extracting non-coded symptom signatures from unstructured electronic health records to support early cancer detection [46]. Additionally, LLMs have demonstrated high accuracy and reliability in standardizing O-RADS categorization directly from free-text ultrasound reports, highlighting their utility as clinical decision-support tools [47].
The principal limitation associated with the lower diagnostic accuracy of ChatGPT is the potential for inappropriate clinical management of adnexal masses. False positive results may lead to unnecessary surgery, whereas false negative classifications may delay referral and treatment of ovarian malignancies. Therefore, the probability of benignity or malignancy generated by ChatGPT should always be interpreted together with the patient’s clinical history, physical examination, laboratory findings, and comprehensive ultrasound assessment.
Future studies should evaluate ChatGPT in real-world clinical settings, where more comprehensive clinical information is available than the limited set of ultrasound descriptors and additional variables included in the present study. Moreover, multimodal artificial intelligence models combining direct ultrasound image analysis with structured clinical data should be explored. Multicenter validation studies are also warranted to assess the generalizability of ChatGPT across different imaging platforms and healthcare settings. Finally, domain-specific fine-tuning of large language models using curated gynecological ultrasound reports may further improve diagnostic performance.
Although the current evidence is encouraging, the integration of LLMs into routine clinical practice will require further improvements in model performance through training on larger, high-quality datasets, together with the development of robust ethical and regulatory frameworks addressing transparency, privacy, data security, and informed consent. Most importantly, LLMs should be regarded as decision-support tools that complement, rather than replace, clinical expertise. Importantly, we acknowledge that evaluating each case with a single prompt is a limitation of the present study as it does not capture output variability; accordingly, ongoing and future trials will incorporate repeated queries to formally assess response stability.

5. Conclusions

Expert subjective assessment remains the most accurate approach for the ultrasound characterization of adnexal masses. Although ChatGPT demonstrated lower diagnostic performance, it showed potential as an additional decision-support aid for less experienced examiners, to be used alongside validated IOTA instruments and never as a substitute for referral to an expert sonographer. Its performance depends on the use of standardized IOTA terminology and should always be interpreted in conjunction with clinical evaluation rather than as a stand-alone diagnostic method. Future research should focus on the paired evaluation of prompting strategies on identical case sets, the formal assessment of response stability across repeated queries, prospective multicenter validation, the comparison of different LLMs, including open-weight models deployable on premise for privacy reasons, and studies measuring the impact on clinician workload and diagnostic efficiency rather than diagnostic agreement alone.

Supplementary Materials

The supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jimaging12090462/s1.

Author Contributions

Conceptualization, G.S., S.D.N. and F.A.; methodology, V.L. and G.C. (Gennaro Cormio); resources, G.S. and P.T.; data curation, G.C. (Giuseppe Colonna); writing—original draft preparation, G.S. and V.L.; writing—review and editing, S.D.N. and F.A.; visualization, D.L.F. and L.G.Z.; supervision, G.C. (Gennaro Cormio); project administration, F.A.; All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Giovanni Paolo II I.R.C.C.S. Cancer Institute, Via Orazio Flacco 65, Bari, Italy (protocol code 18925 on 5 August 2024).

Data Availability Statement

The original contributions presented in this study are included in the article and Supplementary Materials. Further inquiries can be directed to the corresponding author.

Acknowledgments

During the preparation of this study, the authors used ChatGPT (GPT-5, OpenAI, San Francisco, CA, USA) to assist in the classification of adnexal masses and the prediction of the most likely histological diagnosis. The authors have reviewed and edited the output and take full responsibility for the content of this publication.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Armstrong, D.K.; Alvarez, R.D.; Bakkum-Gamez, J.N.; Barroilhet, L.; Behbakht, K.; Berchuck, A.; Chen, L.-M.; Cristea, M.; DeRosa, M.; Eisenhauer, E.L.; et al. Ovarian Cancer, Version 2.2020, NCCN Clinical Practice Guidelines in Oncology. J. Natl. Compr. Cancer Netw. 2021, 19, 191–226. [Google Scholar] [CrossRef] [Scilit]
  2. Webb, P.M.; Jordan, S.J. Epidemiology of epithelial ovarian cancer. Best Pract. Res. Clin. Obstet. Gynaecol. 2017, 41, 3–14. [Google Scholar] [CrossRef] [Scilit]
  3. Timmerman, D.; Schwärzler, P.; Collins, W.P.; Claerhout, F.; Coenen, M.; Amant, F.; Vergote, I.; Bourne, T.H. Subjective assessment of adnexal masses with the use of ultrasonograpgy: An analysis of interobserver variability and experience. Ultrasound Obstet. Gynecol. 1999, 13, 11–16. [Google Scholar] [CrossRef] [Scilit]
  4. Timmerman, D.; Valentin, L.; Bourne, T.H.; Collins, W.P.; Verrelst, H.; Vergote, I. Terms, definitions and measurement to describe the sonographic features of adnexal tumors: A con-sensus opinion from tha International Ovarian Tumor Analysis (IOTA) group. Ultrasound Obstet. Gynecol. 2000, 16, 500–505. [Google Scholar] [CrossRef] [Scilit]
  5. Timmerman, D.; Testa, A.C.; Bourne, T.; Ameye, L.; Jurkovic, D.; Van Holsbeke, C.; Paladini, D.; Van Calster, B.; Vergote, I.; Van Huffel, S.; et al. Simple ultrasound-based rules for the diagnosis of ovarian cancer. Ultrasound Obstet. Gynecol. 2008, 31, 681–690. [Google Scholar] [CrossRef] [Scilit]
  6. SIEOG. Nuove Terminologie in Ecografia Ginecologica; Engineering and Physical Sciences Research Council: Swindon, UK, 2017; pp. 11–15. [Google Scholar]
  7. Meys, E.M.J.; Kaijser, J.; Kruitwagen, R.; Slangen, B.; Van Calster, B.; Aertgeerts, B.; Verbakel, J.; Timmerman, D.; Van Gorp, T. Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis. Eur. J. Cancer 2016, 58, 17–29. [Google Scholar] [CrossRef] [Scilit]
  8. United Kingdom Engineering and Physical Sciences Research Council. Artificial Intelligence Technologies. Available online: https://epsrc.ukri.org/research/ourportfolio/researchareas/ (accessed on 15 January 2026).
  9. Abramoff, M.D.; Lavin, P.T.; Birch, M.; Shah, N.; Folk, J.C. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digit. Med. 2018, 1, 39. [Google Scholar] [CrossRef] [Scilit]
  10. Wang, P.; Berzin, T.M.; Brown, J.R.G.; Bharadwaj, S.; Becq, A.; Xiao, X.; Liu, P.; Li, L.; Song, Y.; Zhang, D.; et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: A prospec-tive randomised controlled study. Gut 2019, 68, 1813–1819. [Google Scholar] [CrossRef] [Scilit]
  11. Annarumma, M.; Withey, S.J.; Bakewell, R.J.; Pesce, E.; Goh, V.; Montana, G. Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks. Radiology 2019, 291, 196–202. [Google Scholar] [CrossRef] [Scilit]
  12. Titano, J.J.; Badgeley, M.; Schefflein, J.; Pain, M.; Su, A.; Cai, M.; Swinburne, N.; Zech, J.; Kim, J.; Bederson, J.; et al. Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat. Med. 2018, 24, 1337–1341. [Google Scholar] [CrossRef] [Scilit]
  13. Liu, X.; Faes, L.; Kale, A.U.; Wagner, S.K.; Fu, D.J.; Bruynseels, A.; Mahendiran, T.; Moraes, G.; Shamdas, M.; Kern, C.; et al. A comparison of deep learning performance against health-care professionals in detecting diseases from med-ical imaging: A systematic review and meta-analysis. Lancet Digit. Health 2019, 1, e271–e297. [Google Scholar] [CrossRef] [Scilit]
  14. Marcus, G.M. The Apple Watch can detect atrial fibrillation: So what now? Nat. Rev. Cardiol. 2020, 17, 135–136. [Google Scholar] [CrossRef] [Scilit]
  15. Ianevski, A.; Giri, A.K.; Gautam, P.; Kononov, A.; Potdar, S.; Saarela, J.; Wennerberg, K.; Aittokallio, T. Prediction of drug combination effects with a minimal set of experiments. Nat. Mach. Intell. 2019, 1, 568–577. [Google Scholar] [CrossRef] [Scilit]
  16. Wang, X.; Terashi, G.; Christoffer, C.W.; Zhu, M.; Kihara, D. Protein Docking Model Evaluation by 3D Deep Convolutional Neural Networks. Bioinformatics 2020, 36, 2113–2118. [Google Scholar] [CrossRef] [Scilit]
  17. Zhdanov, A.; Atluri, S.; Wong, W.; Vaghei, Y.; Daskalakis, Z.J.; Blumberger, D.M.; Frey, B.N.; Giacobbe, P.; Lam, R.W.; Milev, R.; et al. Use of Machine Learning for Predicting Escitalopram Treatment Outcome from Electroencephalography Recordings in Adult Patients with Depression. JAMA Netw. Open 2020, 3, e1918377. [Google Scholar] [CrossRef] [Scilit]
  18. Langlotz, C.P.; Allen, B.; Erickson, B.J.; Kalpathy-Cramer, J.; Bigelow, K.; Cook, T.S.; Flanders, A.E.; Lungren, M.P.; Mendelson, D.S.; Rudie, J.D.; et al. A roadmap for foundational research on artificial intelligence in medical imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019, 291, 781–791. [Google Scholar] [CrossRef] [Scilit]
  19. Safavi, K.C.; Khaniyev, T.; Copenhaver, M.; Seelen, M.; Langle, A.C.Z.; Zanger, J.; Daily, B.; Levi, R.; Dunn, P. Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Sur-gical Care. JAMA Netw. Open 2019, 2, e1917221. [Google Scholar] [CrossRef] [Scilit]
  20. Chang, P.J. Moving Artificial Intelligence from Feasible to Real: Time to Drill for Gas and Build Roads. Radiology 2020, 294, 432–433. [Google Scholar] [CrossRef] [Scilit]
  21. Majkowska, A.; Mittal, S.; Steiner, D.F.; Reicher, J.J.; McKinney, S.M.; Duggan, G.E.; Eswaran, K.; Chen, P.-H.C.; Liu, Y.; Kalidindi, S.R.; et al. Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiolo-gist-adjudicated Reference Standards and Population-adjusted Evaluation. Radiology 2020, 294, 421–431. [Google Scholar] [CrossRef] [Scilit]
  22. Timmerman, D.; Verrelst, H.; Bourne, T.H.; De Moor, B.; Collins, W.P.; Vergote, I.; Vandewalle, J. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet. Gynecol. 1999, 13, 17–25. [Google Scholar] [CrossRef] [Scilit]
  23. Acharya, U.R.; Sree, S.V.; Krishnan, M.M.R.; Saba, L.; Molinari, F.; Guerriero, S.; Suri, J.S. Ovarian tumor characterization using 3D ultrasound. Technol. Cancer Res. Treat. 2012, 11, 543–552. [Google Scholar] [CrossRef] [Scilit]
  24. Aramendia-Vidaurreta, V.; Cabeza, R.; Villanueva, A.; Navallas, J.; Alcázar, J.L. Ultrasound Image Discrimination between Benign and Malignant ADNEXal Masses Based on a Neural Network Approach. Ultrasound Med. Biol. 2016, 42, 742–752. [Google Scholar] [CrossRef] [Scilit]
  25. Sheskin, D.J. Handbook of Parametric and Non-Parametric Statistical Procedures, 5th ed.; Chapman & Hall/CRC: Boca Raton, FL, USA, 2011. [Google Scholar]
  26. Siegel, R.L.; Miller, K.D.; Jemal, A. Cancer statistics, 2020. CA Cancer J. Clin. 2020, 70, 7–30. [Google Scholar] [CrossRef] [Scilit]
  27. Loizzi, V.; Leone, L.; Camporeale, A.; Resta, L.; Selvaggi, L.; Cicinelli, E.; Cormio, G. Neoadjuvant Chemotherapy in Advanced Ovarian Cancer: A Single-Institution Experience and a Review of the Literature. Oncology 2016, 91, 211–216. [Google Scholar] [CrossRef] [Scilit]
  28. Arezzo, F.; Loizzi, V.; La Forgia, D.; Moschetta, M.; Tagliafico, A.S.; Cataldo, V.; Kawosha, A.A.; Venerito, V.; Cazzato, G.; Ingravallo, G.; et al. Radiomics Analysis in ovarian cancer: A narrative review. Appl. Sci. 2021, 11, 7833. [Google Scholar] [CrossRef] [Scilit]
  29. Van Timmeren, J.E.; Leijenaar, R.T.H.; van Elmpt, W.; Reymen, B.; Lambin, P. Feature selection methodology for longitudinal cone-beam CT radiomics. Acta Oncol. 2017, 56, 1537–1543. [Google Scholar] [CrossRef] [Scilit]
  30. Venerito, V.; Angelini, O.; Cazzato, G.; Lopalco, G.; Maiorano, E.; Cimmino, A.; Iannone, F. A convolutional neural network with transfer learning for automatic discrimination between low and high-grade synovitis: A pilot study. Intern. Emerg. Med. 2021, 16, 1457–1465. [Google Scholar] [CrossRef] [Scilit]
  31. Venerito, V.; Angelini, O.; Fornaro, M.; Cacciapaglia, F.; Lopalco, G.; Iannone, F. A machine learning approach for predicting sustained remission in rheumatoid arthritis patients on bio-logic agents. JCR J. Clin. Rheumatol. 2021, 28, e334–e339. [Google Scholar] [CrossRef] [Scilit]
  32. Arezzo, F.; Cormio, G.; La Forgia, D.; Santarsiero, C.M.; Mongelli, M.; Lombardi, C.; Cazzato, G.; Cicinelli, E.; Loizzi, V. A machine learning approach applied to gynaecological ultrasound to predict progression-free survival in ovarian cancer patients. Arch. Gynecol. Obstet. 2022, 306, 2143–2154. [Google Scholar] [CrossRef] [Scilit]
  33. Venerito, V.; Iannone, F. Large language model-driven sentiment analysis for facilitating fibromyalgia diagnosis. RMD Open 2024, 10, e004367. [Google Scholar] [CrossRef] [Scilit]
  34. Venerito, V.; Puttaswamy, D.; Iannone, F.; Gupta, L. Large language models and rheumatology: A comparative evaluation. Lancet Rheumatol. 2023, 5, e574–e578. [Google Scholar] [CrossRef] [Scilit]
  35. World Health Organization. Ethics and Governance of Artificial Intelligence for Health. Guidance on Large Multi-Modal Models; World Health Organization: Geneva, Switzerland, 2024. [Google Scholar]
  36. Zhou, S.; Xu, Z.; Zhang, M.; Xu, C.; Guo, Y.; Zhan, Z.; Fang, Y.; Ding, S.; Wang, J.; Xu, K.; et al. Large language models for disease diagnosis: A scoping review. npj Artif. Intell. 2025, 1, 9. [Google Scholar] [CrossRef] [Scilit] [PubMed] [PubMed Central]
  37. Omar, M.; Brin, D.; Glicksberg, B.; Klang, E. Utilizing natural language processing and large language models in the diagnosis and prediction of infectious diseases: A systematic review. Am. J. Infect. Control 2024, 52, 992–1001. [Google Scholar] [CrossRef] [Scilit]
  38. Giuffrè, M.; Kresevic, S.; You, K.; Dupont, J.; Huebner, J.; Grimshaw, A.A.; Shung, D.L. Systematic review: The use of large language models as medical chatbots in digestive diseases. Aliment. Pharmacol. Ther. 2024, 60, 144–166. [Google Scholar] [CrossRef] [Scilit]
  39. Piazza, D.; Martorana, F.; Curaba, A.; Sambataro, D.; Valerio, M.R.; Firenze, A.; Pecorino, B.; Scollo, P.; Chiantera, V.; Scibilia, G.; et al. The Consistency and Quality of ChatGPT responses Compared to Clinical Guidelines for Ovarian Cancer: A Delphi Approach. Curr. Oncol. 2024, 31, 2796–2804. [Google Scholar] [CrossRef] [Scilit]
  40. Cascella, M.; Montomoli, J.; Bellini, V.; Bignami, E. Evaluating the Feasibility of ChatGPT in healthCare: An analysis of multiple clinical and research sce-narios. J. Med. Syst. 2023, 47, 33. [Google Scholar] [CrossRef] [Scilit]
  41. Psilopatis, I.; Heindl, F.; Cupisti, S.; Fischer, U.; Kohlmann, V.; Schneider, M.; Bader, S.; Krueckel, A.; Emons, J. The role of artificial intelligence in gynecologic and obstetric emergencies. Eur. J. Obstet. Gynecol. Reprod. Biol. 2025, 306, 94–100. [Google Scholar] [CrossRef] [Scilit]
  42. Ługowski, F.; Babińska, J.; Ludwin, A.; Stanirowski, P.J. Comparative analysis of ChatGPT 3.5 and ChatGPT 4 obstetric and gynecological knowledge. Sci. Rep. 2025, 15, 21133. [Google Scholar] [CrossRef] [Scilit] [PubMed] [PubMed Central]
  43. Gao, Y.; Zeng, S.; Xu, X.; Li, H.; Yao, S.; Song, K.; Li, X.; Chen, L.; Tang, J.; Xing, H.; et al. Deep learning-enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: A retrospective, multicentre, diagnostic study. Lancet Digit. Health 2022, 4, e179–e187. [Google Scholar] [CrossRef] [Scilit]
  44. Moro, F.; Ciancia, M.; Zace, D.; Vagni, M.; Tran, H.E.; Giudice, M.T.; Zoccoli, S.G.; Mascilini, F.; Ciccarone, F.; Boldrini, L.; et al. Role of artificial intelligence applied to ultrasound in gynecology oncology: A systematic review. Int. J. Cancer 2024, 155, 1832–1845. [Google Scholar] [CrossRef] [Scilit]
  45. Chiappa, V.; Bogani, G.; Interlenghi, M.; Salvatore, C.; Bertolina, F.; Sarpietro, G.; Signorelli, M.; Castiglioni, I.; Raspagliesi, F. The Adoption of Radiomics and machine learning improves the diagnostic processes of women with Ovarian MAsses (the AROMA pilot study). J. Ultrasound 2021, 24, 429–437. [Google Scholar] [CrossRef] [Scilit]
  46. Funston, G.; Park, N.; Thompson, M.; Yetisgen, M.; Goff, B.A.; Kessler, L.; Walter, F.M. Using large language models to identify prediagnostic clinical features of ovarian cancer from healthcare records: A population-based case-control study. Br. J. Gen. Pract. 2026, 76, e544–e551. [Google Scholar] [CrossRef] [Scilit] [PubMed] [PubMed Central]
  47. Sun, Y.; Shen, H.; Zhang, L.; Jiang, Y.; Zheng, Q.; Du, L.; He, M.; Wu, L.; Xie, H. Transforming adnexal mass assessment: How large language model improve ovarian-adnexal reporting and data system interpretation and sonographer performance. Abdom. Radiol. 2026, 51, 5251–5260. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Parameter-Efficient LoRA-GRL Adaptation for Cross-Center Classification of Benign and Malignant Lung Nodules on Heterogeneous Standard-Dose Chest CT: A Multi-Institutional Study from Palestine
Previous
Added Diagnostic Value of Stress CT Perfusion to Coronary CT Angiography Across Increasing Coronary Calcium Burden
Next