Société Internationale d’Urologie Journal

This issue marks my first year as Editor-in-Chief of the Société Internationale d’Urologie Journal (SIUJ) [...]

Background/Objectives: Recently, the European Association of Urology (EAU) guidelines presented the EAU guidelines bot to assist urologists in the reading of the guidelines; however, to date, no external validation is available. The aim of our study is to assess the accuracy, completeness, and clarity of the guidelines bot in reading renal transplantation guidelines and clinical cases. Methods: A total of 103 questions derived from the recommendations included in the EAU guidelines on renal transplantation, together with 10 distinct clinical scenarios, were created. Each question and clinical case was submitted to the EAU guidelines bot, and the corresponding responses were independently evaluated by two expert urologists in terms of accuracy, completeness, and clarity. Each domain was scored using a 5-point Likert scale. In cases of disagreement between the two reviewers, a third urologist was consulted. Accuracy, completeness, and clarity were analyzed for the clinical cases and per chapter and grade of recommendation for the questions. Results: Overall, 103 questions were generated. Regarding accuracy, 101/103 responses (98%) were classified as accurate, corresponding to scores of 4–5, whereas 2/103 responses (2%) showed fair accuracy, corresponding to a score of 3. No responses were considered inaccurate, defined as a score of 1–2. In terms of completeness, 101/103 responses (98%) were judged to be complete, with scores of 4–5, while 2/103 responses (2%) were rated as fairly complete, with a score of 3. No responses were classified as incomplete. With regard to clarity, 102/103 responses (99%) were considered clear, corresponding to scores of 4–5, whereas 1/103 response (1%) was rated as fairly clear, with a score of 3. No responses were deemed unclear. No significant differences were observed when comparing responses related to strong versus weak recommendations. For the clinical scenarios, 8/10 responses (80%) were rated as accurate, with scores of 4–5, while 2/10 responses (20%) showed fair accuracy, with a score of 3. Regarding completeness, all 10 responses (100%) were classified as fairly complete, corresponding to a score of 3. Finally, in terms of clarity, 8/10 responses (80%) were judged to be clear, with scores of 4–5, whereas 2/10 responses (20%) were rated as fairly clear, with a score of 3. Conclusions: The EAU guidelines bot represents an accurate tool for renal transplantation. Besides some occasional inaccuracies the bot is an excellent tool for urologists, especially to clarify doubt about the EAU guidelines. At the same time, it can give good support when facing a clinical case.

Background/Objectives: Large language models (LLMs) such as ChatGPT, Gemini and Claude are increasingly used by patients to research new diagnoses, including testicular cancer. The aim of this study was to evaluate and compare the quality of responses generated by these three platforms to patient-facing frequently asked questions on testicular cancer, including a focused evaluation of fertility and sexual health questions. Methods: Fifteen questions were developed from established patient education resources and refined by senior urology consultant review. Each question was posed to ChatGPT, Gemini and Claude using a standardised patient-perspective prompt. The 45 responses were de-identified, randomised and assessed in a single consensus meeting by a urology panel using the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool. Comparative analysis used the Friedman test with Bonferroni-corrected pairwise Wilcoxon signed-rank testing. Results: All three models produced moderate quality responses, with a mean total QAMAI score of 21.8 for ChatGPT, 22.4 for Claude, and 22.1 for Gemini (Friedman p = 0.246). Clarity differed significantly between models (adjusted p = 0.022), with Claude and Gemini outperforming ChatGPT. In the fertility and sexual health domain, Claude (24.0) and Gemini (23.0) outperformed ChatGPT (19.7). No response from any model spontaneously cited references or directed the reader to external resources. Conclusions: ChatGPT, Gemini and Claude deliver broadly equivalent overall quality on patient-facing testicular cancer questions, with inter-model differences in clarity and in the fertility and sexual health domain. The absence of source citation is a clinically relevant limitation. These tools should be regarded as a supplement to, rather than a substitute for, specialist counselling.