Société Internationale d’Urologie Journal

Background/Objectives: Large language models (LLM) such as ChatGPT have demonstrated competitive performance on standardised medical examinations. Within the Australian urology training pathway, ChatGPT has been evaluated on the Generic Surgical Sciences Examination (GSSE), which gates entry to surgical training, and on the urology fellowship examination, the exit assessment. This study aimed to assess performance of ChatGPT on the intermediate urology-specific Surgical Sciences Examination (SSE). Methods: A 120-question sample paper was constructed from a question bank compiled by previous urology SSE candidates and reviewed by a senior urology consultant and a urology trainee. Questions were administered to ChatGPT-5.5 across three content domains (anatomy, physiology, and pathology) and three question formats (Type A single-best-answer, Type B stem–reason, and Type X multiple true–false). Items spanned the major urological subspecialty domains. Overall accuracy was compared against the 62.8% pass mark from the July 2025 SSE sitting. Results: ChatGPT scored 332 of 417 marks (79.6%, 95% confidence interval [CI] 75.5–83.2%), exceeding the 62.8% pass mark (p < 0.001). Performance differed significantly across sections (p = 0.002), with physiology (88.0%) outperforming anatomy (70.7%, p < 0.001). Performance also differed significantly across question types (p = 0.035), with Type B questions (50%) performing significantly worse than Type X questions (80.6%, p = 0.026). Conclusions: ChatGPT exceeded the SSE pass standard on a representative sample paper, completing the characterisation of LLM performance across the three principal examinations of Australian urology training. Reproducible weaknesses in anatomical content and relational reasoning preclude its use as a standalone study resource.