Are Urologists Ready for Interpretation of Multiparametric MRI Findings? A Prospective Multicentric Evaluation

Aim: To assess urologists’ proficiency in the interpretation of multiparametric magnetic resonance imaging (mpMRI). Materials and Methods: Twelve mpMRIs were shown to 73 urologists from seven Italian institutions. Responders were asked to identify the site of the suspicious nodule (SN) but not to assign a PIRADS score. We set an a priori cut-off of 75% correct identification of SN as a threshold for proficiency in mpMRI reading. Data were analyzed according to urologists’ hierarchy (UH; resident vs. consultant) and previous experience in fusion prostate biopsies (E-fPB, defined as <125 vs. ≥125). Additionally, we tested for differences between non-proficient vs. proficient mpMRI readers. Multivariable logistic regression analyses (MVLRA) tested potential predictors of proficiency in mpMRI reading. Results: The median (IQR) number of correct identifications was 8 (6–8). Anterior nodules (number 3, 4 and 6) represented the most likely prone to misinterpretation. Overall, 34 (47%) participants achieved the 75% cut-off. When comparing consultants vs. residents, we found no differences in terms of E-fPB (p = 0.9) or in correct identification rates (p = 0.6). We recorded higher identification rates in urologists with E-fBP vs. their no E-fBP counterparts (75% vs. 67%, p = 0.004). At MVLRA, only E- fPB reached the status of independent predictor of proficiency in mpMRI reading (OR: 3.4, 95% CI 1.2–9.9, p = 0.02) after adjusting for UH and type of institution. Conclusions: Despite urologists becoming more familiar with interpretation of mpMRI, their results are still far from proficient. E-fPB enhances the proficiency in mpMRI interpretation.


Introduction
Multiparametric magnetic resonance imaging (mpMRI) of the prostate has become the gold standard imaging modality for suspicious prostate cancer (PCa) [1,2]. Through its different sequences, such as T2, diffusion weighted imaging (DWI), dynamic contrast imaging and spectroscopy, MRI is able to provide information on the cellularity of solid tissues, perfusion parameters in neoplasms and the relative concentration of intracellular metabolites [3]. mpMRI represents a mandatory step in the flowchart of PCa diagnosis as well as a useful tool for staging and preoperative planning of PCa treatment [4][5][6]. The use of mpMRI ranges from the image-fusion technique for the execution of targeted prostate biopsies to an important role in treatment planning, such as the feasibility of nerve-sparing procedures and better assessment of clinical stage [7]. Furthermore, a prebiopsy mpMRI may save unnecessary prostate biopsies and improve the selection of patients for active surveillance in low-risk PCa patients [8].
The interpretation of mpMRI is challenging even in the hands of experienced radiologists as it implies understanding of different MRI sequences, how to manipulate the images and how to assign suspicion scores. Although urologists are not required to have the same skills and precision required as radiologists, the "best practice" requires that the surgeon has adequate skills to read the imaging autonomously and to obtain the basic information for accurate prostate biopsies and treatment delivery [9]. Recently, several training courses have been developed for both radiology residents [10] and urologists [11][12][13], with durable success in increasing diagnostic accuracy. However, to the best of our knowledge, no previous study addressed the ability of urologists (either fully formed or in training) to identify suspicious lesions in mpMRI, with particular attention in identifying those who might be more proficient in the task. To address this void, we relied on a multicentric prospective survey at seven Italian institutions.

Materials and Methods
Suspicious mpMRI images of 12 patients were selected by an expert uro-radiologist (G.C.), whose interpretation was considered as the gold standard, and shown to urologists from seven institutions (five academic and two non-academic) from August 2019 to November 2020.

Definition of the Test Cases
All the selected scans were attained with a 1.5-T mpMRI study (Achieva and Achieva dStream; Philips Medical Systems, Best, The Netherlands) with a phased-array surface coil and an endorectal coil (BPX-15; Bayer Medical Care, Indianola, PA, USA). All 12 cases presented a PIRADS ≥ 3 lesion [14] (Figures 1 and 2

Test Administration
The mpMRIs were administered using the RadiAnt DICOM Viewer 4.0.1 software (Medixant, Poznan, Poland). Every participant had a maximum of 10 min to evaluate each mpMRI. Four sequences were shown at the same time in loop: T2 weighted image (T2), diffusion weighted imaging (DWI), apparent diffusion coefficient (ADC), contrast enhancement (CE).
First, each participant answered an anonymous survey (see Supplementary Materials S1) with demographics and professional data. Second, participants were asked to locate each lesion/nodule on a prostate map (ACR-ESUR-AdMeTech 2019, Pi-Radsv2.1, Page 68, Figure  16: Sector map diagram v2.1). Specifically, the study aimed to only assess the urologists' ability to find the suspicious nodule and not to assign a PIRADS score, which we consider exclusively a radiologists' prerogative. Before starting the test, participants were informed about the presence of zero or maximum one lesion/nodule in each mpMRI. Additionally, participants were informed that all patients were biopsy-naïve and harbored PSA levels higher than 4 ng/mL. Five "presenters" (GM, FE, ME, SS, MO) administered the test in the seven different centers. Correct answer was defined as correctly identifying all the following: anterior vs. posterior, left vs. right, apical vs. median vs. base. We relied on an a priori cut-off of 75% correct identification of the site of suspicious lesions as a threshold for proficiency in mpMRI reading (9/12).

Statistical Analysis
Continuous variables were summarized as medians and interquartile ranges (IQR) and categorical variables as the number of subjects and percentage values.
Statistical analyses were based on two steps. First, data were analyzed according to urologists' hierarchy (resident vs. consultant) as well as according to previous experience in performing fusion prostate biopsies (fPB, defined as <125 vs. ≥125) [15]. Additionally, we tested for differences between non-proficient vs. proficient mpMRI readers. For these analyses, Wilcoxon rank sum test, Pearson's Chi-squared test and Fisher's exact test were used where appropriate.
Second, we aimed to assess potential predictors of proficiency in mpMRI reading. For this purpose, we relied on multivariable logistic regression models. Covariates consisted of type of Institution (academic vs. non-academic), previous experience in fPB and urologists' hierarchy (resident vs. consultant).
For all statistical analyses, R software environment for statistical computing and graphics (version 3.4.3, R Foundation, Vienna, Austria) was used. All tests were 2 sided with a level of significance set at p < 0.05.
Forty-nine participants (67%) were already out of their learning curve for fPB. Similarly, 43 (59%) and 31 (42%) responders came from institutions with more than 300 referrals for PCa and more than 200 prostatectomies per year, respectively.
In our cohort of responders, we found no differences in experience in fPB between consultants vs. residents (p = 0.9). Similarly, no differences were recorded in the percentage of correctly identified lesions (p = 0.6). Only in case number 10, a statistically significant difference was recorded among consultants (59% of correct answers) vs. residents (22%, p = 0.001).
In the comparison according to experience in fPB (Table 1B), we recorded a statistically significantly greater percentage of correct responses in urologists outside their learning curve vs. their in-training counterparts (75% vs. 67%, p = 0.004). Specifically, the former correctly identified more frequently the lesion of cases 1, 3 and 6 (all p < 0.05).
When analyzing proficient vs. non-proficient readers (Table 1C), we identified statistically significant differences only in experience in fPB (47% vs. 21%, p = 0.016). Furthermore, a statistically significantly greater percentage of proficient responders correctly identified the lesions.    In multivariable logistic regression analyses, only experience in fPB reached the status of independent predictor of proficiency in reading mpMRI (OR: 3.4, 95% CI 1.2-9.9, p = 0.02) after adjusting for urologists' hierarchy and type of institution (Table 2).

Discussion
Since the introduction of mpMRI for PCa diagnosis in clinical practice, the concern of the correct identification of site, size and type of lesion by radiologists, as well as by urologists, has emerged. In fact, although the radiologist is responsible for the diagnosis and attribution of a PIRADS class to prostatic lesions found on mpMRI, it can be postulated that the urologists should be able to correctly identify the lesions in order to perform an accurate prostate biopsy as well as to better plan prostatectomy.
In this light, in recent years, inter-reader agreement in interpreting mpMRI among radiologists has been extensively evaluated [16][17][18][19]. The recent feeling is that in clinical practice, the agreement between radiologists in detecting suspicious lesions on magnetic resonance images could be higher than previously thought [20]. Giganti et al. recently demonstrated a strong reproducibility in the assessment of Prostate Imaging Quality (PI-QUAL) score between radiologists with high expertise in prostate mpMRI [21]. Moreover, an improvement in inter-observer agreement among radiologists in reporting mpMRI was reported after the introduction of PIRADSv2.1 compared with PIRADSv2 [22,23].
Furthermore, some studies evaluated the role of computer-aided diagnosis (CAD) of prostate cancer on mpMRI, using artificial intelligence (AI) in order to reduce missed cancers and unnecessary biopsies and increase inter-observer agreement between clinicians [24]. However, there is insufficient evidence to suggest the clinical deployment of AI algorithms at present.
To date, several studies have tested the proficiency of radiologists in mpMRI interpretation, also addressing the impact of different learning tools advocated to enhance their reading skills [12,25]. Contrarily, only sparse data concerned the ability of urologists in the interpretation of mpMRI images. Kasivisvanathan et al. recently emphasized the promising role of mpMRI teaching courses in improving urologists' reading ability [11]. However, predictors of higher proficiency in mpMRI interpretation were not provided, nor was detailed information regarding both mpMRI scans and urologists characteristics. We addressed this void relying on a multicentric survey among seven Italian institutions and drew several noteworthy observations. First, it is worrisome that only 47% of responders achieved our a priori standard of proficiency (75% correct answers). In light of these results, surgeons should still consult radiologists during surgical planning. It is of utmost importance that urological surgeons receive formal training in mpMRI reading.
Second, the least correctly identified lesions were numbers 3, 4, 6 and 10; it is of note that all those nodules were anterior lesions, except for case 10, which was a PIRADS 3 posterior basal lesion. Interestingly, even when responders reached the 75% cut-off of proficiency reading, those lesions were less frequently correctly identified. It can be therefore postulated that anterior and low-PIRADS lesions represent a challenge even for more experienced eyes.
Third, we did not record differences in identifying correct lesions between residents vs. consultants. These results can be justified by the same exposure to fusion prostate biopsy between the two groups (33% vs. 32%, p > 0.9). However, it is of note that consultants correctly identified the only PIRADS 3 lesion included in the study (case 10) almost twice as frequently as residents.
Finally, and foremost, experience in fPB seems to play the leading role in achieving proficiency in mpMRI reading. Indeed, when comparing experienced vs. non-experienced, correct responses were recorded in 75% vs. 67%, respectively (p = 0.004). Moreover, urologists exposed to fPB were more likely to identify anterior lesions (case 3: 71% vs. 22%, p < 0.001; case 6: 71% vs. 29%, p < 0.001). Additionally, among those who reached proficiency cut-off, 47% had experience in fPB compared to only 21% of non-proficient readers (p = 0.02). This result was further confirmed by MVA analyses, where experience in fPB was the only independent predictor of correct mpMRI reading (OR: 3.4, 95% CI 1.2-9.9, p = 0.02) after adjusting for urologists' hierarchy and type of institution.
Our study is not devoid of limitations; first, both the relatively small number of lesions selected for evaluation and the number of participants could possibly affect the statistical robustness of our analyses. However, we believe that the 12 selected lesions represented a good sample of what urologists can face in their activity and, to the best of our knowledge, our study is the first and the largest to address this literature gap.
Second, our data did not allow us to test for possible predictors of increased difficulty in identification of suspicious nodules as side, site, size of lesions and their PIRADS. Additionally, we only selected mpMRI bearing unifocal suspicious lesions, which does not reflect clinical reality. Indeed, future research should aim at also including in such tests normal mpMRIs as well as those with multiple lesions or benign prostatic diseases (prostatitis, ectopic nodules, microcalcifications, previous biopsies, etc.). Moreover, we feel that radiologists, as well as urologists, should be provided with a short patient summary to help with the correct interpretation of mpMRI sequences.
Third, the study is limited by the lack of a training intervention to see whether the accuracy can be improved with an educational program.
Finally, participants of our study all came from Italian institutions. Despite the effect of including academic vs. non-academic, high vs. low-volume centers and residents vs. consultants, we are not able to conclude that our results are generalizable to other European and non-European settings.

Conclusions
Our results suggest that experience in fusion prostate biopsies enhances the ability of correctly identify suspicious nodules at mpMRI. Moreover, anterior lesions may be more challenging to locate. Funding: The research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.