Prostate Cancer Detection by Novice Micro-Ultrasound Users Enrolled in a Training Program

Abstract

Objective Micro-ultrasound is an imaging modality used to visualize and target prostate cancer during transrectal or transperineal biopsy. We evaluated the effectiveness of a micro-ultrasound training program and estimated the learning curve for prostate biopsy. Methods A training program registry was assessed for the rate of clinically significant prostate cancer (csPCa, grade group ≥ 2), negative predictive value, and specificity at each stage of the program. Nine metrics of biopsy quality were evaluated in 4 stages for each practitioner. Non-linear fitting and logistic regression models were used to evaluate the time-course of these metrics over training. Results Thirteen practitioners from 8 institutions completed stages 1 to 3 of the program, and 9 completed all 4 stages. Over 1190 micro-ultrasound biopsy procedures were performed. Detection of csPCa increased from 40% to 57% from stage 1 to stage 4 (P < 0.01). Stage 4 “expert” level was independently associated with higher detection of csPCa when correcting for overall risk factors (OR 1.95; P = 0.03). Limitations include the retrospective analysis and variation in biopsy protocols. Conclusion The micro-ultrasound training program was effective in improving biopsy quality and rate of csPCa detection. The presented learning curve provides an initial guide for acquiring expertise with real-time micro-ultrasound image-guided biopsy.

1. Introduction

Conventional transrectal ultrasound is typically used to guide prostate biopsy. This conventional ultrasound guided systematic biopsy strategy results in a significant proportion of false negatives and frequent under-grading of cancer [1,2]. Micro-ultrasound imaging of the prostate at 29MHz provides an improvement in detail resolution compared with conventional ultrasound. This detailed imaging required the development of educational techniques for interpreting micro-ultrasound images. With appropriate training, micro-ultrasound appears to provide improved sensitivity compared with conventional ultrasound and allows image-guided targeted prostate biopsy [3,4,5]. However, adoption of these new techniques requires training and quality assurance.

Expertly performed, micro-ultrasound guided biopsy has been demonstrated to provide risk stratification [6,7], improve rate of significant cancer detection [8,9,10], and aid in fusion biopsy accuracy [11,12]. Micro-ultrasound is therefore a promising imaging technology to reduce costs and improve accessibility for early detection of clinically significant prostate cancer (csPCa).

While some studies have demonstrated promising results from practitioners new to the technology [4,13], it is unclear how many procedures are required before competence is achieved. To assist in the training of new users, Exact Imaging (manufacturer of the ExactVu Micro-Ultrasound system) instituted a voluntary comprehensive training program in 2018. The program included 4 scheduled feedback stages in which practitioners must score above a set value over 9 metrics of biopsy quality to proceed to the subsequent stage. This retrospective study presents the data from these feedback reports.

2. Methods

2.1. Micro-Ultrasound Guided Biopsy

Biopsy cases were performed transrectally using the ExactVu Micro-Ultrasound system and EV29L transducer (Exact Imaging Inc., Markham, Canada). All biopsy procedures were performed according to site-specific protocols conforming to general practice guidelines and by urologists experienced in prostate biopsy and/or fusion biopsy. These procedures were diverse, including various anesthesia protocols (local or general, or conscious sedation) and service locations (OR, ambulatory surgical center, or clinic). However, all cases included 8 to 14 systematic samples (mean 12), as well as micro-ultrasound-targeted biopsy samples. Target locations were selected based on the PRI-MUS protocol [4], and in some cases were informed by prior mpMRI imaging. Cases including mpMRI-based targets were completed either cognitively [11,12] or using the FusionVu software-based MRI/micro-ultrasound fusion feature of the ExactVu system. We defined clinically significant prostate cancer (csPCa) as any Gleason grade > 3 + 4 = 7 cancer, a convention used by many other groups including in the PRECISION trial and Prostate Biopsy Collaborative Group (PBCG) risk calculator [14,15].

2.2. Training Program

All practitioners completed a standardized training program including 6 online learning modules and 1 hour of didactic instruction before undertaking their first live cases. An expert proctor was present for the first 10 to 15 live cases, until the practitioner demonstrated confidence in image interpretation and biopsy technique, after which the practitioner proceeded independently. The curriculum was developed and implemented by Exact Imaging on the basis of a structured review of cases from the initial clinical trial of micro-ultrasound and expert consensus amongst proctors [5]. De-identified data were collected according to stage, as presented in

Table 1. Summary of training program stages and feedback metrics used in judging whether a practitioner is ready to proceed to the next stage.

. In cases where there were delays in collection leading to additional cases performed, the cases used for analysis were randomly selected to avoid bias. Practitioners had to complete each stage of the program successfully to move to the next stage. After successfully completing stage 4, practitioners were awarded a certificate of quality assurance.

2.3. Feedback Metrics

Nine metrics associated with biopsy quality were selected on the basis of previous findings during the first micro-ultrasound based randomized trial (NCT02079025). These metrics are shown in Table 1. With 3 exceptions, each metric was assigned a point value based on importance and used to judge whether a practitioner could proceed to the next stage. The exceptions were for data saving, cancer detection rate, and anesthesia technique, used only for informational purposes, or in the case of data saving to recommend repeating the stage, because of insufficient data.

2.4. Statistical Analysis

The Mann-Whitney U-test was used to compare non-parametric values, with a threshold of P < 0.05 considered significant. A Gompertz function [16] was used to model learning curve through non-linear curve fitting. A multivariate logistic regression model was used to compare the detection rates at each stage with clinical risk factors such as age, PSA, DRE status, and family history. Clinical risk factors were combined using the validated PBCG nomogram [15] into a single value to prevent overfitting on this limited dataset. Only practitioners who had successfully completed stages 1 to 3 of the program were included in this analysis to avoid bias from having different individuals at stage 1 compared with the later stages. All computational modeling was performed in MATLAB (Mathworks, Natick, United States).

The study was approved by the local ethics committees and the authors certify that the study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments. Informed consent was obtained from all individual participants included in the study.

3. Results

In total, 60 feedback reports from 13 practitioners at 8 institutions in Germany, Austria, the Netherlands, and the United States between January 2018 and January 2020 were included. These 60 reports include data from 412 biopsy sessions, including 200 at stage 1, 69 at stage 2, 79 at stage 3, and 64 at stage 4. In total, this group of practitioners completed over 1190 micro-ultrasound guided prostate biopsy sessions (each with >12 individual biopsy samples) with a range of total cases per practitioner of 40 to 160. Of the feedback reports,12/60 (20%) required the practitioner to repeat the assessment stage. The majority of these repeats were at stage 1 (8/12, 67%), as presented in more detail in

Table 2. Number of feedback sessions completed at each stage, with failure rate (required to repeat stage) and rate of significant cancer detected.

.

Median patient age was 70 (IQR 64 to 74) with PSA 7.6 (IQR 5.9 to 11.7) ng/mL. A total of 95/412 cases had a positive DRE (23%), and 89/412 (22%) had a prior biopsy. A pre-biopsy mpMRI was available in 134/412 cases. The demographics did not vary significantly between feedback stages. Detection of csPCa increased from 40% to 57% from stage 1 to stage 4 (P < 0.01). The rate of csPCa detected is shown in Figure 1 by stage. The result of the multivariate analysis is presented in

Table 3. Multivariate logistic regression model results accounting for patient risk levels.

and shows improvement in stages 3 and 4 “expert” level with odds ratios of 1.71 and 1.95 and P = 0.06 and 0.03, respectively. This model was tested using leave-one-out validation with an area under the curve (AUC) of 0.675. The use of mpMRI was not standardized between centers, with 33 feedback reports incorporating mpMRI targeting (csPCa rate 44%) and 27 feedback reports not incorporating mpMRI targeting (average csPCa rate 54%). In order to investigate this difference in light of confounding variables such as PSA and age, mpMRI was added to the multivariate model presented in Table 3. The multivariate OR was not significant (OR 1.07, P = 0.77) and the model AUC dropped slightly from 0.675 to 0.672 suggesting the use of MRI did not influence the learning curve.

Ability to correctly identify negative cases also varied with stage of training, although the relationship appears more complex. Figure 2 shows an initial negative predictive value (NPV) of 86% in stage 1 with 11% of cases marked as non-suspicious or negative. The number of cases marked as non-suspicious increased through stages 2 and 3 while NPV declined. This trend reversed in stage 4 in which the fraction of cases marked non-suspicious decreased somewhat to 17% while NPV rose to 91%. Despite these changes in false negative rate, specificity showed a simple improvement over the course of the program increasing from 16% to 37% (P = 0.01).

4. Discussion

Appropriate training and quality control are necessary for any diagnostic imaging technology, and have been instrumental in the adoption of CT colonography and mammography. Training and quality control have also been acknowledged as important in the diagnosis of prostate cancer through MRI. Recent expert consensus is that 1000 reads are necessary to be considered an expert in prostate MRI [17]. This educational program analysis demonstrates that a formal training and quality control program was effective in increasing the ability to detect clinically significant cancer by 17%. Optimizing the negative predictive value appears to be a more prolonged process, with the number of false negatives increasing through stage 3 of the program. However, this effect appears transient with higher values in stage 4 and a steady increase in specificity throughout.

The improvement occurred over the duration of the feedback program, which was established at 90 cases.

The ability to consistently evaluate suspicious lesions according to the PRI-MUS protocol, with an impact on the PCa detection rate, is important to ensure adequate biopsy quality performance of the new micro-ultrasound system. The learning curve and reproducibility of the evaluation of the prostate using micro-ultrasound is the basis for the adoption of this new technology and the potential practitioner acceptance.

Of note, there is a large body of literature about the importance of training and feedback during interpretation of mpMRI. Akin et al. demonstrated an increase in AUC from 0.52 to 0.66 for identifying lesions in the peripheral zone using mpMRI after didactic lectures [18]. Similarly, Rosenkrantz et al. demonstrated that with feedback accuracy interpreting prostate MRI using PI-RADS v1 improved from 58.1% to 77.4% over 124 examinations [19]. These studies both report reader accuracy rather than detection rate, which complicates any comparison with the work described here. Meng et al. demonstrate a 26% improvement in targeted detection rates over a 4-year period with mpMRI fusion biopsy, which more closely aligns with the results presented here, while Calio et al. demonstrated an 11.6% improvement in fusion biopsy detection rate over a 9-year period [20,21]. However, in both these cases the population is limited to men with suspicious mpMRIs. These data reflect an improvement in positive predictive value rather than overall detection rate in a general biopsy population.

Similar studies of fusion biopsy accuracy have focused on accuracy in reaching a prespecified target position, showing improvement within the first 98 cases for a robotic fusion system [22]. Other studies have investigated complication rates and differences in use between junior and senior operators without reference to number of cases performed on a particular system, which was not investigated here [23].

The primary limitation of this study is the retrospective nature. As data were compiled from the feedback program registry, the biopsy procedures themselves were not standardized and differences in biopsy technique may be associated with different learning curves. In particular, use of mpMRI was not standardized between centers. To examine this effect, we added mpMRI to the multivariate model presented in Table 3; however, the multivariate OR was not significant, suggesting the use of MRI did not influence the learning curve. Complications during biopsy were not recorded as part of this study, but complication rates for the procedure are generally low and unrelated to the guidance/imaging device used [5]. Further, the broad cross-section of physicians evaluated included those with prior ultrasound fellowship training who may be expected to have developed expertise in the technique earlier than those without this training.

5. Conclusions

A formal training and feedback program for microultrasound establishes a standardized scan, and reporting with micro-ultrasound and adds significant value in improving clinically significant cancer detection rates. The learning curve presented here suggests expert sensitivity is achieved within the first 20 to 40 cases, while expert specificity generally takes 40 to 90 cases to develop. This provides a guide for practitioners who are interested in acquiring expertise with real-time microultrasound image-guided biopsy. Educational program enhancements and prospective registries are currently evolving.