Next Article in Journal
Preclinical Evaluation of a Microwave-Based Accessory Device for Colonoscopy in an In Vivo Porcine Model with Colorectal Polyps
Previous Article in Journal
‘Earlier than Early’ Detection of Breast Cancer in Israeli BRCA Mutation Carriers Applying AI-Based Analysis to Consecutive MRI Scans
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 15
Issue 12
10.3390/cancers15123121
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Identification of Key Elements in Prostate Cancer for Ontology Building via a Multidisciplinary Consensus Agreement

by
Amy Moreno
1,*,
Abhishek A. Solanki
2,
Tianlin Xu
3,
Ruitao Lin
3,
Jatinder Palta
4,
Emily Daugherty
5,
David Hong
6,
Julian Hong
7,
Sophia C. Kamran
8,
Evangelia Katsoulakis
9,
Kristy Brock
10,
Mary Feng
7,
Clifton Fuller
1,
Charles Mayo
11 and
BDSC Prostate Cancer
†
1
Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
2
Department of Radiation Oncology, Loyola University Medical Center, Berwyn, IL 60402, USA
3
Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
4
Department of Medical Physics, Virginia Commonwealth University, Richmond, VA 23284, USA
5
Department of Radiation Oncology, University of Cincinnati College of Medicine, Cincinnati, OH 45267, USA
6
Department of Radiation Oncology, University of Southern California, Los Angeles, CA 90089, USA
7
Department of Radiation Oncology, University of California San Francisco, San Francisco, CA 93701, USA
8
Department of Radiation Oncology, Massachusetts General Hospital, Boston, MA 02129, USA
9
Department of Radiation Oncology, James A Haley VA Medical Center, Tampa, FL 33612, USA
10
Department of Imaging Physics, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
11
Department of Radiation Physics, University of Michigan, Ann Arbor, MI 48109, USA
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Author to whom correspondence should be addressed.
†
Membership of the BDSC Prostate Cancer Consortium is provided in the Supplementary Materials
Cancers 2023, 15(12), 3121; https://doi.org/10.3390/cancers15123121
Submission received: 1 April 2023 / Revised: 25 May 2023 / Accepted: 1 June 2023 / Published: 8 June 2023
(This article belongs to the Section Cancer Informatics and Big Data)
Browse Figures
Review Reports Versions Notes

Abstract

:

Simple Summary

Prostate cancer (PCa) is one of the most common cancers and the second leading cause of cancer-related deaths in men in the United States. Accurate diagnosis, management, and posttherapy surveillance are critical for optimizing survival and patient quality of life. Sharing information among providers is a challenge due to the use of different health information systems or data capture workflows which can lead to ambiguity and misinterpretation of shared information. The aim of our study was to formulate an expert panel-based consensus on PCa-specific key data elements. Using the Delphi method, PCa experts developed a two-tiered thirty-item list of treatment-related toxicities for standardized clinical data capture. Additionally, four multi-domain symptom questionnaires were ranked, and definitions on disease control metrics were formalized. These findings have been used to develop a comprehensive operational ontology for PCa care that can facilitate knowledge sharing and scalable machine learning approaches.

Abstract

Background: Clinical data collection related to prostate cancer (PCa) care is often unstructured or heterogeneous among providers, resulting in a high risk for ambiguity in its meaning when sharing or analyzing data. Ontologies, which are shareable formal (i.e., computable) representations of knowledge, can address these challenges by enabling machine-readable semantic interoperability. The purpose of this study was to identify PCa-specific key data elements (KDEs) for standardization in clinic and research. Methods: A modified Delphi method using iterative online surveys was performed to report a consensus agreement on KDEs by a multidisciplinary panel of 39 PCa specialists. Data elements were divided into three themes in PCa and included (1) treatment-related toxicities (TRT), (2) patient-reported outcome measures (PROM), and (3) disease control metrics (DCM). Results: The panel reached consensus on a thirty-item, two-tiered list of KDEs focusing mainly on urinary and rectal symptoms. The Expanded Prostate Cancer Index Composite (EPIC-26) questionnaire was considered most robust for PROM multi-domain monitoring, and granular KDEs were defined for DCM. Conclusions: This expert consensus on PCa-specific KDEs has served as a foundation for a professional society-endorsed, publicly available operational ontology developed by the American Association of Physicists in Medicine (AAPM) Big Data Sub Committee (BDSC).

1. Introduction

Prostate cancer (PCa) is the most diagnosed cancer and the second leading cause of cancer-related deaths among men in the United States, requiring comprehensive multidisciplinary assessment and personalized management [1,2]. Depending on patient preference, comorbidities, and extent of disease at diagnosis, PCa patients may be eligible for continually evolving local therapy options including radical prostatectomy, low-dose or high-dose-rate brachytherapy, and external beam radiotherapy delivered using photons and/or protons [3,4,5,6,7]. Clinicians are strongly encouraged to provide their PCa patients with individualized risk estimates of post-therapy toxicities and PCa recurrence [8], and to clearly communicate cancer management and outcomes with other multidisciplinary providers. However, these critical goals are becoming increasingly difficult to perform given the exponential rates of electronic data growth, inconsistent data reporting, and the use of ambiguous language in clinical documentation. For example, definitions for locoregional recurrence after radiotherapy in PCa can vary among clinicians (i.e., pelvic and/or aortic nodal regions). Even semantic (i.e., meaning) ambiguity in reporting properties of ‘active surveillance’ versus ‘watchful waiting’ in men with localized PCa was identified as a significant issue in another consensus study [9]. Without expert-endorsed identification and standardization of key data elements (KDEs) related to PCa care, the interpretation and reuse of such data (i.e., advanced analytical techniques, such as machine learning approaches) and interoperable data exchange will continue to be significantly hindered.
Lack of disease-specific KDE definitions is a major barrier to improving research and clinical decision-making processes. A systematic review of 17 consensus-forming studies aimed at identifying clinical outcomes for standardization in general cancer research found that nearly 60% of the studies lacked specific recommendations on measuring or defining KDEs, including those related to posttherapy patient-reported symptoms [10]. Already published consensus-forming studies on PCa care and/or reporting guidelines have focused on elements related to subpopulations of PCa patients (i.e., patients with oligometastatic disease) or defined a narrow scope of clinical care metrics without an explicit nomenclature for specifying standardized value sets [11,12,13,14,15]. Even when KDEs are identified, data are often siloed in different information systems or entered in electronic health record (EHR) systems using free text entries which results in lower data quality and higher risks of semantic ambiguity [16]. The extensive heterogeneity existing in electronic data source-based file formatting and data structuring (i.e., schemas) prevents effective search, retrieval, pooling, and analysis of complex biomedical data [17,18,19]. Therefore, there is a significant need for ongoing data standardization of KDEs, preferably expressed through an ontology, for uniform reporting across all PCa patients.
In 1998, Studer et al. defined an ontology as ‘a formal, explicit specification of a shared conceptualization’ [20]. In other words, ontologies are a consensus-based, machine-readable representation of knowledge with explicit definitions for concepts and the relationships between them. Operational ontologies build upon ‘upper level’ ontologies, such as the Open Biological and Biomedical Ontologies (OBO) Foundry [21], and address operational gaps to provide standardized value sets for sub-domains of knowledge. Once implemented, highly expressive ontologies can vastly improve patient care and cancer research endeavors by preserving semantics during information retrieval and knowledge sharing, enabling knowledge inference, and facilitating the creation of large databases for advanced analytics [22,23]. Despite the numerous advantages associated with ontologies, limited work has been published on the translation of expert-guided guidelines for building publicly available operational ontologies.
The primary objective of this study therefore was to reach a multidisciplinary, expert-based consensus on KDEs requiring standardization in PCa care. Herein, we report the results of a modified Delphi procedure used to establish universal PCa-specific KDEs associated with three major concepts of care: (1) treatment-related toxicities (TRTs), (2) patient-reported outcome measures (PROMs), and (3) disease control metrics (DCMs). This study addresses a critical gap in existing knowledge by providing a tiered list of TRTs, a ranking of PROM tools, and explicit definitions for DCMs. Clinical applications and future directions are also provided. To our knowledge, this is the first expert Delphi consensus study in cancer to be directly applied to the development of an operational ontology.

2. Materials and Methods

American Association of Physicists in Medicine—Big Data Sub Committee. The American Association of Physicists in Medicine (AAPM) is a scientific and professional organization with extensive expertise in improving treatment accuracy and patient safety through the development and implementation of consensus-driven information standards, such as the Standardizing Nomenclatures in Radiation Oncology Report (TG-263) [24,25]. Chartered in 2019, AAPM’s Big Data Sub Committee (AAPM BDSC) is a multi-institutional committee composed of a diverse group of stakeholders including physicians, medical physicists, and informaticians who are also representatives of professional organizations, such as the American Society of Radiation Oncology (ASTRO), the European Society of Therapeutic Radiation Oncology (ESTRO), the Canadian Organization of Medical Physicists (COMP), and NRG Oncology [26]. Members of the BDSC have significant experience in data standardization and clinical/research needs assessments, making the BDSC a suitable expert working group to develop the operational ontology for oncology (O3) for PCa. To more accurately build and express core knowledge concepts in the ontology to be used by PCa specialists, the BDSC created the BDSC Prostate Cancer Consortium, otherwise known as the Expert Panel, to participate in this modified Delphi consensus initiative.
Expert Panel. A list of nationally recognized multidisciplinary specialists located in the United States with domain expertise in the management of genitourinary (GU) malignancies (n = 87) was thoughtfully drafted through recommendations from members of the BDSC and ASTRO. Personalized emails were sent to each expert with a two-week response deadline; 48 experts agreed to participate. Throughout the Delphi process, 39 participants (the Expert Panel) completed at least one remote electronic survey. Panel characteristics are reported in Table 1, with diverse representatives from radiation oncology, radiation physics, medical oncology, and urology. All clinical experts were known to have extensive practice in the management of PCa. Members of the Expert Panel and Delphi biostatisticians are listed in Box 1.
Box 1. Members of the BDSC Prostate Cancer Consortium
Delphi Consensus Lead: Amy Moreno (MD Anderson Cancer Center)
Expert Panel: Neeraj Agarwal (University of Utah Huntsman Cancer Institute), Ana Aparicio (MD Anderson Cancer Center), Jeffrey Cadeddu (University of Texas Southwestern Medical Center), Ronald Chen (University of Kansas Medical Center), Seungtaek Choi (MD Anderson Cancer Center), Matthew Cooperberg (University of California San Francisco), Alan Dal Pra (University of Miami Miller School of Medicine), Indra Das (Northwestern University Feinberg School of Medicine), Neil Desai (University of Texas Southwestern Medical Center), Dayssy Diaz Pardo (Ohio State University Comprehensive Care Center), Weiliang Du (MD Anderson Cancer Center), William Hall (Medical College of Wisconsin), Celestia Higano (University of Washington, Fred Hutchison Cancer Research Center), Karen Hoffman (MD Anderson Cancer Center), Maha Hussain (Northwestern University Feinberg School of Medicine), Sophia Kamran (Massachusetts General Hospital Cancer Center, Harvard Medical School), Amar Kishan (University of Cali-fornia Los Angeles), Bridget Koontz (East Carolina University), Rajat Kudchadker (MD Anderson Cancer Center), Charles Mayo (University of Michigan), Jeff Michalski (Washington University School of Medicine), Alicia Morgans (Northwestern University Feinberg School of Medicine), Himanshu Nagar (Weill Cornell Medicine), Louis Potter (Northwell Health, Zucker School of Medicine), Tyler Robin (University of Colorado Denver School of Medicine), Mihaela Rosu-Bubulac (Virginia Commonwealth University), Howard Sandler (Cedars-Sinai), Neal Shore (Atlantic Urology Clinics), Abhishek Solanki (Loyola Medicine), Cora Sternberg (Weill Cornell Medicine), Rahul Tendulkar (Cleveland Clinic), Ying Xiao (University of Pennsylvania, Perelman School of Medicine), James Yu (Trinity Health of New England), Zachary Zumsteg (Cedars-Sinai), and unnamed panelists (n = 5).
Delphi Statisticians: Ruitao Lin (MD Anderson Cancer Center), and Tianlin Xu (MD Anderson Cancer Center)
Modified Delphi Method. We used a fully remote, two-step modified Delphi technique to formulate the expert consensus on KDEs relevant to PCa care. The Delphi method, initially developed in the 1950s, has been widely adopted and adapted for its utility in information gathering and establishing consensus on policies or decision-making processes [27]. Controlled feedback and anonymity are two strengths of this research approach as experts reply anonymously to a series of questionnaires and can adjust their opinions after reviewing interval feedback or insights from the entire group [28]. Unlike traditional Delphi, the modified Delphi is designed to allow researchers to carefully research and create a curated list of initial items or questions for review by the panel to facilitate the first round of the consensus-seeking process [29]. Panel sizes can vary significantly from 3 to over 70 members [28], therefore, it is advised to have at least one representative from each stakeholder group to ensure that substantive sub-domain knowledge is integrated and communicated throughout the entire process. This modified Delphi was approved by The University of Texas MD Anderson Cancer Center Institutional Review Board.
Survey Design and Administration. We performed an informal but comprehensive literature review to develop a list of potentially important data elements related to PCa. Specifically, PubMed was iteratively queried using a variety of combinations of the following keywords and/or MeSH terms: prostate cancer, guidelines, treatment-related toxicities, adverse effects, disease control metrics, disease response assessment, prostate cancer surveillance, patient-reported outcomes, symptom monitoring, PRO tools and/or validated questionnaires. Abstracts were reviewed for relevance and manuscript review was limited to publications related to large PCa randomized clinical trials and/or PCa guidelines endorsed by nationally/internationally recognized organizations, such as the American Urological Association (AUA), ASTRO, the European Society for Therapeutic Radiation and Oncology (ESTRO), and the European Society of Urogenital Radiology (ESUR). Based on the review of the literature and serial BDSC meetings with GU specialists, data elements were categorized into three overarching concepts that included TRTs (n = 43), PROMs (1 urinary, 1 sexual health, 1 erectile dysfunction, and 4 validated multi-domain questionnaires), and DCMs (open-ended questions). Two rounds of electronic surveys were conducted between October 2020 to February 2021. Experts were given approximately 3 weeks to complete the first survey with several automated reminder emails delivered. Analysis of results from the first questionnaire was performed during the holiday-filled months of November–December and included a manual review and categorization of open-ended questions and new ones submitted by the experts. The second survey was deployed in January and contained anonymous group feedback for review and summarization of final KDEs. After completion of the Delphi consensus process, the BDSC began to work on designing an operational ontology based on the Expert Panel’s recommendations.
Treatment-Related Toxicities. During Round 1, experts were asked to rate 43 CTCAE V5.0, toxicities on a 9-point numerical scale (1 = not important to 9 = very important). This scale was intentionally selected based on prior publications demonstrating unique advantages for granular analysis and easy conversion to a 3-point-based scale [30]. During Round 2, panelists were presented with statistical representations of group responses from Round 1 (see Statistical Analysis below). Based on their own interpretations of the group’s insight, each panelist was then tasked with categorizing toxicity metrics into three tiers: (1) a “minimum required” or tier 1 element list, (2) a “strongly encouraged but not mandatory” or tier 2 list, and (3) a “not required” list. The first tier was defined as a list of critical data elements that should be mandatory for standardized reporting on a national level to ensure equivalent high-quality data exchange and analysis for all PCa patients. The second-tier list was intentionally defined more broadly to include additional metrics that were recommended by the panel for standardized collection, if possible, in an effort to facilitate the development of a highly expressive operational ontology by the BDSC. The third list included all the remaining toxicities that the experts considered to be of least importance to prevent fatigue.
Patient-Reported Outcome Metrics. After answering questions about TRTs during the first survey, experts were asked to score the following validated PROM tools: the International Prostate Symptom Score (IPSS, for evaluation of severity of urinary symptoms and impact on quality of life) [31] the International Index of Erectile Function (IIEF, for evaluation of erectile dysfunction) [32] and the Sexual Health Inventory for Men (SHIM, for evaluation of sexual health and erectile dysfunction) [33]. Additionally, four commonly used PCa-specific, multi-domain PROM questionnaires were reviewed for their usage and ability to adequately assess urinary, sexual, bowel, and systemic functions as well as quality of life. These included the Expanded Prostate Index Composite-26 (EPIC-26), the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Prostate Cancer module (EORTC QLQ-PR25), the Functional Assessment for Cancer Therapy Prostate Cancer module (FACT-P), and the Patient-Reported Outcomes Common Terminology Criteria for Adverse Events (PRO-CTCAE) [34,35]. Since consensus was achieved on the PROM questionnaire rankings during Round 1, no further related questions were asked during the second survey.
Disease Control Metrics. Potential DCM metrics, which are numerous and have greater variability in existing definitions (i.e., the definition of biochemical failure and/or oligometastatic PCa), were open to discussion and item list generation during Round 1. DCM-related questions were intentionally open-ended for comprehensive information gathering from the experts and for qualitative analysis. During Round 2, further refinement of the definitions for biochemical recurrence and oligometastatic disease, as well as the development of a list of recurrent-specific KDEs for standardized reporting was completed by the panel. Figure 1 shows the overall study process flowchart for consensus formation on PCa-related KDEs with translation to create O3.
Statistical Analysis. Descriptive statistics, including central tendencies (i.e., means), counts, and percentages, were performed to describe Expert Panel characteristics and KDE variables according to their data types. Variables that were scored from 1 to 9 during Round 1 were additionally categorized into three agreement levels during Round 2. Scores ranging from 7 to 9 were coded as ‘high’ in agreement, 4 to 6 as ‘medium’, and 1 to 3 as ‘low’ in agreement. Individual item-based agreement indices (AI) were also calculated after each round to aid in statistical ranking. Agreement indices measure the percentage of equal ratings for the question, I (in round t), over a specific number of panelists who responded to the survey, where 0 < AI < 1 (0 denotes total disagreement, and 1 denotes consensus) [36]. To reflect the correlation within each question in Round 2, the intraclass correlation coefficient (ICC) was calculated using a linear mixed model with random effects for TRT-related questions. The ICC was interpreted as follows: excellent (>0.75), fair to good (0.40–0.75), and poor (<0.40) [37]. The chi-square test or Fisher’s exact test (when any category had <5 observations) was used to test whether there was a difference in the panel categorization of TRT items between Rounds 1 and 2. Consensus was considered to be reached when percent scoring and AI per question reached above 50%. Metrics and thresholds for indicating consensus are highly flexible in Delphi studies and include meeting a percent threshold, using standard deviations, or interquartile range constraints [28,38,39]. We selected a lower agreement percentage threshold in combination with the agreement index to account for variability in panel response size per round and to allow for greater inclusion of data elements into our second-tier TRT list. This consensus process was communicated with the Expert Panel and both the panel and BDSC had to review and approve the consensus recommendations on PCa-specific KDEs listed herein to proceed with the development of the operational ontology. Calculations were performed using the R and package lme4 [40].

3. Results

3.1. Panel Characteristics and Participation

Characteristics of the multidisciplinary Expert Panel are presented in Table 1. Two-thirds of the participants were male, 56% were radiation oncologists, and 79% reported practicing in an academic setting. The average number of years in practice and weekly patient caseloads were 18 years and 30 patients, respectively. Of 39 panelists who completed at least one survey, 34 (87%) completed Round 1, 26 (67%) completed Round 2, and 25 (64%) completed both surveys.

3.2. Treatment-Related Toxicity KDEs

Forty-three TRTs were reviewed and scored by the experts, including items related to rectal (i.e., rectal hemorrhage, and rectal fistula), urinary (i.e., urinary incontinence, and urinary frequency), hormonal (i.e., hot flashes, and libido decrease), and constitutional (i.e., fatigue) symptoms. Consensus on the first-tier (minimum KDE), second-tier (strongly encouraged), and third-tier (not required) lists are shown in Table 2 along with corresponding agreement indices. The first tier consisted of 15 toxicities that achieved ≥50% selection for this list and ≥50% calculated agreement index. The second tier included another 15 variables, of which 10 (67%) had ≥50% selection for this list with an agreement index range of 0.35 to 0.49. The remaining five variables met one of the above requirements or were added to this list owing to exclusion from the “not required” list. Twelve remaining TRTs were ranked as low yield for reporting. No additional TRT-related items were recommended for review by the panel.
In general, the panel demonstrated high rates of agreement (i.e., high AI) when reviewing TRTs that were of very high or low importance. Those with higher AI variability (i.e., greater disagreement among panelists) tended to be categorized in the second-tier list. The ICC for TRTs was 0.58, and the chi-square/Fisher’s exact test showed a significant difference in panelist answers between both rounds of surveys for 23 out of the 43 TRTs.

3.3. Patient-Reported Outcome Metrics KDEs

The percentage of panelists reporting ‘almost always’ use of the IPSS, SHIM, and IIEF questionnaires was 56%, 29%, and 6%, respectively. When asked whether panelists preferred the SHIM or IIEF tool, 53% preferred SHIM, 32% preferred IIEF, and 15% preferred neither. Panelists who reported “always” using the EPIC-26, EORTC QLQ-PR25, FACT-P, and PRO-CTCAE multi-domain questionnaires represented 24%, 6%, 0%, and 0% of the group, respectively. Nearly two-thirds of the panel reported never using the FACT-P and PRO-CTCAE tools. Overall, EPIC-26 scored the highest in all domains with an average multi-domain score of 7.5 out of 9 and an agreement index of 0.77 (Figure 2). This was consistent with the panel’s final ranking of the PROM tools: EPIC-26 (highest ranking), EORTC QLQ-PR25, FACT-P, and PRO-CTCAE (lowest ranking).

3.4. Disease Control Metrics KDEs

The panel offered several recommendations for reporting locoregional recurrences. They included a high preference (61%) for reporting detailed documentation on primary site recurrences (i.e., localized in the prostate, seminal vesicles, or prostate bed) and granular documentation on nodal recurrences (i.e., reporting recurrences as “none”, “pelvic”, “aortic”, and/or “distant”). After reviewing the open-ended questions from Round 1, panelists were presented with a list of DCM options during Round 2. Overall ratings and associated agreement indices are shown in Table 3. Using a similar tier categorization as performed for TRTs, 11 out of 14 disease status elements were identified as DCM-specific KDEs by the multidisciplinary panel. “Stable disease”, “Indeterminate (possible pseudo-progression)”, and “Partial Response” were not prioritized as optimal DCMs due to their vague definitions in comparison to the prioritized terms.
Regarding the extent of metastatic disease, several definitions of metastatic burden were reviewed, including the CHAARTED and LATITUDE criteria from randomized clinical trials [41,42]. Based on clinical utility, the panel recommended standardizing the reporting of osseous metastatic lesions using a numerical value up to five and reporting visceral (i.e., organ) metastatic lesions in a binary format (i.e., either present or absent).

4. Discussion

This Delphi method highlights ongoing and significant heterogeneity in how providers manage and report clinical data on PCa patients. By the end of the study, 15 TRTs were identified by the Expert Panel as having high importance in assessing outcomes related to PCa (tier 1), and another 15 were considered to be of moderate value (tier 2) for enhancing the expressivity of a PCa-specific operational ontology [43]. This component of the study is unique as it provides a statistically tiered list approach for meaningful KDEs that are provider-based (i.e., objective) in addition to the corresponding patient-reported symptoms during and after therapy (i.e., subjective). The prioritized lists inform resource investment directions within organizations and allow for greater standardization and expressivity in an operational ontology.

4.1. Recommended Treatment-Related Toxicity Reporting

Panel recommendations for key TRTs to report focused mainly on urinary and bowel-related symptoms. Rectal hemorrhage, urinary incontinence, and urinary retention, all of which had the highest agreement indices, are known to be common acute and chronic treatment-related complications after radiation therapy or surgery [43]. For patients treated with radiotherapy, there is substantial evidence to support the radiation dose-volume effects in radiation-induced rectal and/or urinary injury [44,45]. This has led to the development of treatment planning guidelines with dose volume constraints, such as those outlined by QUANTEC, and the investigation of endorectal balloons and rectal spacers as methods to further reduce risks of rectal injury [45,46,47,48]. Moreover, large randomized clinical trials investigating the efficacy of different regimens (i.e., hypofractionation and/or stereotactic body radiation therapy) also serve as additional references to clinicians to mitigate TRT risks using schedule-dependent dose-volume constraints [3,4,49,50]. Prospective and standardized reporting of TRT-KDEs in clinical practice is critical to validate the long-term outcomes of therapy beyond the time window reported in clinical trials. One method of supporting such data collection is through the use of ‘smart forms’ or structured clinical note templates in the patient’s electronic health record chart. Problem-oriented templates provide several advantages including shareability, defining value sets per data field, and increasing note quality without increasing total charting times [51]. Smart templates can also support scalable and automated data extraction which reduces time and labor costs associated with manual chart data extraction.

4.2. Recommended Patient-Reported Outcome/Symptom Surveying Tools

Of the four validated multi-domain PROM symptom tools, EPIC-26 was prioritized given its ability to capture and track longitudinal, and comprehensive data related to urinary, sexual, bowel, systemic, and quality of life domains. The panel’s selection of the EPIC-26 tool to assess symptom profiles is in alignment with recommendations made by Martin et al. in 2015 who endorsed EPIC-26 as a standard set of PROMs for men with localized prostate cancer [52]. It is also commonly used in large studies, such as the ProtecT Study Group who valued its assessment of the four domains and QOL in PCa patients managed with surveillance, surgery or radiotherapy. However, its adoption in clinical practice is likely low as reported by the Expert Panel (24%), thereby highlighting an ongoing challenge in the ability of organizations/hospitals to address the barriers to longitudinal PRO monitoring implementation [53]. Similar approaches to develop smart forms for TRTs in electronic health record systems can be considered for recreating PROM tools in a structured manner. An alternative approach is the development and delivery of automated surveys to consenting patients through electronic data capture systems, such as REDCap®. The COVID-19 pandemic resulted in an upsurge of REDCap® surveys being remotely distributed to patients to support COVID-19 testing and aggregate reporting [54]. These promising results present new opportunities to facilitate structured collection of PCa-specific PROMs across various healthcare settings.

4.3. Recommended Reporting of Disease Control and Response Metrics

Regarding disease control and response metrics, general definitions for survival (i.e., overall, cancer-specific, metastasis-free, and biochemical recurrence-free survival) in localized prostate cancer have been recommended by other consensus studies without explicit definitions on how to report data related to the extent of disease recurrence, if present [52]. To address this issue, there are continued efforts in specialties, such as radiology, to develop imaging reporting risk stratification systems to better inform clinicians of the disease status of patients after their anatomy has been altered post local therapy [55]. As reported in Table 3, our panel recommended further expansion of value sets related to anatomical sites involved in recurrence patterns (i.e., primary, pelvic nodal, and distant sites). From a comprehensive operational ontology standpoint, this recommendation can be translated into several DCM-based KDEs, such as primary_site_recurrence (i.e., with values including combinations of “none”, “prostate/bed”, or “seminal vesicles”) and nodal_site_recurrence (i.e., values including “none”, “pelvic”, “para-aortic”, and “distant”). Extent of non-nodal metastatic disease can also be classified as a numerical representation of osseous metastasis, with or without the presence of visceral lesions. These elements of metastatic disease have already been incorporated into trial-based criteria [41,42]. However, variations in the definitions of low- versus high-risk/volume metastatic disease present a challenge in comparing patient cohorts and the efficacy of evolving therapies (i.e., the “high volume” definition in CHAARTED includes four or more osseous metastases while LATITUDE defines “high risk” as having two or more factors that can include three or more osseous metastases). A common numerical threshold for defining oligometastatic/oligorecurrent PCa in recent randomized trials is the maximum presence of three to five involved sites (NCT04787744, NCT04115007, NCT03784755, NCT02759783, NCT03630666, NCT03569241, NCT04031378, NCT03940235, and NCT04037358). Therefore, in alignment with our panel consensus, we recommend that the extent of nodal and/or osseous lesions be reported in a numerical format in clinical records with the option of reaching a particular threshold of five lesions or more.

4.4. Study Limitations

Our Delphi study has inherent risks of bias including response bias (i.e., who decided to participate in this study), attrition bias (i.e., attrition in panelist participation during the second survey), and cognitive bias (i.e., framing and anchoring, and the bandwagon effect). We aimed to mitigate these biases by expanding the initial multidisciplinary invitation list, preserving anonymity of participants during structured group interactions, and asking panelists to share additional qualitative comments that could be used to support or oppose arguments for KDE selection. Despite such bias risks, the Delphi method is accepted as a useful technique for information collection and knowledge building on “informed opinion and subjective expert judgments as well as experienced-based interpretations” [38,56]. Another potential limitation includes our use of more relaxed definitions for reaching consensus (i.e., 50% or more). While lower thresholds of agreement are generally not seen in other Delphi studies, the authors felt it was appropriate for the overarching goal of this consensus procedure which was to provide multidisciplinary stakeholder input on KDEs to be included in our operational ontology for PCa. Additionally, the panel had a high representation of radiation oncologists compared to other specialties. This unbalanced distribution of specialists was intentional to optimize the O3 development phase and is counterbalanced by the allowance of more non-radiation oncology specialists on the panel due to the feasible, remote and electronic nature of the Delphi method. Lack of dynamic in-person conversations, variable panelist experiences with managing PCa patients, and restriction of Delphi rounds could have also affected the data elements that were brought up for review and ultimately included in the KDE list. Moreover, this study does not offer recommendations on radiation treatment doses to reduce the risk of TRTs as this was out of the current scope, and radiation schedules are being continuously investigated for efficacy and toxicity effects in several randomized clinical trials [50]. However, O3 currently supports the standardized collection of granular radiotherapy data (i.e., modality, daily dose, and fractionation) in addition to capturing patient and cancer characteristics, TRTs, DCMs, and PROMs as recommended by the BDSC and Expert Panel. This ensures that while treatments and technology evolve, comprehensive capture of core data elements using structured and routinely updated fields remains standardized across institutions. Overall, our work is synergistic with other ongoing efforts to define data elements in prostate cancer care for the construction of analytical pipelines and improved health information sharing [57].

4.5. Operational Ontology Build and Future Directions

Translation to Operational Ontology Build. Effective use and exchange of information in PCa care and research requires identification and standardization of data elements that are meaningful and comprehensive (i.e., pertaining to multiple domains of care including diagnosis, staging, toxicity reporting, and major clinical outcomes). The multidisciplinary Delphi consensus procedure resulted in a detailed report of KDEs related to TRTs, PROMs, and DCMs. Once provided with an endorsed KDE value set, the BDSC, in partnership with multiple external stakeholders, began the expansion of an operational ontology for oncology (O3) with the latest version available online and with links to download the complete set as a spreadsheet or in JavaScript Object Notation (JSON) format for electronic processing [58]. Figure 3 illustrates the O3 website for PCa-specific attributes including designated priority levels as recommended by the Expert Panel (i.e., red box for rectal hemorrhage listed as Priority 1). When feasible, O3 attributes, including those specific to PCa, were mapped to attributes found in other coding systems, such as SNOMED-CT to facilitate interoperability among those systems [59]. Technical details on the BDSC’s iterative process for developing O3 and affiliated consensus-driven information standards are reported by Mayo et al. [60].
To our knowledge, this is the first study of its kind to immediately translate expert-based recommendations from a Delphi study into technical efforts in operational ontology development. Future directions include the use of this existing Delphi-to-ontology protocol by the BDSC to update clinical guidelines and O3 with new knowledge and to promote the identification of KDEs and expansion of O3 related to other disease sites (i.e., head and neck, thoracic, etc). In clinical practice, implementation of recommended symptom monitoring tools and prospective KDE collection can be associated with several barriers including workflow challenges, survey fatigue and limitations in technology [61,62]. Therefore, implementation strategies and studies should be encouraged to ensure that PCa-specific KDEs are consistently captured with high data quality. Formative evaluations are especially helpful for clinical implementation endeavors as they are rigorous, multi-phase (i.e., developmental [pre-implementation interviews], implementation-focused, progress-focused, and interpretive [post-implementation surveys/interviews]) assessment processes that identify facilitators and barriers to change as well as potential solutions for optimizing the desired change [60,61]. Dissemination of successful strategies in various settings (i.e., academic centers, and private practice) would be ideal to facilitate KDE capture at a national level. O3 will also require routine updates by the BDSC, especially as it begins to be adopted by multidisciplinary cancer teams. The overarching goal is to see parallel adoption of prospective KDE collection and use of O3 over time in order to enable the construction and sharing of multi-institutional comprehensive, high-quality “real world” datasets that support robust clinical practice and research applications.

5. Conclusions

Through the application of a modified Delphi technique, a multidisciplinary team of prostate cancer experts was able to develop recommendations for a key dataset of treatment-related toxicities, patient-reported outcome metrics, and disease control metrics. These recommendations are ideal for all clinical practices and research and were successfully translated into building an endorsed, web-based operational ontology that can facilitate scalable and accurate information retrieval and exchange related to cancer care.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers15123121/s1.

Author Contributions

Conceptualization, A.M., A.A.S., J.P., E.D., D.H., J.H., S.C.K., E.K., K.B., M.F., C.F. and C.M.; methodology, A.M., C.F., C.M., T.X. and R.L.; formal analysis, A.M., T.X. and R.L.; data collection, all authors including members of the BDSC Prostate Cancer Consortium; writing—original draft preparation, all authors; writing—review and editing, all authors. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of The University of MD Anderson Cancer Center (protocol code 2020-1096, date of approval: 7 September 2021).

Informed Consent Statement

Informed consent was obtained from all subjects (i.e., panelists) involved in the study.

Data Availability Statement

Data will be made available upon reasonable request to the corresponding author.

Acknowledgments

We would like to acknowledge all members of the AAPM BDSC for their efforts in developing the operational ontology for oncology (O3) with its first case application in the PCa domain.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

AAPMAmerican Association of Physicists in Medicine
AIAgreement Index
BDSCBig Data Sub Committee
DCMDisease Control Metrics
EORTC QLQ-PR25European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Prostate Cancer
EPIC-26Expanded Prostate Index Composite-26
FACT-PFunctional Assessment for Cancer Therapy- Prostate Cancer
GUGenitourinary
ICCInterclass Correlation Coefficient
KDEKey Data Element
O3Operational Ontology for Oncology
PCaProstate Cancer
PRO-CTCAEPatient-Reported Outcomes Common Terminology Criteria for Adverse Events
PROMPatient-Reported Outcome Measure
TRTTreatment-Related Toxicity

References

  1. Siegel, R.L.; Miller, K.D.; Fuchs, H.E.; Jemal, A. Cancer Statistics, 2021. CA Cancer J. Clin. 2021, 71, 7–33. [Google Scholar] [CrossRef] [PubMed]
  2. Henley, S.J.; Ward, E.M.; Scott, S.; Ma, J.; Anderson, R.N.; Firth, A.U.; Thomas, C.C.; Islami, F.; Weir, H.K.; Lewis, D.R.; et al. Annual Report to the Nation on the Status of Cancer, Part I: National Cancer Statistics. Cancer 2020, 126, 2225–2249. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Catton, C.N.; Lukka, H.; Gu, C.S.; Martin, J.M.; Supiot, S.; Chung, P.W.M.; Bauman, G.S.; Bahary, J.P.; Ahmed, S.; Cheung, P.; et al. Randomized Trial of a Hypofractionated Radiation Regimen for the Treatment of Localized Prostate Cancer. J. Clin. Oncol. 2017, 35, 1884–1890. [Google Scholar] [CrossRef] [PubMed]
  4. Dearnaley, D.; Syndikus, I.; Mossop, H.; Khoo, V.; Birtle, A.; Bloomfield, D.; Graham, J.; Kirkbride, P.; Logue, J.; Malik, Z.; et al. Conventional versus Hypofractionated High-Dose Intensity-Modulated Radiotherapy for Prostate Cancer: 5-Year Outcomes of the Randomised, Non-Inferiority, Phase 3 CHHiP Trial. Lancet Oncol. 2016, 17, 1047–1060. [Google Scholar] [CrossRef] [Green Version]
  5. Pollack, A.; Zagars, G.K.; Starkschall, G.; Antolak, J.A.; Lee, J.J.; Huang, E.; Von Eschenbach, A.C.; Kuban, D.A.; Rosen, I. Prostate Cancer Radiation Dose Response: Results of the M. D. Anderson Phase III Randomized Trial. Int. J. Radiat. Oncol. Biol. Phys. 2002, 53, 1097–1105. [Google Scholar] [CrossRef]
  6. Litwin, M.S.; Tan, H.J. The Diagnosis and Treatment of Prostate Cancer: A Review. JAMA J. Am. Med. Assoc. 2017, 317, 2532–2542. [Google Scholar] [CrossRef]
  7. Hamdy, F.C.; Donovan, J.L.; Lane, J.A.; Mason, M.; Metcalfe, C.; Holding, P.; Davis, M.; Peters, T.J.; Turner, E.L.; Martin, R.M.; et al. 10-Year Outcomes after Monitoring, Surgery, or Radiotherapy for Localized Prostate Cancer. N. Engl. J. Med. 2016, 375, 1415–1424. [Google Scholar] [CrossRef] [Green Version]
  8. Eastham, J.A.; Auffenberg, G.B.; Barocas, D.A.; Chou, R.; Crispino, T.; Davis, J.W.; Eggener, S.; Horwitz, E.M.; Kane, C.J.; Kirkby, E.; et al. Clinically Localized Prostate Cancer: AUA/ASTRO Guideline, Part I: Introduction, Risk Assessment, Staging, and Risk-Based Management. J. Urol. 2022, 208, 10–18. [Google Scholar] [CrossRef]
  9. Bruinsma, S.M.; Roobol, M.J.; Carroll, P.R.; Klotz, L.; Pickles, T.; Moore, C.M.; Gnanapragasam, V.J.; Villers, A.; Rannikko, A.; Valdagni, R.; et al. Semantics in Active Surveillance for Men with Localized Prostate Cancer—Results of a Modified Delphi Consensus Procedure. Nat. Rev. Urol. 2017, 14, 312–322. [Google Scholar] [CrossRef] [Green Version]
  10. Ramsey, I.; Eckert, M.; Hutchinson, A.D.; Marker, J.; Corsini, N. Core Outcome Sets in Cancer and Their Approaches to Identifying and Selecting Patient-Reported Outcome Measures: A Systematic Review. J. Patient Rep. Outcomes 2020, 4, 77. [Google Scholar] [CrossRef]
  11. Gillessen, S.; Attard, G.; Beer, T.M.; Beltran, H.; Bjartell, A.; Bossi, A.; Briganti, A.; Bristow, R.G.; Chi, K.N.; Clarke, N.; et al. Management of Patients with Advanced Prostate Cancer: Report of the Advanced Prostate Cancer Consensus Conference 2019. Eur. Urol. 2020, 77, 508–547. [Google Scholar] [CrossRef]
  12. Aluwini, S.S.; Mehra, N.; Lolkema, M.P.; Oprea-Lager, D.E.; Yakar, D.; Stoevelaar, H.; van der Poel, H.; Busstra, M.; de Jong, I.J.; de Reijke, T.; et al. Oligometastatic Prostate Cancer: Results of a Dutch Multidisciplinary Consensus Meeting. Eur. Urol. Oncol. 2019, 3, 231–238. [Google Scholar] [CrossRef]
  13. Ingrosso, G.; Becherini, C.; Lancia, A.; Caini, S.; Ost, P.; Francolini, G.; Høyer, M.; Bottero, M.; Bossi, A.; Zilli, T.; et al. Nonsurgical Salvage Local Therapies for Radiorecurrent Prostate Cancer: A Systematic Review and Meta-Analysis. Eur. Urol. Oncol. 2020, 3, 183–197. [Google Scholar] [CrossRef]
  14. Gómez-Caamaño, A.; González-San Segundo, C.; Henríquez, I.; Maldonado, X.; Zapatero, A.; Cabeza Rodríguez, M.A.; Celada Álvarez, F.; Ferrer Albiach, C.; Ferrer González, F.; Gómez-Iturriaga, A.; et al. Consensus on Management of Castration-Resistant Prostate Cancer on Behalf of the Urological Tumours Working Group (URONCOR) of the Spanish Society of Radiation Oncology. Clin. Transl. Oncol. 2019, 21, 420–432. [Google Scholar] [CrossRef] [Green Version]
  15. Dunn, J.; Green, A.; Ralph, N.; Newton, R.U.; Kneebone, A.; Frydenberg, M.; Chambers, S.K. Prostate Cancer Survivorship Essentials Framework: Guidelines for Practitioners. BJU Int. 2020, 128, 18–29. [Google Scholar] [CrossRef]
  16. Hogan, W.R.; Wagner, M.M. Free-Text Fields Change the Meaning of Coded Data. In Proceedings of the AMIA Annual Fall Symposium; American Medical Informatics Association: Bethesda, MA, USA, 1996; p. 517, PMCID: PMC2233180. [Google Scholar] [PubMed]
  17. Min, H.; Manion, F.J.; Goralczyk, E.; Wong, Y.N.; Ross, E.; Beck, J.R. Integration of Prostate Cancer Clinical Data Using an Ontology. J. Biomed. Inform. 2009, 42, 1035–1045. [Google Scholar] [CrossRef] [Green Version]
  18. Chen, Y.; Yu, C.; Liu, X.; Xi, T.; Xu, G.; Sun, Y.; Zhu, F.; Shen, B. PCLiON: An Ontology for Data Standardization and Sharing of Prostate Cancer Associated Lifestyles. Int. J. Med. Inform. 2021, 145, 104332. [Google Scholar] [CrossRef]
  19. Zhang, H.; Guo, Y.; Li, Q.; George, T.J.; Shenkman, E.; Modave, F.; Bian, J. An Ontology-Guided Semantic Data Integration Framework to Support Integrative Data Analysis of Cancer Survival. BMC Med. Inform. Decis. Mak. 2018, 18, 129–147. [Google Scholar] [CrossRef]
  20. Studer, R.; Benjamins, V.R.; Fensel, D. Knowledge Engineering: Principles and Methods. Data Knowl. Eng. 1998, 25, 161–197. [Google Scholar] [CrossRef] [Green Version]
  21. Jackson, R.; Matentzoglu, N.; Overton, J.A.; Vita, R.; Balhoff, J.P.; Buttigieg, P.L.; Carbon, S.; Courtot, M.; Diehl, A.D.; Dooley, D.M.; et al. OBO Foundry in 2021: Operationalizing Open Data Principles to Evaluate Ontologies. Database 2021, 2021, baab069. [Google Scholar] [CrossRef] [PubMed]
  22. Sahoo, S.S.; Kobow, K.; Zhang, J.; Buchhalter, J.; Dayyani, M.; Upadhyaya, D.P.; Prantzalos, K.; Bhattacharjee, M.; Blumcke, I.; Wiebe, S.; et al. Ontology-Based Feature Engineering in Machine Learning Workflows for Heterogeneous Epilepsy Patient Records. Sci. Rep. 2022, 12, 19430. [Google Scholar] [CrossRef] [PubMed]
  23. Munir, K.; Sheraz Anjum, M. The Use of Ontologies for Effective Knowledge Modelling and Information Retrieval. Appl. Comput. Inform. 2018, 14, 116–126. [Google Scholar] [CrossRef]
  24. AAPM Task Group. Standardizing Nomenclatures in Radiation Oncology the Report of AAPM Task Group 263; AAPM: Alexandria, VA, USA, 2018. [Google Scholar]
  25. AAPM: The American Association of Physicists in Medicine. Available online: https://www.aapm.org/default.asp (accessed on 31 January 2023).
  26. AAPM Committee Tree—Big Data Subcommittee (BDS). Available online: https://www.aapm.org/org/structure/?committee_code=BDS (accessed on 31 January 2023).
  27. Delphi Method|RAND. Available online: https://www.rand.org/topics/delphi-method.html (accessed on 1 August 2021).
  28. Ogbeifun, E.; Agwa-Ejon, J.; Mbohwa, C.; Pretorius, J.H. The Delphi Technique: A Credible Research Methodology. In Proceedings of the 2016 International Conference on Industrial Engineering and Operations Management, Kuala Lumpur, Malaysia, 8–10 March 2016. [Google Scholar]
  29. Avella, J.R. Delphi Panels: Research Design, Procedures, Advantages, and Challenges. Int. J. Dr. Stud. 2016, 11, 305–321. [Google Scholar] [CrossRef] [Green Version]
  30. Lange, T.; Kopkow, C.; Lützner, J.; Günther, K.P.; Gravius, S.; Scharf, H.P.; Stöve, J.; Wagner, R.; Schmitt, J. Comparison of Different Rating Scales for the Use in Delphi Studies: Different Scales Lead to Different Consensus and Show Different Test-Retest Reliability. BMC Med. Res. Methodol. 2020, 20, 28. [Google Scholar] [CrossRef] [Green Version]
  31. Li, X.; Fang, D.; Cooperberg, M.R.; Whitson, J.M.; Lue, T.F.; Zhou, L.; Shinohara, K. Long-Term Follow-up of International Prostate Symptom Score (IPSS) in Men Following Prostate Brachytherapy. World J. Urol. 2014, 32, 1061–1066. [Google Scholar] [CrossRef] [Green Version]
  32. Rosen, R.C.; Riley, A.; Wagner, G.; Osterloh, I.H.; Kirkpatrick, J.; Mishra, A. The International Index of Erectile Function (IIEF): A Multidimensional Scale for Assessment of Erectile Dysfunction. Urology 1997, 49, 822–830. [Google Scholar] [CrossRef] [Green Version]
  33. Cappelleri, J.C.; Rosen, R.C. The Sexual Health Inventory for Men (SHIM): A 5-Year Review of Research and Clinical Experience. Int. J. Impot. Res. 2005, 17, 307–319. [Google Scholar] [CrossRef] [Green Version]
  34. Mercieca-Bebber, R.; Stockler, M.R. Patient-Reported Outcomes and Localized Prostate Cancer Management. Nat. Rev. Urol. 2020, 17, 257–258. [Google Scholar] [CrossRef]
  35. Dueck, A.C.; Scher, H.I.; Bennett, A.V.; Mazza, G.L.; Thanarajasingam, G.; Schwab, G.; Weitzman, A.L.; Rogak, L.J.; Basch, E. Assessment of Adverse Events from the Patient Perspective in a Phase 3 Metastatic Castration-Resistant Prostate Cancer Clinical Trial. JAMA Oncol. 2020, 6, e193332. [Google Scholar] [CrossRef]
  36. Lee, J.; Koh, D.; Ong, C.N. Statistical Evaluation of Agreement between Two Methods for Measuring a Quantitative Variable. Comput. Biol. Med. 1989, 19, 61–70. [Google Scholar] [CrossRef]
  37. Liljequist, D.; Elfving, B.; Roaldsen, K.S. Intraclass Correlation—A Discussion and Demonstration of Basic Features. PLoS ONE 2019, 14, e0219854. [Google Scholar] [CrossRef] [Green Version]
  38. Beiderbeck, D.; Frevel, N.; von der Gracht, H.A.; Schmidt, S.L.; Schweitzer, V.M. Preparing, Conducting, and Analyzing Delphi Surveys: Cross-Disciplinary Practices, New Directions, and Advancements. MethodsX 2021, 8, 101401. [Google Scholar] [CrossRef]
  39. Hasson, F.; Keeney, S.; McKenna, H. Research Guidelines for the Delphi Survey Technique. J. Adv. Nurs. 2000, 32, 1008–1015. [Google Scholar] [CrossRef] [Green Version]
  40. Package “Lme4.” 2022. Available online: https://cran.r-project.org/web/packages/lme4/lme4.pdf (accessed on 30 May 2023).
  41. Parker, C.C.; James, N.D.; Brawley, C.D.; Clarke, N.W.; Hoyle, A.P.; Ali, A.; Ritchie, A.W.S.; Attard, G.; Chowdhury, S.; Cross, W.; et al. Radiotherapy to the Primary Tumour for Newly Diagnosed, Metastatic Prostate Cancer (STAMPEDE): A Randomised Controlled Phase 3 Trial. Lancet 2018, 392, 2353–2366. [Google Scholar] [CrossRef] [Green Version]
  42. Fizazi, K.; Tran, N.; Fein, L.; Matsubara, N.; Rodriguez-Antolin, A.; Alekseev, B.Y.; Özgüroğlu, M.; Ye, D.; Feyerabend, S.; Protheroe, A.; et al. Abiraterone plus Prednisone in Metastatic, Castration-Sensitive Prostate Cancer. N. Engl. J. Med. 2017, 377, 352–360. [Google Scholar] [CrossRef]
  43. Hoffman, K.E.; Penson, D.F.; Zhao, Z.; Huang, L.-C.; Conwill, R.; Laviana, A.A.; Joyce, D.D.; Luckenbaugh, A.N.; Goodman, M.; Hamilton, A.S.; et al. Patient-Reported Outcomes Through 5 Years for Active Surveillance, Surgery, Brachytherapy, or External Beam Radiation with or Without Androgen Deprivation Therapy for Localized Prostate Cancer. JAMA 2020, 323, 149–163. [Google Scholar] [CrossRef]
  44. Gulliford, S.L.; Foo, K.; Morgan, R.C.; Aird, E.G.; Bidmead, A.M.; Critchley, H.; Evans, P.M.; Gianolini, S.; Mayles, W.P.; Moore, A.R.; et al. Dose-Volume Constraints to Reduce Rectal Side Effects from Prostate Radiotherapy: Evidence from MRC RT01 Trial ISRCTN 47772397. Int. J. Radiat. Oncol. Biol. Phys. 2010, 76, 747–754. [Google Scholar] [CrossRef]
  45. Michalski, J.M.; Gay, H.; Jackson, A.; Tucker, S.L.; Deasy, J.O. Radiation Dose-Volume Effects in Radiation-Induced Rectal Injury. Int. J. Radiat. Oncol. Biol. Phys. 2010, 76, S123–S129. [Google Scholar] [CrossRef] [Green Version]
  46. Prada, P.J.; Gonzalez, H.; Menéndez, C.; Llaneza, A.; Fernández, J.; Santamarta, E.; Ricarte, P.P. Transperineal Injection of Hyaluronic Acid in the Anterior Perirectal Fat to Decrease Rectal Toxicity from Radiation Delivered with Low-Dose-Rate Brachytherapy for Prostate Cancer Patients. Brachytherapy 2009, 8, 210–217. [Google Scholar] [CrossRef]
  47. Smeenk, R.J.; van Lin, E.N.J.T.; van Kollenburg, P.; Kunze-Busch, M.; Kaanders, J.H.A.M. Anal Wall Sparing Effect of an Endorectal Balloon in 3D Conformal and Intensity-Modulated Prostate Radiotherapy. Radiother. Oncol. 2009, 93, 131–136. [Google Scholar] [CrossRef] [PubMed]
  48. Van Lin, E.N.J.T.; Hoffmann, A.L.; Van Kollenburg, P.; Leer, J.W.; Visser, A.G. Rectal Wall Sparing Effect of Three Different Endorectal Balloons in 3D Conformal and IMRT Prostate Radiotherapy. Int. J. Radiat. Oncol. Biol. Phys. 2005, 63, 565–576. [Google Scholar] [CrossRef] [PubMed]
  49. Hoffman, K.E.; Voong, K.R.; Pugh, T.J.; Skinner, H.; Levy, L.B.; Takiar, V.; Choi, S.; Du, W.; Frank, S.J.; Johnson, J.; et al. Risk of Late Toxicity in Men Receiving Dose-Escalated Hypofractionated Intensity Modulated Prostate Radiation Therapy: Results from a Randomized Trial. Int. J. Radiat. Oncol. Biol. Phys. 2014, 88, 1074–1084. [Google Scholar] [CrossRef] [PubMed]
  50. Tree, A.C.; Ostler, P.; van der Voet, H.; Chu, W.; Loblaw, A.; Ford, D.; Tolan, S.; Jain, S.; Martin, A.; Staffurth, J.; et al. Intensity-Modulated Radiotherapy versus Stereotactic Body Radiotherapy for Prostate Cancer (PACE-B): 2-Year Toxicity Results from an Open-Label, Randomised, Phase 3, Non-Inferiority Trial. Lancet Oncol. 2022, 23, 1308–1320. [Google Scholar] [CrossRef]
  51. Mehta, R.; Radhakrishnan, N.S.; Warring, C.D.; Jain, A.; Fuentes, J.; Dolganiuc, A.; Lourdes, L.S.; Busigin, J.; Leverence, R.R. The Use of Evidence-Based, Problem-Oriented Templates as a Clinical Decision Support in an Inpatient Electronic Health Record System. Appl. Clin. Inform. 2016, 7, 790. [Google Scholar] [CrossRef] [Green Version]
  52. Martin, N.E.; Massey, L.; Stowell, C.; Bangma, C.; Briganti, A.; Bill-Axelson, A.; Blute, M.; Catto, J.; Chen, R.C.; D’Amico, A.V.; et al. Defining a Standard Set of Patient-Centered Outcomes for Men with Localized Prostate Cancer. Eur. Urol. 2015, 67, 460–467. [Google Scholar] [CrossRef] [Green Version]
  53. Lavallee, D.C.; Chenok, K.E.; Love, R.M.; Petersen, C.; Holve, E.; Segal, C.D.; Franklin, P.D. Incorporating Patient-Reported Outcomes into Health Care to Engage Patients and Enhance Care. Health Aff. 2016, 35, 575–582. [Google Scholar] [CrossRef]
  54. How REDCap Is Being Used in Response to COVID-19—REDCap. Available online: https://projectredcap.org/covid/ (accessed on 23 May 2023).
  55. NI-RADs|American College of Radiology. Available online: https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/NI-RADs (accessed on 3 August 2021).
  56. Winkler, J.; Moser, R. Biases in Future-Oriented Delphi Studies: A Cognitive Perspective. Technol. Forecast. Soc. Change 2016, 105, 63–76. [Google Scholar] [CrossRef]
  57. Keegan, N.M.; Vasselman, S.E.; Barnett, E.S.; Nweji, B.; Carbone, E.A.; Blum, A.; Morris, M.J.; Rathkopf, D.E.; Slovin, S.F.; Danila, D.C.; et al. Clinical Annotations for Prostate Cancer Research: Defining Data Elements, Creating a Reproducible Analytical Pipeline, and Assessing Data Quality. Prostate 2022, 82, 1107–1116. [Google Scholar] [CrossRef]
  58. AAPM Big Data Subcommittee (BDSC). Available online: https://aapmbdsc.azurewebsites.net/Home/About?ReturnUrl=%2F (accessed on 3 August 2021).
  59. Home|SNOMED International. Available online: https://www.snomed.org/ (accessed on 30 March 2023).
  60. Mayo, C.S.; Feng, M.U.; Brock, K.K.; Kudner, R.; Balter, P.; Buchsbaum, J.C.; Caissie, A.; Covington, E.; Daugherty, E.C.; Dekker, A.L.; et al. Operational Ontology for Oncology (O3)—A Professional Society Based, Multi-Stakeholder, Consensus Driven Informatics Standard Supporting Clinical and Research use of “Real -World” Data from Patients Treated for Cancer: Operational Ontology for Oncology, International Journal of Radiation Oncology Biology Physics. Available online: https://www.sciencedirect.com/science/article/pii/S0360301623005254?utm_campaign=STMJ_AUTH_SERV_PUBLISHED&utm_medium=email&utm_acid=28161953&SIS_ID=&dgcid=STMJ_AUTH_SERV_PUBLISHED&CMX_ID=&utm_in=DM376211&utm_source=AC_ (accessed on 30 May 2023).
  61. Dronkers, E.A.C.; Baatenburg de Jong, R.J.; van der Poel, E.F.; Sewnaik, A.; Offerman, M.P.J. Keys to Successful Implementation of Routine Symptom Monitoring in Head and Neck Oncology with “Healthcare Monitor” and Patients’ Perspectives of Quality of Care. Head Neck 2020, 42, 3590–3600. [Google Scholar] [CrossRef]
  62. Rocque, G.B.; Dent, D.N.; Ingram, S.A.; Caston, N.E.; Thigpen, H.B.; Lalor, F.R.; Jamy, O.H.; Giri, S.; Azuero, A.; Young Pierce, J.; et al. Adaptation of Remote Symptom Monitoring Using Electronic Patient-Reported Outcomes for Implementation in Real-World Settings. JCO Oncol. Pract. 2022, 18, e1943–e1952. [Google Scholar] [CrossRef]
Cancers 15 03121 g001 550
Figure 1. Delphi and operational ontology process flowchart.
Figure 1. Delphi and operational ontology process flowchart.
Cancers 15 03121 g001
Cancers 15 03121 g002 550
Figure 2. Mean scores of importance per subdomain of four validated symptom tools (higher score reflects higher priority/relevance).
Figure 2. Mean scores of importance per subdomain of four validated symptom tools (higher score reflects higher priority/relevance).
Cancers 15 03121 g002
Cancers 15 03121 g003 550
Figure 3. Operation ontology for oncology (O3) website for prostate cancer key data elements.
Figure 3. Operation ontology for oncology (O3) website for prostate cancer key data elements.
Cancers 15 03121 g003
Table
Table 1. Characteristics of the Delphi multidisciplinary Expert Panel (n = 39).
Table 1. Characteristics of the Delphi multidisciplinary Expert Panel (n = 39).
Panel CharacteristicsCount (%) or Average
Age, mean (range), years49.5 (34–70)
Gender
Male26 (67)
Female13 (33)
Specialty
Radiation Oncology22 (56)
Radiation Physics7 (18)
Medical Oncology7 (18)
Urology3 (8)
Practice Setting
Academic31 (79)
Private1 (3)
Not answered7 (18)
Approximate years in practice17.7
Average patient caseload per week30
Table
Table 2. Treatment-related toxicity tier list categorization by the Expert Panel.
Table 2. Treatment-related toxicity tier list categorization by the Expert Panel.
Tier 1 TRTs: Minimum KDESelected Tier Percentage AI
Rectal Hemorrhage96.40.93
Urinary Incontinence92.90.86
Urinary Retention92.60.86
Erectile Dysfunction88.90.79
Hematuria85.70.75
Dysuria85.70.74
Rectal Fistula77.80.64
Urinary Urgency77.80.64
Urinary Frequency77.80.63
Urinary Fistula77.80.63
Proctitis74.10.60
Fecal Incontinence70.40.57
Diarrhea70.40.53
Rectal Perforation66.70.54
Rectal Ulcer63.00.52
Tier 2: Strongly Encouraged
Libido Decrease66.70.49
Gynecomastia66.70.48
Depression59.30.43
Ejaculation Disorder59.30.42
Rectal Fissure55.60.43
Hemorrhoids53.90.45
Rectal Mucositis51.90.37
Rectal Stenosis51.90.37
Fatigue51.90.36
Hot Flashes51.90.40
Rectal Pain48.20.48
Cystitis (Non-infective)48.20.41
Urinary Tract Pain48.20.35
Bladder Spasms48.20.34
Urinary Tract Obstruction40.70.42
Tier 3: Not Required
Superficial Fibrosis96.20.92
Anorexia88.50.79
Nausea84.60.73
Peripheral Neuropathy84.60.72
Dehydration80.80.68
Radiation Dermatitis80.80.66
Vomiting80.80.66
Pelvic Infection61.50.48
Anal Mucositis57.70.49
Constipation53.90.42
Prostatic Hemorrhage50.00.39
Urinary Tract Infection44.40.36
Abbreviations: AI, agreement index; KDE, key data elements; TRTs, treatment-related toxicities.
Table
Table 3. Panel ranking of disease response assessment options.
Table 3. Panel ranking of disease response assessment options.
Name% YesAI *
Biochemical Recurrence100%1
Recurrence at Primary, Pelvic Nodal, and Distant Sites92%0.85
No evidence of disease (NED) or Complete response88%0.78
Progressive Disease84%0.72
Recurrence at Distant Site(s) Only80%0.67
Recurrence at Primary and Pelvic Nodal Sites80%0.67
Recurrence at Primary and Distant Sites80%0.67
Recurrence at Pelvic Nodal and Distant Sites80%0.67
Recurrence at Primary Site Only76%0.62
Recurrence at Pelvic Nodal Site(s) Only76%0.62
Under Treatment72%0.58
Stable Disease52%0.48
Indeterminate (possible pseudo-progression)46%0.48
Partial Response40%0.5
* AI, agreement index. Italicized metrics did not reach consensus threshold for inclusion.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Moreno, A.; Solanki, A.A.; Xu, T.; Lin, R.; Palta, J.; Daugherty, E.; Hong, D.; Hong, J.; Kamran, S.C.; Katsoulakis, E.; et al. Identification of Key Elements in Prostate Cancer for Ontology Building via a Multidisciplinary Consensus Agreement. Cancers 2023, 15, 3121. https://doi.org/10.3390/cancers15123121

AMA Style

Moreno A, Solanki AA, Xu T, Lin R, Palta J, Daugherty E, Hong D, Hong J, Kamran SC, Katsoulakis E, et al. Identification of Key Elements in Prostate Cancer for Ontology Building via a Multidisciplinary Consensus Agreement. Cancers. 2023; 15(12):3121. https://doi.org/10.3390/cancers15123121

Chicago/Turabian Style

Moreno, Amy, Abhishek A. Solanki, Tianlin Xu, Ruitao Lin, Jatinder Palta, Emily Daugherty, David Hong, Julian Hong, Sophia C. Kamran, Evangelia Katsoulakis, and et al. 2023. "Identification of Key Elements in Prostate Cancer for Ontology Building via a Multidisciplinary Consensus Agreement" Cancers 15, no. 12: 3121. https://doi.org/10.3390/cancers15123121

APA Style

Moreno, A., Solanki, A. A., Xu, T., Lin, R., Palta, J., Daugherty, E., Hong, D., Hong, J., Kamran, S. C., Katsoulakis, E., Brock, K., Feng, M., Fuller, C., Mayo, C., & BDSC Prostate Cancer. (2023). Identification of Key Elements in Prostate Cancer for Ontology Building via a Multidisciplinary Consensus Agreement. Cancers, 15(12), 3121. https://doi.org/10.3390/cancers15123121

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop