Next Article in Journal
Totally Excited about Moving Mobility and Exercise (TEAM Me): A Successful Multidisciplinary Exercise Quality Improvement Initiative for Pediatric and Young Adult Oncology and Stem Cell Transplant Inpatients
Previous Article in Journal
Utility of Three-Dimensional Printed Model in Biventricular Repair of Complex Congenital Cardiac Defects: Case Report and Review of Literature
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Children
Volume 9
Issue 2
10.3390/children9020185
children-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Barriers and Opportunities for Patient-Reported Outcome Implementation: A National Pediatrician Survey in the United States

by
Ming Chen
1,
Conor M. Jones
2,
Hailey E. Bauer
3,
Onyekachukwu Osakwe
4,
Pavinarmatha Ketheeswaran
5,
Justin N. Baker
6 and
I-Chan Huang
3,*
1
College of Medicine, University of Tennessee Health Science Center, Memphis, TN 38163, USA
2
Weill Medical College, Cornell University, New York, NY 10021, USA
3
Department of Epidemiology and Cancer Control, St. Jude Children’s Research Hospital, Memphis, TN 38105, USA
4
School of Medicine, University of Mississippi, Jackson, MS 39216, USA
5
Alpert Brown Medical School, Brown University, Providence, RI 02903, USA
6
Department of Oncology, St. Jude Children’s Research Hospital, Memphis, TN 38105, USA
*
Author to whom correspondence should be addressed.
Children 2022, 9(2), 185; https://doi.org/10.3390/children9020185
Submission received: 16 December 2021 / Revised: 27 January 2022 / Accepted: 28 January 2022 / Published: 2 February 2022
Browse Figure
Review Reports Versions Notes

Abstract

:
Purpose: To characterize pediatricians’ perceived barriers and areas of confidence in assessing patient-reported outcomes (PROs) in the U.S., and to test associations of these factors with implementing PRO assessment. Methods: Using a random sample from the members of American Medical Association, we recruited general pediatricians and pediatric subspecialists to complete a survey (July 2011 to December 2013). Perceived barriers and confidence in PRO assessment were compared by age, pediatric specialty (general pediatrics, seven subspecialties), practice settings (academic, private), and region of practice. Multivariable logistic regressions tested associations of demographic factors, barriers, and confidence factors with the implementation of PRO assessment. Findings: The survey was completed by 458 participants (response rate 48.5%); of these, 40.4%, 15.9%, 15.5%, and 8.1% were general pediatricians, cardiology, hematology/oncology, and pulmonary specialists, respectively. PRO assessment was implemented by 29.0% of the pediatricians. The top five barriers for PRO assessment included limited time/manpower (79.0%), limited training (77.4%), lengthy PRO instruments (76%), lack of meaningful cut-offs on PRO scores (75.5%), and unavailable PRO instruments (75.0%). Limited knowledge of PROs (OR 4.10; 95% CI 2.21, 7.60) and unavailability of PRO instruments (OR 1.87; 95% CI 1.01, 3.49) increased the odds of not implementing the assessment, whereas confidence in PRO assessments compatible with norms (OR 0.41; 95% CI 0.23, 0.72) and perceived benefit over clinical judgment alone (OR 0.53; 95% CI 0.31, 0.93) decreased the odds of not implementing the assessment. Interpretation: significant barriers to PRO assessment in pediatric settings suggest the need for providing training, resources, and practical guidance toward implementation. Patient or Public Contribution: healthcare service users contributed to this study by completing a survey and providing feedback about the barriers and areas of confidence in assessing PROs for pediatric populations.

1. Introduction

Patients’ symptoms, quality of life, functional status, and life satisfaction (known as “patient-reported outcomes” or “PROs”) are rarely observable but can be measured through self-reports. Various PRO instruments have been developed [1,2], serving as useful tools to track patients’ health status and facilitate clinical decision-making [3]. In addition, PROs are considered as an important endpoint in clinical trials, providing evidence of responsiveness and efficacy for therapeutic regimens [4]. While PRO assessment plays an important role in measuring children’s health status, the use of PRO assessment in pediatric settings is still limited compared to the use of vital signs and physiologically-based laboratory tests. Despite the robust evidence of psychometric and clinical validity in assessing pediatric PROs [5,6], real-world barriers to implementing PROs among pediatric patients are unclear [7].
The literature has documented the advantages of assessing PROs, such as screening patients at risk of health problems and facilitating patient–physician communication [6,8]. In addition, PRO assessment can improve patients’ awareness of symptomatic problems, empower them to discuss concerns with clinicians, and facilitate tailored interventions with proper palliative care support [9,10]. Indeed, PRO assessment has been associated with the improvement of patients’ emotional, social, and psychological functioning status [8,11,12,13]. Nevertheless, physicians have reported barriers to PRO assessment, typically limited time, inadequate PRO collection infrastructure, lack of knowledge of PROs and score interpretations, and skepticism over the validity of PRO assessment [6,14,15,16].
PRO utilization and constraint issues in pediatrics is more complex than in adults given the distinct developmental issues. The assessment of pediatric PROs is hindered by the patient’s younger age and therefore cognitive capability, which may result in the need for caregiver involvement [11,17]. Although few studies have examined the associations between physician characteristics and PRO assessment, these studies are largely restricted to oncologists, radiation oncologists, and urologists, typically in adult settings [18,19]. Only three studies have investigated PRO implementation issues from the viewpoint of pediatricians, the reported barriers including uncertain value of PRO assessment [20], lack of confidence or self-efficacy regarding PRO uses [21], and limited knowledge regarding PRO analysis [22].
Using a sample from a large representative physician database, we conducted a cross-sectional postal mail survey among pediatricians to investigate their preferences, perceived barriers, and confidence in assessing PROs in routine practice. Specifically, we examined pediatricians’ preference for using specific PRO domains in clinical practice, identified the barriers and confidence factors in conducting PRO assessment, and evaluated how demographic, practice, and census region characteristics were associated with the barrier and confidence factors. Finally, we tested associations of barrier and confidence factors with the implementation of PRO assessment in pediatric practice.

2. Materials and Methods

2.1. Study Design and Participant Identification

A national postal mail survey of general pediatricians and pediatric subspecialists in the U.S. was conducted between July 2011 and December 2013. Using a dataset provided by the American Medical Association (AMA) consisting of a random sample of 1000 general pediatricians and 8488 pediatric subspecialists from four census regions (Midwest, Northeast, South, West), we randomly selected 1000 physicians (400 general pediatricians and 600 subspecialists) to take part in this study. We purposely recruited more subspecialists over the general pediatricians because subspecialists often manage patients who are more severe and require complex interventions compared to general pediatricians. Therefore, it is important to collect sufficient samples of subspecialists to better understand their PRO implementation status.
Eligible study participants were general pediatricians or pediatric subspecialists who were active in practice in the U.S., either as residents, fellows, or attendees. We focused on seven pediatric subspecialty groups: cardiology, endocrinology, gastroenterology, hematology-oncology, nephrology, pulmonology, and rheumatology. We excluded pediatricians with unverified mailing addresses and those who were retired. Pediatricians were required to complete an informed consent form, as a prerequisite to complete the questionnaire. The Institutional Review Board at the University of Florida approved this study.

2.2. Survey Questionnaire

This study created a 60-item survey package for assessing PRO implementation issues. The content included the physician’s demographic characteristics, practice background, interest in important PRO domains for assessment, and perceived barriers and areas of confidence in assessing PROs. Demographic characteristics included attained age, sex, and race/ethnicity. Practice background included years in pediatric practice since residency, practice setting, and proportion of pediatric patients whose PROs were assessed. PRO domains of interest included global, physical, emotional, social and spiritual well-being, pain, sleep, and fatigue.
In the survey package, 9 items measured demographic information, and 16 items measured issues relevant to PRO implementation with aspects of physicians’ knowledge, attitude, confidence, and barriers to implementation. In addition, 1 item captured the current status of implementing PRO assessment. Participants were asked about their interest in particular PRO domains using a binary response category (interested or not interested). A five-point Likert scale (1 = strongly disagree; 5 = strongly agree), was used for items measuring physicians’ confidence in using PRO instruments. Another five-point Likert scale was used to measure physicians’ perceived constraints (1 = extremely; 5 = not at all), as well as the extent of implementation (1 = did not intend to assess PROs; 2 = have thought to assess PROs but unlikely in foreseeable future; 3 = plan to assess PROs; 4 = occasionally assess PROs; 5 = regularly assess PROs). We further categorized the implementation status for PRO assessment as “Yes” if the response scores were 4 or 5, and “No” for otherwise.

2.3. Data Collection

We sent a research packet via postal mail to eligible participants, followed by up to five reminders through telephone calls and/or postal mail if they did not respond to the invitation within three months. The research packet comprised the informed consent form, survey questionnaire, and a $20-value gift card as incentive. Participants could choose to return the completed survey along with the signed informed consent forms via postal mail, e-mail (scanned document) or facsimile, or to decline by not returning the survey.
We replaced selected participants who had invalid mailing addresses (per notifications of postal offices) by other samples randomly selected from the general pediatric or subspecialty group they belonged to. We excluded 55 selected participants from the original sampling frame (N = 1000) due to retired (N = 7), deceased (N = 1) or uncertain (N = 48) status, resulting in 945 confirmed eligible participants in the final statistical analyses.

2.4. Statistical Analysis

Chi-square tests were performed to examine the crude associations of important PRO domains for assessment and perceived barriers, respectively, with pediatricians’ age, specialty, practice setting, and census region in which they practiced. Multivariable logistic regressions were used to examine the associations of pediatricians’ perceived barriers and confidence factors, respectively, with the status of PRO assessment through two analytic models. Model 1 tested associations of an individual barrier and confidence factor with the status of implementing PRO assessment, controlling for demographic (age, sex, race), practice (specialty, practice duration and setting), and census region (South, Northeast, Midwest, West) factors. Model 2 used a stepwise backward selection to remove statistically nonsignificant barrier or confidence factors (p-value ≥ 0.2) from the analysis, controlling for aforementioned covariates. All analyses were performed using STATA 15. Statistical significance was set at p-value < 0.05 (two-sided).

3. Results

Table 1 shows the characteristics of the 458 pediatricians who returned the questionnaires (response rate 48.5% among 945 confirmed eligible pediatricians): 72% were non-Hispanic white; 62% were over 40 years old; and 38% practiced in the South region. For clinical background, 40.4% were general pediatricians, 15.9% were cardiologists, 15.5% were hematology/oncology specialists, and 8.1% were pulmonary specialists. In addition, 58.5% had been practicing for at least 10 years, and 55.4% practiced in an academic setting. Approximately 30.0% of pediatricians were currently implementing PRO assessment.
Table 2 shows pediatricians’ interest in specific PRO domains for assessment by age, specialty, and practice setting. Over 50% of pediatricians expressed a common interest across nine PRO domains. Of these, the top five domains commonly endorsed by pediatricians were emotional well-being (74.0%), global well-being (73.1%), physical well-being (67.0%), social well-being (59.4%), and pain (57.6%). Younger pediatricians aged 20 to 40 years had more interest in PRO assessment compared to those aged over 40 years, with statistically significant differences in the domains of global well-being (p-value = 0.001), emotional well-being (p-value = 0.001), social well-being (p-value = 0.002), school activities (p-value = 0.004), sleep (p-value = 0.005), pain (p-value = 0.013), and family functioning (p-value = 0.028). Pediatricians’ interest in important PRO domains for assessment also varied by specialty. In contrast to other specialties, pediatric hematology/oncology specialties had more interest in assessing fatigue, pain, sleep, and spiritual well-being (all p-values < 0.05), whereas gastroenterology or nephrology specialties had more interest in assessing global well-being, school activities, and family functioning (all p-values < 0.05). General pediatricians reported lower interest in all PRO domains compared to other specialties. Pediatricians in academic settings had more interest in assessing all PRO domains (except emotional and social well-beings) compared to those who worked in private settings (all p-values < 0.05).
Table 3 shows knowledge of and logistic barriers to PRO assessment by pediatricians’ age, practice setting, and census region. The top five reported constraining factors were limited time and manpower (79.0%), limited training (77.4%), long length of PRO instruments (76.0%), lack of clinically meaningful cut-offs for scoring (75.5%), and lack of appropriate PRO instruments for use (75.0%). By pediatrician characteristics, those aged over 40 years reported more barriers to PRO assessment than aged 20–40 years. The statistically significant barrier factors included skepticism about the validity of PRO instruments (p-value = 0.002), inadequate reimbursement incentives for assessing PROs (p-value = 0.007), lack of evidence that PRO assessment improves care (p-value = 0.008), and unavailability of computerized modes for administering PRO assessment (p-value = 0.015).
Table 3 additionally shows that pediatricians who practiced in the South or Northeast encountered more barriers than those in the Midwest or West. The frequently cited barriers by pediatricians in the South were limited training in PRO instruments (82.0%), lack of clinically meaningful cutoffs (79.7%), and limited skills in scoring PROs (79.5%). There were notable differences in the experience of barriers across census regions, including poor knowledge of PRO concepts, scoring and interpretation, varying capabilities of children, unavailability of computerized modes of administration, lack of evidence that PRO assessment improves care, and skepticism about the validity of PRO instruments (all p-values < 0.05). Multivariable analyses suggest that pediatricians who were older (versus younger) in age and resided in the South and Midwest (versus West) encountered more barriers significantly associated with PRO assessment (Table S1).
Despite these barriers, 44% of pediatricians felt confident that PRO assessment provides more benefits to patients than relying on clinical judgement alone, and 40% of the pediatricians indicated that PRO assessment is compatible with their norms (Figure 1). However, only 26% of the pediatricians were confident in their ability to administer PRO instruments, and 20% expressed concerns about the availability of instruments that could accurately evaluate PROs. Multivariable analyses suggest that pediatricians who practiced in the private (versus academic) setting perceived less benefit and norm compatibility in assessing PROs (Table S2).
Table 4 shows associations of perceived barrier and confidence factors with currently not implementing PRO assessment, adjusting for the covariates mentioned in the Methods section (Model 1). Significant barrier factors relevant to knowledge issues included limited training in administering PRO instruments (OR 3.29; 95% CI 1.98, 5.47), and insufficient knowledge of PRO concepts (OR 5.80; 95% CI 3.50, 9.60), scoring (OR 3.32; 95% CI 2.03, 5.41) and interpretation (OR 3.04; 95% CI 1.89, 4.93). Significant barriers relevant to logistic or resource issues included unavailability of appropriate PRO instruments (OR 2.90; 95% CI 1.77, 4.74), skepticism about the validity of PRO instruments (OR 2.78; 95% CI 1.72, 4.50), lack of effectiveness that PRO assessment improves care (OR 2.48; 95% CI: 1.55, 3.97), lack of clinically meaningful cutoffs (OR 2.30; 95% CI 1.40, 3.77), and lack of recommendations on follow-up and referral services (OR 2.14; 95% CI 1.33, 3.44).
Using a parsimonious method with a backward variable selection, Table 4 shows significant barrier and confidence factors for not implementing PRO assessment, adjusting for covariates mentioned in the Methods section (Model 2). The significant barrier factors included limited knowledge of PRO concepts (OR 4.10; 95% CI 2.21, 7.60) and unavailability of appropriate PRO instruments (OR 1.87; 95% CI 1.01, 3.49), whereas the identified confidence factors included the assessment’s compatibility with norms (OR 0.41; 95% CI 0.23, 0.72) and perceived benefit over clinical judgment alone (OR 0.53; 95% CI 0.31, 0.93).

4. Discussion

This is one of the largest studies to examine pediatricians’ perceived barriers and areas of confidence associated with the implementation of PRO assessment. Although it is always a challenge to achieve high response rates from physician surveys, the approximately 50% response rate in our study is comparable to previous physician surveys [23,24,25,26,27,28]. Through a comprehensive survey of general pediatricians and pediatric subspecialists across the U.S., we were able to associate physician-related characteristics with the preferences for various PRO domains. In addition, we identified important knowledge, logistic, and resource barriers that affected the implementation of PRO assessment in routine pediatric practice. This study provides robust evidence for the addressing of PRO implementation issues in pediatrics.
We found that pediatricians who specialized in hematology-oncology, gastroenterology, or nephrology, or who practiced in academic settings, viewed the vast majority of PRO domains as more important versus those who were specialized in general pediatrics, cardiology, pulmonology, and endocrinology, or practiced in private settings. Compared to other specialties, hematologists and oncologists identified pain, fatigue, sleep, and spiritual well-being as important domains because these PRO issues are prevalent in pediatric cancer patients as a result of toxic anticancer therapies and a high stress disease processes [29,30,31,32,33,34]. In addition, hematologists and oncologists were interested in assessing cognitive functioning, which is another side effect resulting from central nervous system-directed therapies, especially prevalent in brain cancer patients [35,36,37,38].
Younger versus older pediatricians reported fewer barriers in PRO assessment. This finding reflects a shift in professional training received in medical education via which the topics of patient centeredness and patient-physician communication have been increasingly emphasized [39]. Younger physicians (including those in training) having fewer barriers against PRO assessment, suggesting that older physicians may have difficulty in instituting new assessment procedures. Expanding educational and organizational efforts (e.g., medical school curriculum, continuous medical education, the development of PRO champions within hospitals) may help to eliminate knowledge constraints and increase health professionals’ confidence or perceived benefits of PRO assessment [39,40].
We found significant variation in geographic locations regarding different barrier factors of PRO implementation. Pediatricians who practiced in the South and Northeast regions reported more barriers compared to those in the West and Midwest. It is likely that varied clinical practice styles or culture, available resources for PRO assessment, and patient characteristics and preferences contribute to the regional differences. The significant barriers reported by pediatricians are in line with previous studies regarding the geographic variation of the assessment of PROs in clinical practice for adults (e.g., constraints in physician education and confidence, clinical organization factors, and specificity of PRO assessments) [40,41,42,43].
Independent of age or geographic location, pediatricians who identified constraints such as limited knowledge of PROs and unavailability of appropriate PRO instruments were significantly less likely to assess PROs. Despite the rapid growth of PRO instruments for use, real-world evidence on the implementation status of PRO assessment is still limited, which requires practical implementation guidance (e.g., integrative PRO data collection platforms and actional referral/treatment plans) [44,45]. We found that, although 45% of the pediatricians acknowledged the usefulness of PRO assessment in providing more benefits than clinical judgment alone, only 26% felt confident in their ability to administer PRO assessment. To remedy this issue, professional PRO or medical societies should recommend actionable, interpretable, and clinically meaningful implementation frameworks for pediatrics [46,47,48]. Special attention should be paid to the aspects of PRO instrument selection, score calculation and interpretation per the needs of pediatric subspecialties and patient populations. For example, the International Society for Quality of Life Research has developed a general guideline, which can be tailored by various pediatric societies for use [44]. Pediatric professional societies should provide recommendations regarding implementation strategies. A recent study found that providing financial incentives significantly increased the rate of PRO assessment [49]. Therefore, health insurance agencies may facilitate appropriate payment mechanisms to optimize PRO data collection, which is now considered an important component of value-based performance programs [50,51].
It is also critical to consider an integrative approach to systematically address barriers in assessing PROs by improving clinical workflow and patient accessibility. Currently, technology-based strategies, e.g., electronic PROs (ePROs), to address PRO implementation issues would be particularly useful as more patients and clinicians utilize mobile and electronic devices. The use of ePROs and computer-adaptive tests (CATs) to monitor PROs can increase the feasibility of PRO assessment remotely, reduce administrative burden, and increase the patient engagement in clinical care [52,53,54,55,56]. Moreover, ePRO platforms allow for tailored care delivery to patients and such information can be linked with electronic health records (EHRs) to facilitate “big-data” initiatives [55,57]. In pediatric settings, PROMIS® and KIDSCREEN have developed free, user-friendly ePROs and CATs. The approach of seamlessly integrating ePROs and CATs with EHRs contributes to real-time clinical decisions.
One of the overlooked implementation issues that can impact PRO integration into clinical workflow is sustainability, especially for longitudinal PRO assessment. This issue is critical for children who are disabled or experience chronic conditions. By establishing meaningful cut-points for PRO domains, developing methods for interpreting score changes over time, and integrating PRO data onto EHRs, we can employ longitudinal data to predict deteriorating health conditions [57]. Developing standardized ePRO platforms will maximize PRO information to be utilized in a variety of care settings and timepoints [47,48,52,53,56].
This study has some weaknesses. First, representativeness of our findings may be limited. Our surveys were collected between 2011 and 2013, and the results may not reflect the contemporary practice of pediatric PRO assessments because modern technology (e.g., mobile technology for PRO data collection, integrating PRO reporting into electronic health record systems) may have been used to overcome some PRO implementation barriers in pediatric oncology. Second, our results may not be generalized to all pediatricians in the U.S. because the participating pediatricians were randomly selected from AMA’s members. Future studies are warranted to confirm our findings by using other professional databases (e.g., American Academy of Pediatrics).
In summary, wide variations in the implementation of PRO assessment were found among a national sample of pediatricians. Pediatricians’ preferences in PRO domains significantly differed by age, specialty, and practice setting. Barriers faced by pediatricians varied mostly by attained age and census region, rather than the health systems in which they were affiliated. To overcome these barriers, targeted interventions in disseminating education and training in PRO topics, developing useful PRO cut-points, promoting ePROs, professional society advocacy, advancing reimbursement strategies, and integrating ePROs with EHRs to deliver tailored longitudinal care to children are warranted.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/children9020185/s1, Table S1: Multivariable Analyses for Pediatricians’ Demographic, Practice, and Census Region Factors Associated with Human and Logistic Barriers to PRO Assessment. Table S2: Multivariable Analyses for Pediatricians’ Demographic, Practice, and Census Region Factors Associated with Confidence in PRO Assessment

Author Contributions

M.C.: results interpretation, manuscript drafting, and manuscript revision. C.M.J.: data analysis, and results interpretation. H.E.B.: data analysis, results interpretation, and manuscript revision. O.O.: data collection, and data analysis. P.K.: data collection, and data analysis. J.N.B.: funding requisition, results interpretation, and manuscript revision. I.-C.H.: study design, funding requisition, data collection, data analysis, results interpretation, manuscript drafting, and manuscript revision. All authors have read and agreed to the published version of the manuscript.

Funding

Supported in part by the U.S. National Institutes of Health grants through the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K23 HD057146) and the National Cancer Institute (R01 CA238368 and P30 CA021765), as well as the American Lebanese Syrian Associated Charities. Funding sources had no involvement in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of University of Florida (protocol code: 2011-U-0463; approval date: 10 May 2011).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data is available upon request from the corresponding author.

Acknowledgments

We thank the physicians for their participation and the research staff who contributed to the recruitment process of this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chen, J.; Ou, L.; Hollis, S.J. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv. Res. 2013, 13, 211–224. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Krohe, M.; Hao, Y.; Lamoureux, R.E.; Galipeau, N.; Globe, D.; Foley, C.; Mazar, I.; Solomon, J.; Shields, A.L. Patient-Reported Outcomes in Metastatic Breast Cancer: A Review of Industry-Sponsored Clinical Trials. Breast Cancer 2016, 10, 93–102. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Kenzik, K.M.; Tuli, S.Y.; Revicki, D.A.; Shenkman, E.A.; Huang, I.-C. Comparison of 4 Pediatric Health-Related Quality-of-Life Instruments: A Study on a Medicaid Population. Med. Decis. Mak. 2014, 34, 590–602. [Google Scholar] [CrossRef] [Green Version]
  4. Miller, T.P.; Li, Y.; Kavcic, M.; Troxel, A.; Huang, Y.-S.V.; Sung, L.; Alonzo, T.A.; Gerbing, R.; Hall, M.; Daves, M.H.; et al. Accuracy of Adverse Event Ascertainment in Clinical Trials for Pediatric Acute Myeloid Leukemia. J. Clin. Oncol. 2016, 34, 1537–1543. [Google Scholar] [CrossRef] [PubMed]
  5. Bickman, L.; Rosof-Williams, J.; Salzer, M.S.; Summerfelt, W.T.; Noser, K.; Wilson, S.J.; Karver, M.S. What Information Do Clinicians Value for Monitoring Adolescent Client Progress and Outcomes? Prof. Psychol. 2000, 31, 70–74. [Google Scholar] [CrossRef]
  6. Huang, I.-C.; Thompson, L.A.; Chi, Y.-Y.; Knapp, C.A.; Revicki, D.A.; Seid, M.; Shenkman, E.A. The Linkage between Pediatric Quality of Life and Health Conditions: Establishing Clinically Meaningful Cutoff Scores for the PedsQL. Value Health 2009, 12, 773–781. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Huang, I.-C.; Revicki, D.A.; Schwartz, C.E. Measuring pediatric patient-reported outcomes: Good progress but a long way to go. Qual. Life Res. 2014, 23, 747–750. [Google Scholar] [CrossRef] [Green Version]
  8. Detmar, S.B.; Muller, M.J.; Schornagel, J.H.; Wever, L.D.V.; Aaronson, N.K. Health-Related Quality-of-Life Assessments and Patient-Physician Communication: A randomized controlled trial. JAMA 2002, 288, 3027–3034. [Google Scholar] [CrossRef]
  9. Espallargues, M.; Valderas, J.M.; Alonso, J. Provision of Feedback on Perceived Health Status to Health Care Professionals: A systematic review of its impact. Med. Care 2000, 38, 175–186. [Google Scholar] [CrossRef] [Green Version]
  10. Greenhalgh, J.; Gooding, K.; Gibbons, E.; Dalkin, S.; Wright, J.; Valderas, J.; Black, N. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J. Patient Rep. Outcomes 2018, 2, 42. [Google Scholar] [CrossRef]
  11. Engelen, V.; Detmar, S.; Koopman, H.; Maurice-Stam, H.; Caron, H.; Hoogerbrugge, P.; Egeler, R.M.; Kaspers, G.; Grootenhuis, M. Reporting health-related quality of life scores to physicians during routine follow-up visits of pediatric oncology patients: Is it effective? Pediatr. Blood Cancer 2011, 58, 766–774. [Google Scholar] [CrossRef] [PubMed]
  12. Rubenstein, L.V.; McCoy, J.M.; Cope, D.W.; Barrett, P.A.; Hirsch, S.H.; Messer, K.S.; Young, R.T. Improving patient quality of life with feedback to physicians about functional status. J. Gen. Intern. Med. 1995, 10, 607–614. [Google Scholar] [CrossRef] [PubMed]
  13. Seid, M.; Varni, J.W.; Segall, D.; Kurtin, P.S. Health-related quality of life as a predictor of pediatric healthcare costs: A two-year prospective cohort analysis. Heal. Qual. Life Outcomes 2004, 2, 48. [Google Scholar] [CrossRef] [Green Version]
  14. Deyo, R.A.; Patrick, D.L. Barriers to the Use of Health Status Measures in Clinical Investigation, Patient Care, and Policy Research. Med. Care 1989, 27, S254–S268. [Google Scholar] [CrossRef] [PubMed]
  15. Greenhalgh, J.; Long, A.F.; Flynn, R. The use of patient reported outcome measures in routine clinical practice: Lack of impact or lack of theory? Soc. Sci. Med. 2005, 60, 833–843. [Google Scholar] [CrossRef] [PubMed]
  16. Hatfield, D.R.; Ogles, B.M. Why Some Clinicians Use Outcome Measures and Others Do Not. Adm. Ment. Health 2007, 34, 283–291. [Google Scholar] [CrossRef] [PubMed]
  17. Janssens, A.; Rogers, M.; Coon, J.T.; Allen, K.; Green, C.; Jenkinson, C.; Tennant, A.; Logan, S.; Morris, C. A Systematic Review of Generic Multidimensional Patient-Reported Outcome Measures for Children, Part II: Evaluation of Psychometric Performance of English-Language Versions in a General Population. Value Health 2015, 18, 334–345. [Google Scholar] [CrossRef] [Green Version]
  18. Detmar, S.B.; Muller, M.J.; Wever, L.D.V.; Schornagel, J.H.; Aaronson, N.K. Patient-Physician Communication During Outpatient Palliative Treatment Visits. JAMA 2001, 285, 1351–1357. [Google Scholar] [CrossRef] [Green Version]
  19. Kim, S.P.; Karnes, R.J.; Nguyen, P.L.; Ziegenfuss, J.Y.; Han, L.C.; Thompson, R.H.; Trinh, Q.-D.; Sun, M.; Boorjian, S.A.; Beebe, T.J.; et al. Clinical Implementation of Quality of Life Instruments and Prediction Tools for Localized Prostate Cancer: Results from a National Survey of Radiation Oncologists and Urologists. J. Urol. 2012, 189, 2092–2098. [Google Scholar] [CrossRef]
  20. Michniacki, T.F.; Walkovich, K.J.; Merz, L.; Sturza, J.; Abraham, R.S. Provider Perceptions of Quality of Life, Neurocognition, Physical Well-being, and Psychosocial Health in Patients with Primary Immunodeficiency/Immune Dysregulation Conditions. J. Clin. Immunol. 2019, 39, 805–813. [Google Scholar] [CrossRef]
  21. Heinemann, A.W.; Nitsch, K.P.; Ehrlich-Jones, L.; Malamut, L.; Semik, P.; Srdanovic, N.; Kocherginsky, M.; Hobbs, S. Effects of an Implementation Intervention to Promote Use of Patient-Reported Outcome Measures on Clinicians’ Perceptions of Evidence-Based Practice, Implementation Leadership, and Team Functioning. J. Contin. Educ. Health Prof. 2019, 39, 103–111. [Google Scholar] [CrossRef] [PubMed]
  22. Edbrooke-Childs, J.; Barry, D.; Rodriguez, I.M.; Papageorgiou, D.; Wolpert, M.; Schulz, J. Patient reported outcome measures in child and adolescent mental health services: Associations between clinician demographic characteristics, attitudes and efficacy. Child Adolesc. Ment. Health 2016, 22, 36–41. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Cook, J.V.; Dickinson, H.O.; Eccles, M.P. Response rates in postal surveys of healthcare professionals between 1996 and 2005: An observational study. BMC Health Serv. Res. 2009, 9, 160. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Cull, W.L.; O’Connor, K.G.; Sharp, S.; Tang, S.-F.S. Response Rates and Response Bias for 50 Surveys of Pediatricians. Health Serv. Res. 2005, 40, 213–226. [Google Scholar] [CrossRef] [Green Version]
  25. Cummings, S.M.; Savitz, L.A.; Konrad, T.R. Reported response rates to mailed physician questionnaires. Health Serv. Res. 2001, 35, 1347–1355. [Google Scholar]
  26. Kellerman, S.E.; Herold, J. Physician response to surveys. A review of the literature. Am. J. Prev. Med. 2001, 20, 61–67. [Google Scholar] [CrossRef]
  27. Martins, Y.; Lederman, R.I.; Lowenstein, C.L.; Joffe, S.; Neville, B.A.; Hastings, B.T.; Abel, G.A. Increasing response rates from physicians in oncology research: A structured literature review and data from a recent physician survey. Br. J. Cancer 2012, 106, 1021–1026. [Google Scholar] [CrossRef] [Green Version]
  28. VanGeest, J.B.; Johnson, T.; Welch, V.L. Methodologies for Improving Response Rates in Surveys of Physicians: A systematic review. Eval. Health Prof. 2007, 30, 303–321. [Google Scholar] [CrossRef]
  29. Cheung, Y.T.; Brinkman, T.M.; Mulrooney, D.A.; Mzayek, Y.; Liu, W.; Banerjee, P.; Panoskaltsis-Mortari, A.; Srivastava, D.; Pui, C.-H.; Robison, L.L.; et al. Impact of sleep, fatigue, and systemic inflammation on neurocognitive and behavioral outcomes in long-term survivors of childhood acute lymphoblastic leukemia. Cancer 2017, 123, 3410–3419. [Google Scholar] [CrossRef]
  30. Hockenberry-Eaton, M.; Hinds, P.S. Fatigue in children and adolescents with cancer: Evolution of a program of study. Semin. Oncol. Nurs. 2000, 16, 261–272. [Google Scholar] [CrossRef]
  31. Jibb, L.; Nathan, P.C.; Stevens, B.J.; Seto, E.; Cafazzo, J.A.; Stephens, N.; Yohannes, L.; Stinson, J.N. Psychological and Physical Interventions for the Management of Cancer-Related Pain in Pediatric and Young Adult Patients: An Integrative Review. Oncol. Nurs. Forum 2015, 42, E339–E357. [Google Scholar] [CrossRef] [Green Version]
  32. Kaleyias, J.; Manley, P.; Kothare, S.V. Sleep Disorders in Children with Cancer. Semin. Pediatr. Neurol. 2012, 19, 25–34. [Google Scholar] [CrossRef] [PubMed]
  33. Liu, W.; Cheung, Y.T.; Brinkman, T.M.; Banerjee, P.; Srivastava, D.; Nolan, V.G.; Zhang, H.; Gurney, J.G.; Pui, C.-H.; Robison, L.L.; et al. Behavioral symptoms and psychiatric disorders in child and adolescent long-term survivors of childhood acute lymphoblastic leukemia treated with chemotherapy only. PsychoOncology 2018, 27, 1597–1607. [Google Scholar] [CrossRef] [PubMed]
  34. Nagarajan, R.; Gerbing, R.; Alonzo, T.; Johnston, D.L.; Aplenc, R.; Kolb, E.A.; Meshinchi, S.; Barakat, L.P.; Sung, L. Quality of life in pediatric acute myeloid leukemia: Report from the Children’s Oncology Group. Cancer Med. 2019, 8, 4454–4464. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Cheung, Y.T.; Eskind, A.; Inaba, H.; Hudson, M.M.; Pui, C.-H.; Krull, K.R.; Wolf, J. Association of Bacteremic Sepsis with Long-term Neurocognitive Dysfunction in Pediatric Patients With Acute Lymphoblastic Leukemia. JAMA Pediatr. 2018, 172, 1092. [Google Scholar] [CrossRef] [PubMed]
  36. Cheung, Y.T.; Sabin, N.D.; Reddick, W.E.; Bhojwani, D.; Liu, W.; Brinkman, T.M.; Glass, J.O.; Hwang, S.N.; Srivastava, D.; Pui, C.-H.; et al. Leukoencephalopathy and long-term neurobehavioural, neurocognitive, and brain imaging outcomes in survivors of childhood acute lymphoblastic leukaemia treated with chemotherapy: A longitudinal analysis. Lancet Haematol. 2016, 3, e456–e466. [Google Scholar] [CrossRef] [Green Version]
  37. Huang, I.-C.; Brinkman, T.M.; Cheung, Y.T.; Pui, C.-H.; Hudson, M.M.; Krull, K.R. Functional consequence of cognitive impairment in survivors of childhood acute lymphoblastic leukemia (ALL): The role of cancer symptoms as mediators. J. Clin. Oncol. 2016, 34, 235. [Google Scholar] [CrossRef]
  38. Liu, W.; Cheung, Y.T.; Conklin, H.M.; Jacola, L.M.; Srivastava, D.; Nolan, V.G.; Zhang, H.; Gurney, J.G.; Huang, I.-C.; Robison, L.L.; et al. Evolution of neurocognitive function in long-term survivors of childhood acute lymphoblastic leukemia treated with chemotherapy only. J. Cancer Surviv. 2018, 12, 398–406. [Google Scholar] [CrossRef]
  39. Parent, K.; Jones, K.; Phillips, L.; Stojan, J.N.; House, J.B. Teaching Patient- and Family-Centered Care: Integrating Shared Humanity into Medical Education Curricula. AMA J. Ethics 2016, 18, 24–32. [Google Scholar] [CrossRef]
  40. Antunes, B.; Harding, R.; Higginson, I.J.; Euroimpact. Implementing patient-reported outcome measures in palliative care clinical practice: A systematic review of facilitators and barriers. Palliat. Med. 2014, 28, 158–175. [Google Scholar] [CrossRef] [Green Version]
  41. Boyce, M.B.; Browne, J.P.; Greenhalgh, J. The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: A systematic review of qualitative research. BMJ Qual. Saf. 2014, 23, 508–518. [Google Scholar] [CrossRef] [PubMed]
  42. Duncan, E.A.S.; Murray, J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: A systematic review. BMC Health Serv. Res. 2012, 12, 96. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Swinkels, R.A.; Van Peppen, R.P.; Wittink, H.; Custers, J.W.; Beurskens, A.J. Current use and barriers and facilitators for implementation of standardised measures in physical therapy in the Netherlands. BMC Musculoskelet. Disord. 2011, 12, 106. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Anatchkova, M.; Donelson, S.M.; Skalicky, A.M.; McHorney, C.A.; Jagun, D.; Whiteley, J. Exploring the implementation of patient-reported outcome measures in cancer care: Need for more real-world evidence results in the peer reviewed literature. J. Patient Rep. Outcomes 2018, 2, 64. [Google Scholar] [CrossRef]
  45. Gerhardt, W.E.; Mara, C.A.; Kudel, I.; Morgan, E.M.; Schoettker, P.J.; Napora, J.; Britto, M.T.; Alessandrini, E.A. Systemwide Implementation of Patient-Reported Outcomes in Routine Clinical Care at a Children’s Hospital. Jt. Comm. J. Qual. Patient Saf. 2018, 44, 441–453. [Google Scholar] [CrossRef]
  46. Kaur, G.; Kyte, D.; Reeve, B.B.; Basch, E.; Calvert, M. Patient-reported outcome monitoring in a routine paediatric oncology setting: Challenges and opportunities. Lancet Oncol. 2019, 20, 19–20. [Google Scholar] [CrossRef] [Green Version]
  47. Reeve, B.B.; Wyrwich, K.W.; Wu, A.W.; Velikova, G.; Terwee, C.B.; Snyder, C.F.; Schwartz, C.; Revicki, D.A.; Moinpour, C.M.; McLeod, L.; et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual. Life Res. 2013, 22, 1889–1905. [Google Scholar] [CrossRef]
  48. Snyder, C.F.; Aaronson, N.K.; Choucair, A.K.; Elliott, T.; Greenhalgh, J.; Halyard, M.Y.; Hess, R.; Miller, D.M.; Reeve, B.B.; Santana, M. Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Qual. Life Res. 2012, 21, 1305–1314. [Google Scholar] [CrossRef]
  49. Sisodia, R.C.; Dankers, C.; Orav, J.; Joseph, B.; Meyers, P.; Wright, P.; Amand, D.S.; Del Carmen, M.; Ferris, T.; Heng, M.; et al. Factors Associated with Increased Collection of Patient-Reported Outcomes Within a Large Health Care System. JAMA Netw. Open 2020, 3, e202764. [Google Scholar] [CrossRef]
  50. Basch, E.; Spertus, J.; Dudley, R.A.; Wu, A.; Chuahan, C.; Cohen, P.; Smith, M.L.; Black, N.; Crawford, A.; Christensen, K.; et al. Methods for Developing Patient-Reported Outcome-Based Performance Measures (PRO-PMs). Value Health 2015, 18, 493–504. [Google Scholar] [CrossRef] [Green Version]
  51. Bevans, K.B.; Moon, J.; Carle, A.C.; Mara, C.A.; Lai, J.-S.; DiMarco, L.; Muller, N.; Woods, D. Patient Reported Outcomes as Indicators of Pediatric Health Care Quality. Acad. Pediatr. 2014, 14, S90–S96. [Google Scholar] [CrossRef] [PubMed]
  52. Ashley, L.; Jones, H.; Thomas, J.; Newsham, A.; Downing, A.; Morris, E.; Brown, J.; Velikova, G.; Forman, D.; Wright, P. Integrating Patient Reported Outcomes with Clinical Cancer Registry Data: A Feasibility Study of the Electronic Patient-Reported Outcomes from Cancer Survivors (ePOCS) System. J. Med Internet Res. 2013, 15, e230. [Google Scholar] [CrossRef] [PubMed]
  53. Barthel, D.; Fischer, K.; Nolte, S.; Otto, C.; Meyrose, A.-K.; Reisinger, S.; Dabs, M.; Thyen, U.; Klein, M.O.; Muehlan, H.; et al. Implementation of the Kids-CAT in clinical settings: A newly developed computer-adaptive test to facilitate the assessment of patient-reported outcomes of children and adolescents in clinical practice in Germany. Qual. Life Res. 2016, 25, 585–594. [Google Scholar] [CrossRef]
  54. Holch, P.; Warrington, L.; Bamforth, L.C.A.; Keding, A.; Ziegler, L.E.; Absolom, K.; Hector, C.; Harley, C.; Johnson, O.A.; Hall, G.; et al. Development of an integrated electronic platform for patient self-report and management of adverse events during cancer treatment. Ann. Oncol. 2017, 28, 2305–2311. [Google Scholar] [CrossRef] [PubMed]
  55. LeBlanc, T.; Abernethy, A.P. Patient-reported outcomes in cancer care—hearing the patient voice at greater volume. Nat. Rev. Clin. Oncol. 2017, 14, 763–772. [Google Scholar] [CrossRef] [PubMed]
  56. Stover, A.; Stricker, C.T.; Hammelef, K.; Henson, S.; Carr, P.; Jansen, J.; Deal, A.M.; Bennett, A.V.; Basch, E.M. Using Stakeholder Engagement to Overcome Barriers to Implementing Patient-reported Outcomes (PROs) in Cancer Care Delivery. Med. Care 2019, 57, S92–S99. [Google Scholar] [CrossRef]
  57. Schepers, S.A.; Nicolaas, S.M.S.; Haverman, L.; Wensing, M.; Van Meeteren, A.Y.S.; Veening, M.A.; Caron, H.N.; Hoogerbrugge, P.M.; Kaspers, G.J.; Verhaak, C.M.; et al. Real-world implementation of electronic patient-reported outcomes in outpatient pediatric cancer care. PsychoOncology 2016, 26, 951–959. [Google Scholar] [CrossRef]
Children 09 00185 g001 550
Figure 1. Areas of Confidence in PRO Assessment.
Figure 1. Areas of Confidence in PRO Assessment.
Children 09 00185 g001
Table
Table 1. Characteristics of Participating Pediatricians (N = 458).
Table 1. Characteristics of Participating Pediatricians (N = 458).
CharacteristicsN (%)
Age in years
20–40174 (38.0)
≥41284 (62.0)
Sex
Female220 (48.6)
Male233 (51.4)
Race/ethnicity
White, non-Hispanic327 (71.9)
Asian or Pacific Islander78 (17.1)
Hispanic25 (5.5)
Black, non-Hispanic13 (2.9)
Other12 (2.6)
Years in pediatric practice since residency
0–10188 (41.5)
≥11265 (58.5)
Specialty
General Pediatrics185 (40.4)
Pediatric Cardiology 73 (15.9)
Pediatric Hematology Oncology71 (15.5)
Pediatric Pulmonology37 (8.1)
Pediatric Endocrinology36 (7.9)
Pediatric Gastroenterology34 (7.4)
Pediatric Nephrology18 (3.9)
Pediatric Rheumatology4 (0.9)
Practice setting
Academic251 (55.4)
Private Practice202 (44.6)
U.S. Census Region
South173 (37.8)
Northeast101 (22.1)
Midwest99 (21.6)
West85 (18.6)
Proportion of patients with PROs being assessed
<10321 (71.2)
≥11130 (28.8)
Implementing PRO assessment
Currently not being implemented †327 (71.0)
Currently being implemented ‡127 (29.0)
† Did not intend to assess PROs, have thought to assess PROs but unlikely in foreseeable future, or plan to assess PROs. ‡ Regularly or occasionally assess PROs.
Table
Table 2. Important PRO Domains for Assessment Associated with Pediatricians’ Age, Specialty, and Practice Setting †.
Table 2. Important PRO Domains for Assessment Associated with Pediatricians’ Age, Specialty, and Practice Setting †.
OverallAge in YearsSpecialty ‡Practice Setting
20–40≥41X2
(p-Value)		GPC/PE/RHOG/NX2
(p-Value)		AcademicPrivateX2
(p-Value)
PRO DomainsN (%)N (%)N (%)N (%)N (%)N (%)N (%)N (%)N (%)N (%)
Emotional well-being339
(74.0)		144
(83.8)		195
(68.7)		11.15 (0.001)137 (74.1)79 (71.8)33 (82.5)50 (70.4)40 (76.9)2.45
(0.648)		195 (77.7)141 (69.8)3.64 (0.057)
Global well-being335
(73.1)		146
(82.2)		192
(67.6)		11.67 (0.001)120 (64.9)86 (79.2)32 (80.0)53 (74.6)44 (84.6)12.40 (0.015)203 (80.9)130 (64.4)15.69 (<.001)
Physical well-being307
(67.0)		118
(67.8)		189
(66.5)		0.08
(0.780)		111 (60.0)81 (73.6)29 (72.5)53 (74.6)33 (63.5)9.02
(0.061)		180 (71.7)126 (62.4)4.45 (0.035)
Social well-being272
(59.4)		119
(68.4)		153
(53.9)		9.43
(0.002)		108 (58.4)61 (55.5)27 (67.5)43 (60.6)33 (63.5)2.27
(0.686)		159 (63.3)112 (55.4)2.91 (0.088)
Pain264
(57.6)		113
(64.9)		151
(53.2)		6.13
(0.013)		87 (47.0)58 (52.7)26 (65.0)56 (78.9)37 (71.2)27.51 (<0.001)170 (67.7)93 (46.0)21.62 (<0.001)
Family functioning257
(56.1)		109
(62.6)		148
(52.1)		4.86
(0.028)		91 (49.2)58 (52.7)25 (62.5)46 (64.8)37 (71.2)11.72 (0.020)153 (61.0)102 (50.5)4.98 (0.026)
Cognitive functioning250
(54.6)		105
(60.3)		145
(51.1)		3.76
(0.053)		90 (48.6)66 (60.0)22 (55.0)46 (64.8)26 (50.0)7.36
(0.118)		154 (61.4)94 (46.5)9.92 (0.002)
School activities242
(52.8)		107
(61.5)		135
(47.5)		8.44
(0.004)		82 (44.3)60 (54.5)23 (57.5)44 (62.0)33 (63.5)10.59 (0.032)149 (59.4)91 (45.0)9.20 (0.002)
Sleep241
(52.6)		106
(60.9)		135
(47.5)		7.75
(0.005)		81 (43.8)58 (52.7)20 (50.0)48 (67.6)34 (65.4)15.70 (0.003)154 (61.4)87 (43.1)15.03 (<0.001)
Fatigue215
(46.9)		86
(49.4)		129
(45.4)		0.69
(0.405)		63 (34.1)54 (49.1)20 (50.0)52 (73.2)26 (50.0)32.60 (<0.001)141 (56.2)74 (36.6)17.14 (<0.001)
Spiritual well-being121
(26.4)		50
(28.7)		71
(25.0)		0.78
(0.379)		35 (18.1)31 (28.2)15 (37.5)29 (39.4)12 (23.1)14.54 (0.006)79 (31.5)42 (20.8)6.52 (0.011)
† Values represents the number of participating pediatricians who agreed that specific PRO domain was important for assessment. ‡ Specialty: GP = General Pediatrics; C/P = Pediatric Cardiology or Pulmonology; E/R = Pediatric Endocrinology or Rheumatology; HO = Pediatric Hematology Oncology; G/N = Pediatric Gastroenterology or Nephrology.
Table
Table 3. Bivariate Analyses for Pediatricians’ Age, Practice Setting, and Census Region Associated with Human and Logistic Barriers to PRO Assessment †.
Table 3. Bivariate Analyses for Pediatricians’ Age, Practice Setting, and Census Region Associated with Human and Logistic Barriers to PRO Assessment †.
OverallAge in YearsPractice SettingCensus Region
20–40≥41X2
(p-Value)		AcademicPrivate PracticeX2
(p-value)		NortheastMidwestSouthWestX2
(p-Value)
ConstraintsN (%)N (%)N (%)N (%)N (%)N (%)N (%)N (%)N (%)
Limited time and manpower for assessing PROs338 (79.0)130 (77.4)208 (80.0)0.42
(0.516)		193 (80.4)141 (77.0)0.71
(0.400)		81 (85.3)81 (83.5)121 (76.1)55 (71.4)6.89
(0.075)
Limited training on how to administer PRO instruments333 (77.4)133 (79.2)200 (76.3)0.47
(0.493)		185 (76.8)146 (79.3)0.41
(0.525)		75 (78.9)73 (75.3)132 (82.0)53 (68.8)5.56
(0.135)
Long length of PRO instruments323 (76.0)129 (76.8)194 (75.5)0.09
(0.759)		184 (76.7)136 (75.6)0.07
(0.791)		76 (80.9)73 (76.0)125 (79.1)49 (63.6)8.51
(0.037)
Lack of clinically meaningful cut-offs for PRO scores321 (75.5)122 (73.5)199 (76.8)0.61
(0.435)		179 (74.9)139 (76.8)0.20
(0.653)		75 (79.8)72 (75.0)126 (79.7)48 (62.3)9.71
(0.021)
Unavailability of appropriate PRO instruments318 (75.0)124 (74.3)194 (75.5)0.08
(0.774)		182 (76.2)133 (73.9)0.28
(0.596)		75 (79.8)70 (72.9)121 (77.1)52 (67.5)4.02
(0.259)
Limited skills on scoring PRO results 316 (73.7)119 (71.3)197 (75.2)0.81
(0.367)		174 (72.5)140 (76.1)0.70
(0.404)		72 (76.6)66 (68.0)128 (79.5)50 (67.9)7.85
(0.049)
Limited ability to interpret PRO results 307 (71.7)116 (69.5)191 (73.2)0.70
(0.405)		170 (70.8)135 (73.8)0.45
(0.505)		70 (74.5)67 (69.1)125 (78.1)45 (58.4)10.62
(0.014)
Limited knowledge of PRO concepts 309 (71.2)120 (71.0)189 (71.3)0.01
(0.944)		173 (71.5)133 (71.1)0.01
(0.934)		72 (75.8)67 (68.4)124 (76.5)46 (58.2)10.10
(0.018)
Lack of recommendations on follow-up and referral services299 (70.5)116 (69.5)183 (71.2)0.15
(0.700)		165 (69.0)131 (72.8)0.69
(0.405)		72 (76.6)63 (65.6)117 (74.5)47 (61.0)7.31
(0.063)
Varying capabilities of children284 (66.8)108 (64.7)176 (68.2)0.58
(0.448)		159 (66.5)121 (66.9)0.01
(0.944)		69 (73.4)54 (56.3)116 (73.4)45 (58.4)12.22
(0.007)
Unavailability of computerized mode for administering PROs249 (58.8)86 (51.5)163 (63.4)5.94
(0.015)		134 (56.1)111 (61.7)1.33
(0.250)		61 (64.9)41 (42.7)100 (63.7)47 (61.0)13.41
(0.004)
Lack of reimbursement incentives for assessing PROs241 (56.6)51 (48.5)160 (61.8)7.28
(0.007)		129 (54.0)110 (60.4)1.76
(0.185)		58 (61.7)56 (57.7)90 (57.0)37 (48.1)3.35
(0.341)
Lack of evidence that PRO assessment improves care220 (51.8)73 (43.7)147 (57.0)7.14
(0.008)		115 (48.1)103 (56.9)3.19
(0.074)		55 (58.5)43 (44.8)95 (60.1)27 (35.1)16.61
(0.001)
Skepticism about the validity of PRO instruments208 (49.1)66 (39.5)142 (55.3)10.03 (0.002)110 (46.0)96
(53.3)		2.19
(0.139)		50 (53.2)46 (47.9)89 (56.7)23 (29.9)15.69
(0.001)
† See Table S1 for the Results of Multivariable Analyses.
Table
Table 4. Multivariable Analyses for the Barrier and Confidence Factors Associated with Currently Not Implementing PRO Assessment in Clinical Practice &.
Table 4. Multivariable Analyses for the Barrier and Confidence Factors Associated with Currently Not Implementing PRO Assessment in Clinical Practice &.
FactorsModel 1 †
Regular Multivariable Logistic Regression for Currently not Implementing PRO Assessment		Model 2 ‡
Stepwise Backward Multivariable Logistic Regression for Currently not Implementing PRO Assessment
OR (95% CI)OR (95% CI)
Constraint factors
Limited time and manpower for assessing PROs1.98 (1.18, 3.31) *NS
Limited training on how to administer PRO instruments3.29 (1.98, 5.47) ***NS
Long length of PRO instruments1.66 (1.01, 2.74) *NS
Lack of clinically meaningful cut-offs for PRO scores2.30 (1.40, 3.77) **NS
Unavailability of appropriate PRO instruments2.90 (1.77, 4.74) ***1.87 (1.01, 3.49) *
Limited skills on scoring PRO results3.32 (2.03, 5.41) ***NS
Limited ability to interpret PRO results3.04 (1.89, 4.93) ***NS
Limited knowledge of PRO concepts5.80 (3.50, 9.60) ***4.10 (2.21, 7.60) ***
Lack of recommendations on follow-up and referral services2.14 (1.33, 3.44) **NS
Varying capabilities of children1.17 (0.73, 1.87)NS
Unavailability of computerized mode for administering PROs1.83 (1.16, 2.90) *NS
Lack of reimbursement incentives for assessing PROs1.38 (0.88, 2.16)NS
Lack of evidence that PRO assessment improves care2.48 (1.55, 3.97) ***NS
Skepticism about the validity of PRO instruments2.78 (1.72, 4.50) ***NS
Confidence factors
More benefits of PRO assessment than clinical judgments alone0.32 (0.20, 0.50) ***0.53 (0.31, 0.93) *
PRO assessment compatible with my norms0.21 (0.13, 0.33) ***0.41 (0.23, 0.72) **
Abilities to administer PRO instruments0.18 (0.11, 0.29) ***NS
Available instruments accurately evaluate PROs0.22 (0.13, 0.37) ***NS
OR = odds ratio; CI = confidence interval; NS = variables not selected into the final mode per stepwise approach (p-value ≥ 0.2). * p-value < 0.05; ** p-value < 0.01; *** p-value < 0.001. & Status of implementation: currently not implementing PRO assessment (do not intend to assess PROs, have thought to assess PROs but unlikely in foreseeable future, or plan to assess PROs) and currently implementing PRO assessment (regularly or occasionally assess PROs). † Multivariable logistic regression model estimated ORs for PRO assessment currently not being implemented (vs. implemented) associated with individual constraint and confidence variables by adjusting for pediatricians’ demographic, practice and census region covariates. ‡ Stepwise backward multivariable logistic regression model estimated ORs for PRO assessment currently not being implemented (vs. implemented) associated with all constraint and confidence variables by adjusting for pediatricians’ demographic, practice and census region covariates.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chen, M.; Jones, C.M.; Bauer, H.E.; Osakwe, O.; Ketheeswaran, P.; Baker, J.N.; Huang, I.-C. Barriers and Opportunities for Patient-Reported Outcome Implementation: A National Pediatrician Survey in the United States. Children 2022, 9, 185. https://doi.org/10.3390/children9020185

AMA Style

Chen M, Jones CM, Bauer HE, Osakwe O, Ketheeswaran P, Baker JN, Huang I-C. Barriers and Opportunities for Patient-Reported Outcome Implementation: A National Pediatrician Survey in the United States. Children. 2022; 9(2):185. https://doi.org/10.3390/children9020185

Chicago/Turabian Style

Chen, Ming, Conor M. Jones, Hailey E. Bauer, Onyekachukwu Osakwe, Pavinarmatha Ketheeswaran, Justin N. Baker, and I-Chan Huang. 2022. "Barriers and Opportunities for Patient-Reported Outcome Implementation: A National Pediatrician Survey in the United States" Children 9, no. 2: 185. https://doi.org/10.3390/children9020185

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop