- freely available
JPM 2013, 3(1), 1-13; doi:10.3390/jpm3010001
Published: 27 December 2012
Abstract: Cancer care is often inconsistently delivered with inadequate incorporation of patient values and objective evidence into decision-making. Utilization of time limited trials of care with predefined decision points that are based on iteratively updated best evidence, tools that inform providers about a patient’s experience and values, and known information about a patient’s disease will allow superior matched care to be delivered. Personalized medicine does not merely refer to the incorporation of genetic information into clinical care, it involves utilization of the wide array of data points relevant to care, many of which are readily available at the bedside today. By pushing uptake of personalized matching available today, clinicians can better address the triple aim of improved health, lowers costs, and enhanced patient experience, and we can prepare the health care landscape for the iterative inclusion of progressively more sophisticated information as newer tests and information become available to support the personalized medicine paradigm.
The delivery of medical care is undergoing fundamental change. In no area is this more apparent than in cancer. For decades, the treatment armamentarium available to oncologists remained limited despite substantial progress in other areas. More recently, this paradigm has shifted, with many of the drugs that are newly approved by the United States Food and Drug Administration being used to treat cancer . As an example, five new therapies have been approved to treat prostate cancer over the past five years, many compared only to placebo prior to approval. As such, there is little comparative data about their relative efficacy and the appropriate treatment sequence. Accumulating insights regarding the genomic underpinnings of disease and the relative efficacy of treatments in different patient populations holds great promise to tailor anti-cancer interventions to those most likely to benefit, but fulfillment of this potential remains elusive [2,3,4]. Within this backdrop, it is evident that the value of the care that patients receive can be better defined and optimized if early progress on a few fronts continues to gain momentum.
Understanding the value of treatments relies first on the generation of more robust data about the care being delivered. New data infrastructure is being developed that will enable change under the American Recovery and Reinvestment Act (ARRA) of 2009 and the Patient Protection and Affordable Care Act (PPACA) of 2010. Meaningful Use requirements included in ARRA are fueling the uptake of electronic health records (EHRs) across the country, which provide more robust data than legacy systems . PPACA has led to a new focus on patient centered care and comparative effectiveness research through the Patient Centered Outcomes Research Institute (PCORI) . Biorepositories are being embraced that allow for the storage and analysis of tissue samples in order to understand the role of genetics and epigenetics within oncology and other diseases . Federated data systems like the Mini-Sentinel Network are being built that allow large-scale aggregation of data . The list of evolving opportunities is long and critically important, as data is the key substrate if we hope to make progress.
To date, gains in care efficiency and personalization from these movements remain limited, but it is still too early to judge their ultimate impact. It is often stated that health care has one of the lowest productivity gains from technology of any industry because of a lack of uptake of new tools and that it can take 17 years for new findings to enter routine clinical practice [9,10]. Driving forces behind this failure include misaligned incentives, a lack of shared understanding between the clinical community and the data scientists, deficiencies in infrastructure (both in care delivery and information technology), and reservations among the clinical community about the role of technology in the health care. A national focus on data, data sharing, biomarkers (of all types), and the integration of technology will start to break down the walls and catalyze progress towards personalized medicine, but it is critical that the clinical and research communities help to guide change instead of leaving it in the hands of industry and policy experts.
Perspective is important when defining the value of personalized medicine. Patients define health care value in terms of quality of the patient-physician relationship and alignment of care with personal goals . Oncologists and third party payers often focus on objective definitions of effectiveness and efficiency in specific clinical scenarios (e.g., impact of targeted treatment for BRAF V600E mutated melanoma on overall survival). Perhaps, within the personalized medicine community, we are so focused on biomarker driven care that we have missed the opportunity to define the near-term value statement for personalizing care within the context of patient-defined value and patient-centered care. One way to make progress is to demonstrate what has value and what does not by using accessible language and concepts with personal meaning. Outcomes such as progression free survival or time to event are abstract concepts, whereas more patient-centric outcomes like quality of life, ability to walk or run, and symptoms such as pain are more easily understood.
The role of biomarkers is often focused upon as the heart of personalized medicine. Over the past decade, many new discoveries have been made about the inherent genetic heterogeneity within previously defined groups such as those with lung or colorectal cancer . More recently, even treatments such as aspirin have found increasingly targeted populations such as its use with PIK3CA-mutated colorectal cancer . There are innumerable reviews of the role of various biomarkers such as KRAS in colorectal cancer  or BRAF in melanoma . The purpose of this discussion is not to focus on these issues, but instead to show how genomics can be one of many factors integrated into the new model of care delivery in order to improve understanding and outcomes. Instead of waiting for new genomic discoveries, we are working to build the systems that will integrate them quickly into practice and drive new discoveries.
In this manuscript, we will highlight our experience with personalized medicine in the research and clinical contexts. We will describe how the collection of patient reported information such as patient reported outcomes (PROs) has facilitated the inclusion of the patient voice in care and allowed for personalization in real-time in a manner easily understood by patients and the health care team. We will show how informatics-enabled registries that combine PROs and data from sources such as EHRs and biorepositories allow for the integration of the patient experience and biological factors in order to inform multivariate prediction models and to generate automated, tailored interventions and education. Finally, we highlight how new data can be used to enhance care through time-limited trials (TLTs) that facilitate communication between patients and clinicians. The basic framework that connects the different pieces is shown in Figure 1. Ultimately, new data approaches should be embraced and coordinated in order to better understand the impact of care on a patient’s experience and outcomes, providing new insights into value from both a patient-defined and ultimately societally-defined perspective. The pathways we describe highlight demonstration models of how the largely untold stories of personalized medicine are already moving the field forward using a unique approach that, on the surface, looks exceptionally different from biomarker driven prescription of anti-neoplastic therapies, but, in reality, uses the same principles of matching health care intervention to patient need based upon progressively more tailored real-time data.
2. Data Generation
New data sources are a critical piece of the puzzle. Without them, personalization will remain elusive. As examples, we will highlight how our work with PRO data collection systems and informatics-enabled registries has supported personalization.
2.1. Patient Reported Outcomes
PROs represent the collection of data on a patient’s health status directly from the patient, bypassing the traditional approach of interpretation and documentation by a clinician. Ample evidence shows that collection of PROs avoids misinterpretation by physicians and provides more robust data to be used across clinical care, research, and business operations [16,17,18]. Over the past decade, the Duke Center for Learning Health Care (CLHC) has been one of a handful of organizations to develop a robust PRO capture system, referred to as the electronic patient reported outcomes (ePRO) system [19,20,21]. Our most commonly used PRO instrument is the Patient Care Monitor v2.0 (PCM), because it simultaneously facilitates a clinical review of systems, patient reporting of day-to-day concerns, and research data collection. The PCM consists of 86 questions for women and 80 for men. It has been shown to take an average of 11 min to complete and is both feasible and reliable [20,22,23,24]. We have designed the ePRO platform to be accessible from any internet-enabled device, whether in the clinic (usually by tablet device) or outside the clinic (e.g., tablet, laptop, desktop, smartphone).
The ePRO system has been used at more than 30 academic and community oncology sites in the United States. Drivers of uptake of the ePRO system include financial justifications from improvements in clinical efficiency, a desire for greater patient engagement, and the need for new quality reporting and research capabilities. We have already seen the impact at sites using ePRO. For instance, the prevalence of sexual distress among our cancer patients was fully appreciated for the first time following implementation of the ePRO system at the Duke Cancer Institute. More than 30% of patients with breast, lung and gastrointestinal cancer were found to have moderate to severe sexual distress [25,26,27]. As a result of these findings, new educational and clinical care processes have been put in place and a randomized controlled trial opened to help understand how to address the issue. In another example, cancer pain treatments are being matched to people who need them in real time. Reports generated by the ePRO system highlight patients with pain as they present to the clinic; characteristics of the pain and its severity are documented, and the type and intensity of intervention is matched to the problem. On subsequent clinical encounters, the impact of the treatment choice is monitored using the ePRO system and therapies can then be further adjusted, using iterative cycles of assessment, targeted intervention, and review.
Many believe that implementation of such a system rests purely on the hardware and software tools involved. In our experience, these are among the easiest hurdles to overcome. One can relatively easily develop a software interface that allows patients to fill out clinical surveys. Much more challenging is the development of valid and reliable survey tools, the process redesign required in clinics to align the use of information systems within the clinical care being delivered, and the requirement for analytic capabilities that convert the raw data into actionable information that clinicians can utilize in real time. These stumbling blocks frequently relegate real-time PRO data collection and use to an “untold story” of personalized medicine, but, when surmounted, result in a clinical care delivery system that is highly patient-centric and directly fosters personalization of care.
2.2. Informatics Enabled-Registries
A registry is a collection of patient records assimilating longitudinal information that characterizes and monitors people included in the cohort . Typically registries have pre-specified criteria for patient inclusion, such as all people with a specific disease (e.g., stage IV cancer), exposed to a specific intervention (e.g., spine surgery), or receiving similar health care (e.g., accountable care organization). Outcomes are tracked over time, such as cancer-related survival, change in tumor burden, toxicity of anti-neoplastic therapies, symptoms, quality of life, and health resource utilization. In the case of biospecimens, the primary purpose of a registry can be to provide clinical annotation to document the phenotype that aligns with specific biology; or, the primary purpose of the registry can be focused on the clinical cohort and the biospecimen data can be just one data type among the overall dataset.
Informatics-enabled registries draw from many data sources to automatically aggregate and harmonize data in a centralized data warehouse, allowing far more robust queries and insights than would be possible from any single source. Architecturally, the warehouse can be a single organized database (albeit usually portioned into smaller units called “datamarts”), a loosely laced group of databases accessed when needed to create a virtual warehouse (also known as a “federated” model), or a combination of these models [28,29]. As an example at Duke, strides have been made to aggregate data across the EHR, tumor registry, billing, ePRO and clinical trials systems in order to design a database that can provide deeper insights into critical questions . These registries, once formed, augment all of the systems that they draw from. For instance, the Duke data warehouse acts as clinical annotation system for our biorepository, ensuring that the biological specimens are annotated with continuously aggregating data. Without the registry to bridge the two systems, it would take an overwhelming amount of work to match and analyze clinical data with biological data derived from the biospecimen. Importantly, the pool of data in the electronic data warehouse is continuously aggregating in line with the longitudinal evolution of day-to-day health care that naturally occurs. For example, a patient with newly diagnosed colon cancer might interact with the health care system and have data collected during his annual checkup, appointment with a gastroenterologist, screening colonoscopy, pathology’s review of the biopsy, appointment with an oncologist, and chemotherapy treatment visits. All of the health care data collected during each of these events is tracked in the registry and provides a longitudinal clinical picture that is matched to the cancer tissue samples that were collected. Data can be analyzed at any level of aggregation—institutional, across a network of sites, regional or national, depending upon whether adequate data dictionaries are available and information can flow from one system to the next.
In order to accomplish this, we have needed to work through a series of complex challenges such as concerns about data quality and the derivation of key variables. As this type of data aggregation represents a new and evolving territory, basic data quality concerns have required the development of systematic solutions. For instance, there is substantial variation in the likelihood that certain data fields are collected. If only the final result of an analysis is reported (i.e., 40% of patients are positive for a given mutation) then the nuances of the data are often unknown; in this example, only a small subset of patients may have had the mutation analyzed because known predictors of presence or absence of the mutation make routine biomarker testing unnecessary, thereby skewing the results. As another example of the challenges encountered with large-scale clinical data aggregation, data drawn from an ever-growing number of systems may result in conflicts among the data elements replicated across systems; for example, patient race may be recorded four different ways in eleven different medical data systems and the correct value must be defined. This arises for a number of reasons, including varying definitions of fields between sources and inaccuracies of input entry. Using models to predict the reliability of different sources, an algorithm can be developed to resolve such conflicts, choose the most likely appropriate answer, and add a “tag” to the data element (also known as metadata) indicating the likelihood that the data point is indeed correct. Another interesting and critical issue is the complex challenge of defining and populating critical variables that aren’t readily accessible within systems today, such as tumor response. While many data elements are critical to comparative effectiveness research, they are not always codified and are often buried in radiology reports and clinical notes. Electronic methods to harvest the data, such as prediction algorithms and natural language processing, may or may not be accurate and must be validated. Attempts to add more codified fields and to harness the growing computational power of networks to start sifting through the growing amount of data will help address the gap over time. Meanwhile, clinical chart review and completion of supplementary case report forms can serve as a backup method when needed.
As these complexities are resolved, informatics-enabled registries have great potential to provide critical data needed to support the practice of personalized medicine and personalized health care discoveries. Clinicians interacting with the registry can derive honed understanding of a patient’s risk and likelihood of response. Meanwhile data aggregating in the registry can support the transition of research from a linear to an iterative process. New biological discoveries can be tested in a succession of closely-linked clinical trials from phase I to phase III, put into practice if the findings are promising, and monitored for impact in the general population. Aggregating data from the general health care population then feeds the discovery engine for further innovation. In this way, informatics-enabled registries help to support the concept of learning health care. Collaborative high level stakeholder meetings, such as the Learning Health Summit by the Kaiser Family Foundation , Digital Learning Collaborative of the Institutes of Medicine (IOM) , and the IOM’s National Cancer Policy Forum are helping to shape a national data and infrastructure roadmap to move the field forward [33,34,35].
Once data are available, they can be used to support personalized medicine and tailored care while simultaneously augmenting our understanding of the quality, efficacy, and value of the care being delivered. Key features include data use and reuse, integration of the patient voice, and real-time decision-support.
3.1. Structured Support
Data collected in real-time can be used to support multiple actions directly intended to tailor care for the individual patient based upon patient-level data. For instance, we are developing the capability to deliver targeted education and structured interventions through the use of the ePRO system. Patients can access the system across settings, whether in the clinic waiting room or at home. Tailored education and interventions can then be provided, based on a variety of inputs including the patient’s diagnosis, clinical findings, and PROs. If a patient documents a concerning progression of pain or meets a predefined threshold for troublesome insomnia, a tailored video is delivered which provides personalized education regarding the symptom, self-management solutions, and directions to additional resources. Education received is documented in aggregating databases in order to meet clinical documentation and billing requirements. Activities occurring during what would previously have been down time for the patient (e.g., sitting in the clinic waiting room), enable increased insights for the patient and allow the provider to spend more time focusing on higher-level issues. We hypothesize that improved patient education can improve outcomes and this is currently being studied.
Similarly, systems are being developed in which PRO responses, coupled with additional data points, prompt automated interventions from clinical staff. Instead of waiting for a patient’s pain to progress to the point of an unplanned clinic visit or hospitalization, interventions will be proactively triggered. This might consist of a call from a clinician or a visit from a hospice nurse. In another solution being tested, PRO responses are triggering behavioral medicine interventions delivered via video-conferencing on mobile devices to the patients who are most likely to derive benefit (e.g., people with pain scores >3 and positive sense of self efficacy).
3.2. Clinical Decision Support
Clinical decision support (CDS) refers to the analysis and presentation of aggregated data to provide guidance to clinicians to tailor care for the individual patient. Numerous studies have outlined the benefits of clinical checklists [36,37,38]. CDS is a logical extension of this approach, meant to help provide reminders and guidance about the latest and most evidence-based care patterns. A key feature is the matching of patient-level data to current best evidence and practices, which can accelerate diffusion of new evidence . A common concern of randomized clinical trials is their limited applicability to routine clinical practice settings . CDS, driven by real time clinical findings, can fundamentally change this paradigm.
There are innumerable types of CDS systems in development. Let us provide an example of their use to support care pathways and personalized medicine today. Individual patient data can be used to make personalized cancer treatment recommendations such as advising vemurafenib for a person with metastatic melanoma and a BRAF V600E mutation. Now imagine a panel of patients all of whom have melanoma with the BRAF V600E mutation and who receive vemurafenib; their data are aggregating in the informatics-enabled registry. Perhaps a cluster of patients in the panel is noted to develop peripheral neuropathy. A multivariable prediction model is derived using the registry data including information on the sites of disease involvement, lab variations, and genomic markers to identify people at risk of developing the peripheral neuropathy. A cluster of early symptoms serves as an early warning. By inputting a person’s personal characteristics into the multivariable model, a risk score for development of peripheral neuropathy is generated. When the telltale symptom cluster starts to develop in a person at high risk, the vemurafenib is stopped; if the person is low risk, other reasons for the symptom cluster might be sought before dose adjustments to the vemurafenib are made. The ability to use aggregating registries to generate continuously updating prediction models that allow continual refinement of the evidence being used to drive care is an exciting opportunity. While pilot projects must be very carefully controlled to prevent unnecessary risks to patients, we believe that combining these insights with data about the patient’s personal preferences will help us to determine what to do in the context of patient defined goals and values.
Initial systematic reviews have shown the ability of CDS to improve practice patterns but have not yet shown an impact on outcomes [40,41]. This may be largely due to limitations of the available CDS systems studied. CDS is an inevitable part of the future of medical care, so it is in the best interest of clinicians to be involved early and actively in the development process to ensure that, instead of complicating and confining practices, CDS systems provide benefit.
4. Time Limited Trials
Another near-term application of the data is through its use in Time Limited Trials (TLTs). TLTs have been gaining recent momentum in which patients and clinicians use a structured process to agree on a management approach [42,43]. As described in a recent manuscript in the Journal of the American Medical Association, “a TLT is an agreement between clinicians and a patient/family to use certain medical therapies over a defined period to see if the patient improves or deteriorates according to agreed upon outcomes” . Far too often in oncology, as in other areas of medicine, we choose to continue therapies in an open ended fashion, with no clearly established decision points or acknowledgment about a lack of efficacy. TLTs are an attempt to deal with this divide, and are a critical part of personalization of care at the bedside.
At the outset of therapy, both parties discuss their goals from care. What are the patient’s/ caregiver’s primary values and how can this knowledge be used to direct care? Tools like the ePRO system can help to gather the necessary information to both define goals and document the ongoing balance of risks and benefits. The ePRO system provides an objective tool with which to assess the outcomes that are decided upon such as changes in pain control or functional status. Paired with data from the patient record about trends in lab values (supplied through the informatics-enabled registry), robust systems can be developed to support TLTs and better define the course of care. For instance, a patient with advanced prostate cancer may decide to start docetaxol therapy and agree to a trial of three months. Severe toxicity over that time may prematurely end therapy; agreement at the outset as to his health care goals helps to guide decisions. If a patient’s primary concern is the balance between loss of functional status and the need to shrink the disease, there is a likely a tradeoff he is willing to make. If at the end of the three months, functional status has declined significantly and tumor response is limited as defined by the PSA test and radiographic changes, a discussion of alternative therapies would be appropriate, as previously agreed. Alignment of goals and outcomes empowers the patient. While there is nothing set in stone about the agreement that is reached, it allows patients to be active participants, and leverages the use of data sources to truly support care. Instead of subjectively following functional status, it can now be collected as a data point, explicitly incorporated into the decision-making algorithm and approach to care, and monitored over time.
The “untold stories about personalized medicine” are exciting. They are about progress on the ground in areas like the collection of PROs, aggregation of data across various sources, and the use of novel data analyses in the provision of care and generation of new evidence. There is angst about the effect of new information technology systems on efficiency and autonomy. The time available in the clinic room is limited. The use of technology to restructure care, improve efficiency, and match the right intervention to the right patient at the right time is critical. This is in direct opposition to the initial introduction of technology into clinics to date, in which it merely replicated the paper chart and negatively impacted clinical flow . But, if done wisely, potential gains to patients and the system through personalization of care far outweigh the drawbacks. Instead of interrupting the clinical flow, discrete data collection systems will allow clinicians to focus on higher-order issues and support personalization of care.
Adoption of the PDSA (plan-do-study-act) cycle framework is an important component of this process. As problems are identified, solutions are chosen that must then be applied, tested and updated in an iterative fashion. Each new analysis is planned within routine clinical care, data is efficiently gathered about the outcomes of the care being delivered, results are studied and then acted on through CDS, guideline updates, and other real-time mechanisms. In this process, the care of patients is continuously updated with the best available evidence. While not a new idea, the PDSA cycle has not been routinely applied, largely due to the lack of an available infrastructure . Instead of always relying on the results of robust trials that can take hundreds of millions of dollars and many years to conduct, insights can now be generated quickly within new data systems and iteratively tested as part of a learning health system [46,47,48]. There are a number of alternative names for PDSA cycles, such as a process recently described in Annals of Internal Medicine as design-implement-evaluate-adjust , but all have the same principles. It is time that those providing medical care embrace the ability of technology to move the field forward.
The new approaches we have discussed end in the discussion of value. Value is variably defined depending on the stakeholder. For oncology patients, the value equation may include aspects of care such as the patient experience, response rate to treatments, symptoms, and quality of life. For payers, value may be defined by the balance of cost, utilization of resources, and impact on symptoms/ survival. Many have discussed the notion of value and it remains difficult to define. The greatest potential of the new approaches outlined in the manuscript is in their ability to objectively begin to characterize value on a large scale and enable a far more informed debate about how to optimize and prioritize care.
Conflict of Interest
Bradford R. Hirsch has research funding from Academy Health, the Robert Wood Johnson Foundation, and the Center for Medical Technology Policy. Bradford R. Hirsch holds equity in Eagle Hospital Physicians and Pioneer Surgical.
Amy P. Abernethy has research funding from the US National Institutes of Health, US Agency for Healthcare Research and Quality, Robert Wood Johnson Foundation, Biovex, DARA, Helsinn, MiCo and Pfizer; these funds are all distributed to Duke University Medical Center to support research including salary support for Amy P. Abernethy. In the last 2 years she has had nominal consulting agreements with or received honoraria from (<$5,000 annually) Novartis and Pfizer. Consulting with Bristol Meyers Squibb is pending in 2012, for role as Co-Chair of a Scientific Advisory Committee. In 2013, Amy P. Abernethy will have paid leadership roles with American Academy of Hospice & Palliative Medicine (President-Elect) and Pillars4Life (Medical Director). She has corporate leadership responsibility in Advoset (an education company that has a contract with Novartis) and Orange Leaf Associates LLC (an IT development company).
References and Notes
- FDA Approved Drugs by Therapeutic Area. Available online: http://www.centerwatch.com/drug-information/fda-approvals/drug-areas.aspx?AreaID=12 (accessed on 12 August 2012).
- Arteaga, C.L.; Baselga, J. Impact of genomics on personalized cancer medicine. Clin. Cancer Res. 2012, 18, 612–618, doi:10.1158/1078-0432.CCR-11-2019.
- Collins, F. Has the revolution arrived? Nature 2010, 464, 674–675.
- Rosenfeld, J.A.; Mason, C.E.; Smith, T.M. Limitations of the human reference genome for personalized genomics. PLoS One 2012, 7, e40294.
- Blumenthal, D.; Tavenner, M. The “meaningful use” regulation for electronic health records. N. Engl. J. Med. 2010, 363, 501–504, doi:10.1056/NEJMp1006114.
- Selby, J.V.; Beal, A.C.; Frank, L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA 2012, 307, 1583–1584.
- Elliott, P.; Peakman, T.C. The UK Biobank sample handling and storage protocol for the collection, processing and archiving of human blood and urine. Int J. Epidemiol. 2008, 37, 234–244, doi:10.1093/ije/dym276.
- Behrman, R.E.; Benner, J.S.; Brown, J.S.; McClellan, M.; Woodcock, J.; Platt, R. Developing the Sentinel System—A national resource for evidence development. N. Engl. J. Med. 2011, 364, 498–499, doi:10.1056/NEJMp1014427.
- Balas, E.; Boren, S. Managing clinical knowledge for health care improvement. Yearb. Med. Inform. 2000, 9, 65–70.
- Blumenthal, D. Stimulating the adoption of health information technology. N. Engl. J. Med. 2009, 360, 1744–1479.
- Institute of Medicine. Value in Health Care: Accounting for Cost, Quality, Safety, Outcomes, and Innovation: Workshop Summary; The National Academies Press: Washington, D.C., USA, 2010.
- Ogino, S.; Fuchs, C.S.; Giovannucci, E. How many molecular subtypes? Implications of the unique tumor principle in personalized medicine. Expert Rev. Mol. Diagn. 2012, 12, 621–628, doi:10.1586/erm.12.46.
- Liao, X.; Lochhead, P.; Nishihara, R.; Morikawa, T.; Kuchiba, A.; Yamauchi, M.; Imamura, Y.; Qian, Z.R.; Baba, Y.; Shima, K.; et al. Aspirin use, tumor PIK3CA mutation, and colorectal-cancer survival. N. Engl. J. Med. 2012, 367, 1596–1606, doi:10.1056/NEJMoa1207756.
- Febbo, P.G.; Ladanyi, M.; Aldape, K.D.; De Marzo, A.M.; Hammond, M.E.; Hayes, D.F.; Iafrate, A.J.; Kelley, R.K.; Marcucci, G.; Ogino, S.; et al. NCCN Task Force report: Evaluating the clinical utility of tumor markers in oncology. J. Natl. Compr. Canc. Netw. 2011, 9, S1–S32.
- De Roock, W.; De Vriendt, V.; Normanno, N.; Ciardiello, F.; Tejpar, S. KRAS, BRAF, PIK3CA, and PTEN mutations: Implications for targeted therapies in metastatic colorectal cancer. Lancet Oncol. 2011, 12, 594–603.
- Basch, E.; Jia, X.; Heller, G.; Barz, A.; Sit, L.; Fruscione, M.; Appawu, M.; Iasonos, A.; Atkinson, T.; Goldfarb, S.; et al. Adverse symptom event reporting by patients vs. clinicians: Relationships with clinical outcomes. J. Natl. Cancer Inst. 2009, 101, 1624–1632, doi:10.1093/jnci/djp386.
- Cleeland, C.S.; Sloan, J.A. Assessing the Symptoms of Cancer Using Patient-Reported Outcomes (ASCPRO): Searching for standards. J. Pain Symptom Manage. 2010, 39, 1077–1085, doi:10.1016/j.jpainsymman.2009.05.025.
- Pakhomov, S.V.; Jacobsen, S.J.; Chute, C.G.; Roger, V.L. Agreement between patient-reported symptoms and their documentation in the medical record. Am. J. Manag. Care 2008, 14, 530–539.
- Abernethy, A.P.; Ahmad, A.; Zafar, S.Y.; Wheeler, J.L.; Reese, J.B.; Lyerly, H.K. Electronic patient-reported data capture as a foundation of rapid learning cancer care. Med. Care 2010, 48, S32–S38, doi:10.1097/MLR.0b013e3181db53a4.
- Abernethy, A.P.; Herndon, J.E., 2nd.; Wheeler, J.L.; Day, J.M.; Hood, L.; Patwardhan, M.; Shaw, H.; Lyerly, H.K. Feasibility and acceptability to patients of a longitudinal system for evaluating cancer-related symptoms and quality of life: Pilot study of an e/Tablet data-collection system in academic oncology. J. Pain Symptom Manage. 2009, 37, 1027–1038, doi:10.1016/j.jpainsymman.2008.07.011.
- Abernethy, A.P.; Herndon, J., 2nd.; Wheeler, J.L.; Patwardhan, M.; Shaw, H.S.; Lyerly, H.K.; Weinfurt, K. Improving health care efficiency and quality using tablet personal computers to collect research-quality, patient-reported data. Health Serv. Res. 2008, 43, 1975–1991, doi:10.1111/j.1475-6773.2008.00887.x.
- Suh, S.Y.; Leblanc, T.W.; Shelby, R.A.; Samsa, G.P.; Abernethy, A.P. Longitudinal patient-reported performance status assessment in the cancer clinic is feasible and prognostic. J. Oncol. Pract. 2011, 7, 374–381, doi:10.1200/JOP.2011.000434.
- Abernethy, A.P.; Zafar, S.Y.; Uronis, H.; Wheeler, J.L.; Coan, A.; Rowe, K.; Shelby, R.A.; Fowler, R.; Herndon, J.E., 2nd. Validation of the Patient Care Monitor (Version 2.0): A review of system assessment instrument for cancer patients. J. Pain Symptom Manage. 2010, 40, 545–558, doi:10.1016/j.jpainsymman.2010.01.017.
- Abernethy, A.P.; Herndon, J.E., 2nd.; Coan, A.; Staley, T.; Wheeler, J.L.; Rowe, K.; Smith, S.K.; Shaw, H.; Lyerly, H.K. Phase 2 pilot study of Pathfinders: A psychosocial intervention for cancer patients. Support. Care Cancer 2010, 18, 893–898.
- Dupont, A.; Wheeler, J.; Herndon, J.E., 2nd.; Coan, A.; Zafar, S.Y.; Hood, L.; Patwardhan, M.; Shaw, H.S.; Lyerly, H.K.; Abernethy, A.P. Use of tablet personal computers for sensitive patient-reported information. J. Support. Oncol. 2009, 7, 91–97.
- Reese, J.B.; Keefe, F.J.; Somers, T.J.; Abernethy, A.P. Coping with sexual concerns after cancer: The use of flexible coping. Support. Care Cancer 2010, 18, 785–800, doi:10.1007/s00520-010-0819-8.
- Reese, J.B.; Shelby, R.A.; Keefe, F.J.; Porter, L.S.; Abernethy, A.P. Sexual concerns in cancer patients: A comparison of GI and breast cancer patients. Support. Care Cancer 2010, 18, 1179–1189, doi:10.1007/s00520-009-0738-8.
- Gliklich, R.E.; Dreyer, N.A.E. Registries for Evaluating Patient Outcomes: A User’s Guide, 2nd ed., (Prepared by Outcome DEcIDE Center [Outcome Sciences, Inc. d/b/a Outcome] under Contract No. HHSA29020050035I TO3.); Agency for Healthcare Research and Quality: Rockville, MD, USA, 2010.
- Maro, J.C.; Platt, R.; Holmes, J.H.; Strom, B.L.; Hennessy, S.; Lazarus, R.S.; Brown, J.S. Design of a national distributed health data network. Ann. Intern. Med. 2009, 151, 341–344.
- Horvath, M.M.; Winfield, S.; Evans, S.; Slopek, S.; Shang, H.; Ferranti, J. The DEDUCE Guided Query tool: Providing simplified access to clinical data for research and quality improvement. J. Biomed. Inform. 2011, 44, 266–279.
- Kanter Health Learning Health System Summit. Available online: http://kanterhealth.org/featured/2012-summit/ (accessed on 12 August 2012).
- Institute of Medicine Digital Learning Collaborative: Advancing the Digital Infrastructure for the Learning Health System. Available online: http://www.iom.edu/Activities/Quality/VSRT/~/ media/Files/Activity Files/Quality/VSRT/Core Documents/Digital Learning Collaborative.pdf (accessed on 12 August 2012).
- Institute of Medicine. Achieving Data Liquidity in the Cancer Community: Proposal for a coAlition of Stakeholders; The National Academies Press: Washington, D.C., USA, 2012.
- Institute of Medicine. Informatics Needs and Challenges in Cancer Research: Workshop Summary; The National Academies Press: Washington, D.C., USA, 2012.
- Institute of Medicine. The Learning Healthcare System: Workshop Summary (IOM Roundtable on Evidence-Based Medicine); The National Academies Press: Washington, D.C., USA, 2007.
- Hales, B.M.; Pronovost, P.J. The checklist—A tool for error management and performance improvement. J. Crit. Care 2006, 21, 231–235, doi:10.1016/j.jcrc.2006.06.002.
- Pronovost, P.J.; Needham, D.; Berenholtz, S.; Sinopoli, D.; Chu, H.; Cosgrove, S.; Sexton, B.; Hyzy, R.; Welsh, R.; Roth, G.; et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N. Engl. J. Med. 2006, 355, 2725–2732, doi:10.1056/NEJMoa061115.
- Sawyer, M.; Weeks, K.; Goeschel, C.A.; Thompson, D.A.; Berenholtz, S.M.; Marsteller, J.A.; Lubomski, L.H.; Cosgrove, S.E.; Winters, B.D.; Murphy, D.J.; et al. Using evidence, rigorous measurement, and collaboration to eliminate central catheter-associated bloodstream infections. Crit. Care Med. 2010, 38, S292–S298.
- Dreyer, N.A.; Tunis, S.R.; Berger, M.J.; Ollendorf, D.; Mattox, P.; Gliklich, R. Why observational studies should be among the tools used in comparative effectiveness research. Health Aff. 2010, 29, 1818–1825, doi:10.1377/hlthaff.2010.0666.
- Kawamoto, K.; Houlihan, C.A.; Balas, E.A.; Lobach, D.F. Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. BMJ 2005, doi:10.1136/bmj.38398.500764.8F.
- Wolfstadt, J.I.; Gurwitz, J.H.; Field, T.S.; Lee, M.A.; Kalkar, S.; Wu, W.; Rochon, P.A. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: A systematic review. J. Gen. Intern. Med. 2008, 23, 451–458, doi:10.1007/s11606-008-0504-5.
- Abernethy, A.P.; Currow, D. Time-Limited Trials (letter to editor). JAMA 2012, 307, 33–34.
- Quill, T.E.; Holloway, R. Time-limited trials near the end of life. JAMA 2012, 306, 1483–1484.
- Poissant, L.; Pereira, J.; Tamblyn, R.; Kawasumi, Y. The impact of electronic health records on time efficiency of physicians and nurses: A systematic review. J. Am. Med. Inform. Assoc. 2005, 12, 505–516, doi:10.1197/jamia.M1700.
- Cleghorn, G.D.; Headrick, L.A. The PDSA cycle at the core of learning in health professions education. Jt. Comm. J. Qual. Improv. 1996, 22, 206–212.
- Friedman, C.P.; Wong, A.K.; Blumenthal, D. Achieving a nationwide learning health system. Sci. Transl. Med. 2010, 2, doi:10.1126/scitranslmed.3001456.
- Miriovsky, B.J.; Shulman, L.L.; Abernethy, A.P. Importance of Health Information Technology, Electronic Health Records, and Continuously Aggregating Data to Comparative Effectiveness Research and Learning Health Care. J. Clin. Oncol. 2012, 30, 4243–4248, doi:10.1200/JCO.2012.42.8011.
- Abernethy, A.P.; Etheredge, L.M.; Ganz, P.A.; Wallace, P.; German, R.R.; Neti, C.; Bach, P.B.; Murphy, S.B. Rapid-learning system for cancer care. J. Clin. Oncol. 2010, 28, 4268–4274.
- Greene, S.M.; Reid, R.J.; Larson, E.B. Implementing the learning health system: From concept to action. Ann. Intern. Med. 2012, 157, 207–210.
© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).