Next Article in Journal / Special Issue
Under-Detection of Lyme Disease in Canada
Previous Article in Journal
The Extremely High Adsorption Capacity of Fluoride by Chicken Bone Char (CBC) in Defluoridation of Drinking Water in Relation to Its Finer Particle Size for Better Human Health
Previous Article in Special Issue
Systematic Approach to the Diagnosis and Treatment of Lyme Carditis and High-Degree Atrioventricular Block
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Removing the Mask of Average Treatment Effects in Chronic Lyme Disease Research Using Big Data and Subgroup Analysis

MyLymeData, Chico, CA 95927, USA
Analytic Designers LLC., Bethesda, MD 20817, USA
Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA 98195, USA
Author to whom correspondence should be addressed.
Healthcare 2018, 6(4), 124;
Submission received: 4 September 2018 / Revised: 9 October 2018 / Accepted: 9 October 2018 / Published: 12 October 2018


Lyme disease is caused by the bacteria borrelia burgdorferi and is spread primarily through the bite of a tick. There is considerable uncertainty in the medical community regarding the best approach to treating patients with Lyme disease who do not respond fully to short-term antibiotic therapy. These patients have persistent Lyme disease symptoms resulting from lack of treatment, under-treatment, or lack of response to their antibiotic treatment protocol. In the past, treatment trials have used small restrictive samples and relied on average treatment effects as their measure of success and produced conflicting results. To provide individualized care, clinicians need information that reflects their patient population. Today, we have the ability to analyze large data bases, including patient registries, that reflect the broader range of patients more typically seen in clinical practice. This allows us to examine treatment variation within the sample and identify groups of patients that are most responsive to treatment. Using patient-reported outcome data from the MyLymeData online patient registry, we show that sub-group analysis techniques can unmask valuable information that is hidden if averages alone are used. In our analysis, this approach revealed treatment effectiveness for up to a third of patients with Lyme disease. This study is important because it can help open the door to more individualized patient care using patient-centered outcomes and real-world evidence.

1. Introduction

Lyme disease, caused by the spirochete Borrelia burgdorferi and transmitted via tick bite, is the most common vector-borne disease in the United States. The Centers for Disease Control and Prevention (CDC) estimates that 300,000 new cases of Lyme disease occur annually [1]. Although most patients who are diagnosed and treated early are restored to health, treatment failures ranging from 10% to 35% have been reported, and many patients are not diagnosed until later in the disease when treatment success is much harder to achieve [2,3,4,5,6,7,8,9]. Compared to other diseases, very little research has been conducted on how best to treat patients who do not respond to short-term treatment approaches [10].
Different nomenclature has been used to describe this group of patients. In this paper, we use the term chronic Lyme disease, which is the name most commonly used by patients and their treating clinicians [11,12,13]. The term “Post Treatment Lyme Disease Syndrome” (PTLDS), which was first proposed in 2006, incorporates the restrictive CDC surveillance criteria. However, as the CDC notes, the goals of surveillance and clinical diagnosis are very different [14,15]. While restrictive case definitions may be useful for surveillance or for specific research purposes, they exclude a large portion of the clinical population of patients whose symptoms persist despite their having received some antibiotic therapy. Hence, surveillance-based definitions should not be used by healthcare providers for making a clinical diagnosis, and the generalizability of definitions based on these criteria to the clinical population is limited [15].
Just three research studies have been funded by the National Institutes of Health (NIH) on the retreatment of chronic Lyme disease, and the last retreatment trial was funded over 15 years ago [16,17,18]. The patients enrolled in these three randomized controlled trials were from highly selective and small samples. The trials produced conflicting results and relied on average treatment effects as their measure of success. This has created considerable uncertainty in the treatment of patients with chronic Lyme disease.
While randomized controlled trials and average treatment effects contribute to our knowledge base, they can also mask important information that can provide insights into the range of treatment responses existing within a patient group. For example, although it has long been contended that tick-borne coinfections are rare in patients with Lyme disease [19], coinfections are common in patients with persistent Lyme disease symptoms [5]. One reason patients may develop chronic disease is because they are coinfected with other tick-borne pathogens. A better understanding of the difference between acute and chronic Lyme disease characteristics could identify patients with a heightened risk of developing chronic disease, and awareness of these risk factors might prompt more aggressive treatment in those patients—towards a goal of preventing the development of chronic Lyme disease.
Using a big data sample of almost 4000 people diagnosed with Lyme disease from the MyLymeData patient registry developed by, we looked at responses to a widely used global rating of change (GROC) scale to assess the degree to which patients reported that their condition improved, worsened, or remained unchanged following antibiotic treatment. The survey questions used were both patient-centered and reflective of the types of questions posed by clinicians in practice. To study treatment response, we created subgroups of patient-reported changes in the condition of their health. Our findings indicated that more than a third of people with Lyme disease in this group of patients were “high responders” to antibiotic treatment, suggesting the potential value of subgroup analysis as a standard component of future study designs.
Our big data sample included the experience of a much broader group of patients than were included in previous Lyme disease clinical trials [11,12,13]. In addition, as the central participants in the healthcare process, patients can draw upon many sources of personal health data that are traditionally locked in separate electronic health records, insurance records, clinician notes, or research silos. Thus, data collected from patients, while self-reported (and thus to some degree subjective), is also more complete than many of the other sources. This approach complements randomized controlled trials (RCTs), which can provide the high internal validity essential to demonstrate cause and effect but may do so at the expense of generalizability to clinical care of individual patients [20,21]. This is because as a research design, randomized controlled trials may employ highly restrictive entry criteria to eliminate factors that could be confounding (such as other potential causal agents for change) and rely on average treatment effects to demonstrate treatment effectiveness. This inherently limits the generalizability of these types of trials to the clinical population, with some trials excluding up to 95% of people who might use the intervention [20]. In contrast, our approach not only encourages broader study participation, which increases the applicability of the results to the patient population seen clinically, but also produces larger sample sizes that enable the type of robust subgroup analysis necessary to identify treatment response variation within the group.
Our subgroup analysis is important because it can help open the door to a more personalized approach to patient care. In healthcare, the primary goal is to improve outcomes that are important to individual patients. Clinicians are less interested in what works for the average patient than what is likely to work for their particular patient given the patient history, severity of presentation, comorbidities, treatment responsiveness, and ability to tolerate treatment side effects [20]. Clinicians require information about real-world outcomes that reflect the heterogeneity of treatment response typically seen in clinical care. Subgroup analysis of big data samples is therefore important as it may help facilitate a more personalized approach to treatment of Lyme disease patients [20]. Over the past ten years, the technological tools required to pool large amounts of data have become faster, cheaper, and more readily available.

2. Materials and Methods

2.1. Data Sources

This study analyzed the responses of those who completed Phase 1 of the MyLymeData project. Survey items used in Phase 1 were drawn from the Agency for Healthcare Research and Quality’s (AHRQ, Rockville, MD, USA) recommended data elements for registries, prior surveys, and peer-reviewed published literature for Lyme disease and other conditions [4,5,21,22]. (Appendix A generally describes registry items included in MyLymeData). We also relied heavily on survey questions derived from standard government question banks, such as the CDC Behavioral Risk Factor Surveillance System, National Health Interview Survey, National Ambulatory Medical Care Survey, and National Center for Health Statistics as well as the AHRQ Medical Expenditure Panel Survey [23,24,25,26,27,28,29]. Specific items, including the GROC scale to assess whether a patient’s health condition has improved or deteriorated over time, were based on published studies using GROC as discussed further below [30,31,32]. The beta version of the MyLymeData survey was pilot-tested and adjusted as recommended by AHRQ.
To promote participation in the survey, various recruitment strategies were utilized, including blogs and social media as well as professional conference presentations about the registry. Participation in the registry is voluntary, and all respondent identities remain strictly confidential. The registry and survey was approved by the Advarra Institutional Review Board at the time it was launched. Analysis of the survey data (see Table S1) was exempted from review by the Washington University Institutional Review Board (IRB) because none of the data used for this study contained personally identifiable information. SPSS and SAS® JMP Pro® 13.1.0 were used to conduct this analysis.

2.2. Study Participants

Unlike randomized trials, which have highly restrictive entry criteria, the enrollment criteria for patient registries is generally expansive and includes US patients who self-report being diagnosed with a particular condition [33]. Within the patient registry sample, further patient selection can be done by stratifying patients at risk of disease progression (e.g., stage of illness at diagnosis or presence of coinfections) or by diagnostic validation criteria [5]. Commonly used Lyme disease diagnostic validation criteria include exposure, recollection of a tick bite, presence of an erythema migrans (EM) rash, signs or symptoms consistent with Lyme disease, positive laboratory tests, and clinician diagnosis. Because the goal of this study was to establish the usefulness of GROC scales as a proof of concept to describe treatment heterogeneity and identify treatment responders, this study focused on patients who had been diagnosed with Lyme disease by a healthcare provider, regardless of diagnostic method. Future studies could confirm these findings with a more restrictive patient sample. For example, a previous study by the authors examined survey respondents who either had an EM rash or two-tiered CDC positive testing [5]. Appendix B lists additional criteria used in a clinical research definition of chronic Lyme disease that might be used to create a more strictly characterized research sample [34].
A total of 5872 patients had enrolled in Phase 1 of the registry study, which had a rolling enrollment period. Patients who had either not completed their registration or had not completed all their Phase 1 surveys were rolled forward into the Phase 2 survey when it launched. A total of 4719 had completed registration and all surveys for Phase 1.
In addition to completing the Phase 1 surveys, participants were required to answer “yes” to questions in the baseline survey confirming that they were residents of the US and that they had been diagnosed with Lyme disease by a healthcare provider.
At the end of the baseline survey, participants classified their current health status as either well or unwell and were given an additional survey based on their response to that question. After completing the baseline survey, participants were required to complete either the unwell or well survey.
Hence, the preliminary sample of 4719 participants who had completed all required surveys was subsequently reduced to 3903 after eliminating duplicate participants (through their unique registration identification number) as well as those who did not designate a US state of residency or report diagnosis of Lyme disease by a healthcare provider. Of the final sample, 3556 completed the unwell survey and 447 completed the well survey.
Figure 1 shows the original sample of 4719 respondents and the exclusion criteria that led to a final sample of 3903 subjects. The 3903 participants used in this analysis included US residents clinically diagnosed with Lyme disease who had completed the Phase 1 survey and the unwell or well survey assigned to them.
Table 1 shows the demographic characteristics of the resulting group of patients. The sample here was predominantly female (83%), and the average age was 49. This sample also skewed more towards higher education and income levels than the general population. Note that while all participants completed the baseline (which covered topics such as diagnosis, tick exposure, and early symptoms), only those patients who reported their status as unwell were served the GROC survey questions and were asked to identify their current three worst symptoms. Of these 3556 respondents, 20 failed to complete the GROC survey questions and were excluded in that analysis.
Table 2 below shows the current stage of illness of those in the sample as well as some of their diagnostic characteristics. These diagnostic characteristics and stage of illness criteria might be used to further refine a sample to conform to a recently proposed research definition of chronic Lyme disease [34]. (See Appendix B).

2.3. Global Rating of Change Scale

An important focus of our work was to take a patient-centered approach. Outcome measures that are patient-centered reflect the patient’s perspective on response to treatment. In Lyme disease, these include restoration of health, prevention of health deterioration, ability to engage in work and other activities, and improvement in quality of life [28]. GROC scales allow the patients themselves to determine what factors are important in assessing their condition [31]. Because they are useful in observational trials, clinical practice, and randomized controlled trials, they provide a bridge of interoperability among these three critical arenas. GROC scales also make it possible to employ a highly granular approach that can help differentiate treatment responses within a group.
GROC scales are widely used in both research and clinical practice [35,36]. The face validity of GROC scales is high as they are intuitive and easy to understand for patients, clinicians, and researchers alike [37]. They can also be used to collect treatment response data from a variety of sources, including patient registries, clinicians, and researchers with high levels of concordance and allow an anchoring point of time for comparison that can lend itself to different treatment durations [20]. Scores obtained using GROC scales have been found to correlate with pain, disability, and quality of life measures [38].
GROC scales ask patients to identify whether their symptoms are better, worse, or unchanged from a previous point in time and utilize a Likert scale ranging from 7 to 17 points to assess the granularity of treatment response [30,35]. Highly granular outcome measures permit treatment response to be categorized more accurately [38]. Outcome measures also provide objective measurement of treatment response for clinical trials and may help clinicians monitor treatment effectiveness and allow researchers and clinicians to predict which patients will benefit most from a particular intervention [38].
We used a common 17-point Likert scale to explore the potential use of this approach to unmask subgroup variation of treatment response compared to the average treatment effect [36]. To make the scale more patient-centered, we followed the lead of an earlier study that asked the question in a two-step process [32]. First, patients were given the choice of responding “better”, “worse”, or “unchanged” to the statement “In general overall, I would say that with antibiotic therapy, my Lyme symptoms are ___.” Those who responded “better” or “worse” were asked a follow-up question regarding the magnitude of change perceived by the patient on an 8-point scale. A 17-point scale presented as one question would likely be difficult for a patient to answer, while the two-step question approach used here reduced that cognitive load.
The end product of this two-step or branched question sequence was a 17-point Likert scale, ranging from −8 to +8, with the unchanged midpoint pegged at 0. For example, compared with no change, those responding “better” would select a magnitude ranging from (a) “almost the same”, (b) “hardly better at all”, (c) “a little better”, (d) “somewhat better”, (e) “moderately better”, (f) “a good deal better”, (g) “a great deal better”, and (h) “a very great deal better”. Those responding “worse” would choose among similar responses in the negative (e.g., “hardly worse at all” to “a very great deal worse”) [32].
In our responses, very few patients (2%) selected “almost the same”, and the distribution was evenly split between those who reported being better or worse (1% each). In conformity with other studies, we elected to include the response “almost the same” in the unchanged category, thereby reducing the final scale to a 15-point scale ranked +1 to +7, ranging from “hardly better to all” to “a very great deal better” on the positive end of the scale and −1 to −7 on the negative end of the scale for comparable categories [30].
We additionally grouped these responses as Better, Unchanged, and Worse and further characterized those responding as High Responders (score between 4 and 7), Low Responders (score between 1 and 3), and Nonresponders (score between −7 and 0). We chose a cut-score of +4 or greater to define High Responders based on the assumption that most patients would regard a treatment response of “moderately better” to “a very great deal better” as an important improvement.
Prior work suggests that the appropriate cut-off to use is disease-specific, with cut-offs ranging from +1 to >+3 or greater based on the amount of change regarded as important to patients and clinicians [30]. Some studies ask patients to specifically identify what degree of improvement they regard as the minimally important amount of change or minimally important clinical difference [32]. Indeed, the scale we used was derived from a study by Jaeschke and Guyatt to ascertain minimal clinically important difference (MCID) [36]. This study did not seek to establish the MCID for Lyme disease, although future studies may do so.

3. Results

3.1. General Observations

Misdiagnosis and delayed diagnosis were recurring themes in our sample, which consisted predominately of patients with chronic Lyme disease (61%) who reported having remained ill for six or more months after 10–21 days of antibiotic treatment. More than half (51%) reported that it took them more than three years to be diagnosed and roughly the same proportion (54%) saw five or more clinicians before diagnosis. These diagnostic delays occurred despite the fact that 45% of participants reported early symptoms of Lyme disease within days to weeks of exposure. Causes of identified diagnostic delays included false negative lab tests (37%) or positive test results that were dismissed as “false-positives” (13%). While delayed diagnosis appears to be common in those with chronic Lyme disease, the extent of the problem for the full disease spectrum is unknown. However, some researchers suggest that delayed diagnosis may occur in as many as 40% of cases [3].
The majority of patients (72%) reported being misdiagnosed with another condition prior to their Lyme diagnosis. Of those misdiagnosed, the most common misdiagnosis was psychiatric disorder (52%), followed by fibromyalgia (43%), followed by chronic fatigue (42%). Although misdiagnosis with a psychiatric illness was common (52%), only (18%) reported psychiatric symptoms as being among their three worst symptoms.
The most frequently reported three worst symptoms included neurologic-associated symptoms (84%) and fatigue (62%), followed by musculoskeletal-associated symptoms (57%). Neurologic symptoms included cognitive impairment, sleep impairment, memory loss, psychiatric manifestations, headaches, neuropathy, and twitching, with cognitive impairment (30%) and neuropathy (29%) most often reported among those with neurologic symptoms. Musculoskeletal symptoms included muscle aches and joint pain.
Symptomatic relief using other prescription medications was associated with some of the commonly reported symptoms. For example, sleep impairment was widely reported as among the top three worst symptoms (20%), with sleep medications taken at higher rates (34%) than the general age-adjusted population (9%) [39]. Thyroid medication, which is sometimes associated with managing fatigue, was also much higher (33%) than the general age-adjusted population (8%) [39]. Similarly, pain-associated symptoms (joint pain, muscle aches, or neuropathy) were common (71%), and prescription pain medication usage (26%) was higher than the age-adjusted rate in the US general population (16%) [39].
Most patients did not recall or did not know if they had a tick bite (59%) but 29% of those who did, reported that the tick was attached for less than 24 h. Very few (2%) had the tick tested for pathogens. The majority (78%) reported that their Lyme disease diagnosis was supported by positive lab tests, and 45% of these reported that their tests were positive by either CDC two-tiered or western blot interpreted by CDC banding criteria.
Consistent with a prior study published in 2014, coinfections appeared to be common (60%; 45% with positive lab test confirmation), rather than the exception [5]. Although the rate of laboratory-confirmed coinfections in this sample was slightly lower than reported in the 2014 study (45% vs. 53%), the difference likely reflects differences in the inclusion criteria between the two samples.
The most commonly reported coinfections were Babesia (23% with supporting lab tests, 21% without) and Bartonella (19% with supporting lab tests, 23% without). Reported rates of Ehrlichia or Anaplasma were substantially lower (11% with supporting labs, 5% without). (Note that the rate of Bartonella in the general population is reported to be 3–6% [40] but has been reported to be as high as 28% in higher-risk populations such as veterinarians [41]. The 19% rate of positive Bartonella serology in our sample suggests that patients with chronic Lyme disease may be at higher risk of acquiring this coinfection.) These coinfection rates should be viewed in the context that many physicians do not test for coinfections and those who do, may only test for select coinfections.
Consistent with other studies, a majority (65%) reported their health status as fair or poor [5,17]. Moreover, 32% reported their work status as disabled (whether or not receiving disability payments). Finally, the great majority reported being treated by internists, family practitioners, or physicians who specialize in the diagnosis and treatment of tick-borne diseases. Few (6%) reported being treated by infectious disease physicians.

3.2. Global Rating of Change Scale

We compared the responses to the GROC scale with the average treatment effect to determine if this highly granular Likert scale was able to detect subgroups of patients who responded better or worse than the mean.
As shown in Table 3 below, 17.26% of the responses translated to a Likert scale value of 1 (“hardly better at all”), 2 (“a little better”), or 3 (“somewhat better”); 34.64% of patient responses were 4 or higher (“moderately better”, “a good deal better”, “a great deal better”, “a very great deal better”), and the remaining 48.11% of responses were 0 or lower (“unchanged” or from “hardly worse at all” to “a very great deal worse”).
Approximately 52% of this group of patients (Low and High Responders) reported some improvement in their condition. As shown in Figure 2, High Responders (Likert score of 4–7) made up 34.64% of study participants, while Low Responders constituted 17.26%. About 36% of these patients reported their healthcare status as unchanged; less than 12% reported that their condition was worse.
As Figure 3 below illustrates, if patient responses are evaluated by looking at the GROC average treatment effect or mean Likert score alone (1.7), one might conclude that there has been very little improvement in the health of this sample of unwell patients; using only the median (2.0) would lead to a similar conclusion. However, examining the distribution of the values for the unwell patients unmasks significant improvement (5.3 points out of 7 on a 7-point scale) among the 35% who were High Responders. Very few patients reported worsening (12%), and the deterioration among total Nonresponders as a category (which includes those who were unchanged) averaged −1.1 points out of 7.
Rather than merely calculating measures of central tendency as treatment effectiveness outcomes, using finer granularity of patient response enhances the ability to detect treatment response. Examining characteristics of this subgroup may lead to greater insights about risk identification of different groups of patients and help predict which patients are most likely to respond to treatment.

4. Discussion

4.1. The Value of Patient-Generated Health Data, a grassroots nonprofit organization that supports the interests of Lyme disease patients, has conducted and published peer-reviewed big data patient surveys for over 10 years [4,5]. In November 2015, it launched a patient-powered registry, MyLymeData, that enables patients to pool their healthcare data as a community research resource [42]. Patient-powered registries are similar to researcher-generated patient registries, but the registry is managed and controlled by patients who set the research agenda for the data [42,43].
The purpose of MyLymeData is four-fold: to conduct observational research, to assist researchers in conducting and recruiting for traditional and innovative clinical trials, to improve the quality of care through standard of care studies, and to create healthcare policy change. Since its launch, over 11,000 patients have enrolled, and the National Science Foundation awarded a grant to a team of researchers to explore data analytic techniques using registry data.
Patient-powered registries rely primarily on patient-generated data, which is defined by the Patient Outcomes Research Institute as:
“Health-related data—including health history, symptoms, biometric data, treatment history, lifestyle choices, and other information—created, recorded, gathered, or inferred by or from patients or their designees (i.e., care partners or those who assist them) to help address a health concern.”
Patient-generated data is increasingly recognized as a valuable source of real-world evidence by government projects, including PCORnet (launched by the Patient Centered Outcomes Research Institute), the NIH Collaboratory, the CDC National Amyotrophic Lateral Sclerosis (ALS) Registry as well as professional, commercial, and patient organizations such as the American Society of Clinical Oncology, PatientsLikeMe, CancerBase, and the Duchenne Registry [20,44,45,46,47,48,49].
Big data analytics are expected to play a critical role in the emergence of personalized medicine and individualized care. The NIH notes that patient registries can help
  • improve recruitment,
  • identify patient research cohorts,
  • conduct natural history studies,
  • integrate patient-reported and clinical data from multiple sources into single registry,
  • stimulate new research on the causes, treatments, and outcomes of diseases,
  • accelerate research, knowledge discovery, and scientific insights from patients with under-researched diseases, and
  • enhance creative data mining within and across diseases [50].
Patients hold an enormous amount of inexpensive, underutilized data. Unlike traditional trials, patient registries permit researchers to
  • enroll diverse patient populations,
  • evaluate care as it is actually provided in real-world practice,
  • assess complex treatment patterns and treatment combinations, and
  • offer the ability to evaluate patient outcomes when clinical trials are not practical or are difficult to conduct (for example, when long-term outcomes are important) [22].
To harness the full benefits of personalized healthcare will require more efficient research practices and big data analytics to discover deep knowledge about patient similarities, personalized disease risk profiles for individual patients, and treatment response heterogeneity [51].

4.2. The Need to Accelerate Research in Treatment of Lyme Disease

Research on chronic Lyme disease needs to move forward at a much more rapid pace than it has historically. Although Lyme disease is a common disease, until the CDC revised its estimated annual incidence to 300,000 cases in 2013, this was not commonly recognized, and very few clinical studies of Lyme disease have been conducted compared to other infectious diseases. The CDC estimates that the annual incidence of Lyme disease is 300,000 and has grown over 300% since the late 1990s. However, although the incidence of Lyme disease is nearly 8 times higher than the number of people diagnosed with HIV/AIDS each year in the US (38,500), the number of clinical studies for Lyme disease trails behind leprosy, which has an incidence of less than 200 cases a year [52,53,54,55]. This is illustrated in Figure 4 below derived from a study by Goswami on clinical trials for infectious diseases listed on [10].
The few clinical trials that have received NIH funding on Lyme disease have been hampered by highly restrictive selection criteria for study participation, leading to very small samples (ranging from 37 to 129) and long recruitment times (2.5 to 4 years). Table 4 delineates these factors and compares the NIH trials with a big data study of over 3000 patients recruited in six months. The big data study detailed in the table included chronic Lyme patients (a) who were clinician diagnosed, (b) had either an EM rash or positive serology, and (c) who remained ill for six or more months following treatment with antibiotics.
The largest randomized controlled NIH trial screened out 93% of the 1966 patients who attempted to enroll [17]. In trials with small sample sizes, it is difficult to detect small or moderate treatment effects that most patients would regard as meaningful [56,57]. This was highlighted in an important critique of the Klempner trial [58]. Ultimately, these trials are not generalizable to most patients seen clinically and are too small for subgroup analysis.

4.3. Samples and Outcomes in Lyme Disease Studies Reflect a Heterogeneous Patient Population

There are a wide range of factors that may contribute to heterogeneity among patients with Lyme disease in either severity of illness or responsiveness to treatment. Patients with chronic Lyme disease have a variety of symptoms, incidence of coinfections, and delays in diagnosis. In addition, strain variation and epigenetics likely play a role in disease presentation. Heterogeneity of treatment response may arise from individualized risk of disease progression, responsiveness to treatment, vulnerability to adverse side effects from treatment, the person’s unique assessment of trade-offs involved in different health states, and the treatment-associated risks and benefits (often referred to as utilities) [59].
To further compound the problem, we do not yet fully understand the distribution of the population of Lyme disease patients because the full spectrum of the disease is not yet well understood. In Lyme disease, many studies focus on acute, well-characterized patients who evidence objective manifestation that the CDC uses for surveillance purposes [19] or other similarly restrictive definitions of Lyme disease [15,60]. This may be useful for randomized controlled trials and surveillance purposes where being highly selective achieves an associated goal. However, this approach does little to inform us about the characteristics of the disease that do not meet that narrow definition, for example, a patient without a rash or Bell’s palsy or who does not test positive on two-tiered serology. Thus, other more inclusive research definitions of Lyme disease that reflect those used in clinical care might prove useful in real-world clinical trials [34]. (See Appendix B).
Prior work by the authors compared restrictive diagnostic criteria on a sample of 3000 patients with either an EM rash or serological evidence for Lyme disease [5]. This work found that patients who met the CDC vs. less restrictive laboratory diagnostic criteria were similar demographically and on other outcomes measures used in that study. However, patients who did not meet the more restrictive CDC definition were more likely to have their diagnosis delayed. This is problematic because a preliminary analysis of patient outcomes from the same MyLymeData dataset used in this paper found evidence suggesting that patients diagnosed early were far more likely to be well than those with a delayed diagnosis [61].
Hence, patients who experience delayed diagnosis may be more likely to develop chronic Lyme disease [62,63]. For example, females may be at higher risk of contracting chronic Lyme disease due to diagnostic delays if laboratory diagnostic testing is more effective in males, as has been suggested [64,65]. They may also be at higher risk of developing chronic Lyme disease generally as indicated in a large, insurance-based big data study [66]. These factors may explain why our sample was predominantly female.
However, a less restrictive definition of Lyme disease alone is not sufficient to inform treatment decisions because treatment response may vary in different subgroups. Identifying subgroup variation requires not only larger sample sizes but also the recognition that assessing treatment response using average treatment effects will fail to capture treatment response heterogeneity and may overlook minimum clinically important differences for the disease [58].
As Kravitz illustrated (see Figure 5 below), the individual patient seen in clinical practice is not average but unique in terms of treatment response [22]. He also highlighted the fact that averages can only reflect the sample population of a particular study and will change with different patient populations to reflect the study entry restrictions.
Misapplying averages to a heterogeneous group of individual patients, such as those in our study, can create harm by either providing treatment that is unlikely to benefit a patient or by denying a patient treatments that would be beneficial to them [59]. Unfortunately, treatment guidelines often base their recommendations on average treatment effects. Some commentators contend that average treatment effects should not be used to constrain individualized care unless the studies used by guideline committees are sufficiently powered to detect and rule out treatment response heterogeneity [37]. However, medical decisions are made for individuals, and assessment of the heterogeneity of treatment effects is critical as medicine seeks to become more personalized and patient-centered [67].
Our analysis compared the average treatment response with subgroups of patients who were categorized as High Responders, Low Responders, and Nonresponders. Using the GROC scale to assess treatment response, rather than studying average treatment effectiveness, we demonstrated the value of subgroup analysis to unmask heterogeneous treatment response within a sample. Although average treatment response for the sample showed negligible improvement, the GROC scale permitted a more granular analysis that differentiated treatment responders in the sample. This approach indicated that a majority of those participating in the study (approximately 52%) improved with treatment, and a significant subgroup (approximately 35%) were High Responders, which is in line with the effectiveness of therapies in other diseases [68]. This is important because a better understanding of the 1 out of 3 patients who respond well to treatment can potentially guide the development of treatment mechanisms and allow a more personalized approach to treatment.
This study demonstrates the power of using patient data to better understand the course of disease and the success of treatment. The highly granular classification value of the GROC scale when used with a large data set is demonstrated as this tool was able to successfully delineate Lyme disease patients into subgroups according to their response to treatment. To our knowledge, this is the first published study using patient-generated data from a patient registry to assess GROC in a big data study. We suggest more research is needed to assess the applicability of using patient-reported data and clinical data on patients with chronic Lyme disease to discover what illness, patient, and treatment characteristics may lead to improvements in health status. This research is critically important to inform the development of medical guidelines for chronic Lyme disease that are more patient-centered in their approach.
An understanding of the characteristics of the underlying sample may also prove useful in developing strategies to prevent patients from progressing to the persistent form of the disease through earlier or more targeted intervention. For example, diagnosing patients earlier or using more aggressive forms of intervention for those at higher risk, such as patients who had delayed diagnosis or coinfections, might improve treatment outcomes.

4.4. Strengths and Limitations

All data sources have their strengths and limitations depending on their source and characteristics. Hence, great care needs to be taken for comparison purposes and for any meta-analyses. For example, convenience samples drawn from clinicians may have inherent sample bias reflecting the geographic location of the office as well as the treating style of the physician (and whether patients select to be treated by the physician). Insurance databases are broad but are limited to the data they collect and only include those whose treatment is covered by the insurance plan. (Many Lyme patients report that their physicians do not accept insurance.) Electronic health records may be limited by their intended utilitarian focus (billing claims). Appendix C lists various data sources and their limitations.
Applying average treatment effects from small, highly selective randomized trials to the clinical population is problematic because of the lack of generalizability of these trials. Many clinical trials conducted in the US report on average treatment results, with some concluding that there is no treatment effect [59]. In essence, these studies are reporting on results for the “average” patient despite the variation in patient characteristics and outcomes seen in clinical practice [69].
Because patient registries reflect real-world behavior and practices and employ fewer inclusion and exclusion criteria, they are more inclusive than randomized controlled trials and more generalizable to patients seen in clinical practice [70]. Patient registries are also not tied to a geographic locus and may hence reflect greater geographic diversity than traditional research trials previously conducted.
However, registries also pose unique challenges because patients are recruited directly in situations where the underlying sampling frame is unknown and registry samples are often self-selected (as is the case with MyLymeData) [70]. These participants have access to the internet and are not a randomly drawn sample. Those who elect to participate may have been sick longer and may have been more severely ill, which could lead them to seek online support and resources for their illness [5].
The patient registry results presented here are based on self-reported information without independent diagnostic confirmation. However, self-reported information is reported to improve accuracy of patient data and has been found to have acceptable levels of reliability when compared to medical chart information [70,71,72,73,74].
MyLymeData has a relatively low rate of participation from those with early disease (6%). This suggests that patients without chronic disease may have less motivation to join and see less need to pool data assets as a community. For example, patients with a short-lived acute illness like the common cold have no need for community support; they simply get on with their lives. Accordingly, we believe that the results reported here capture a segment rather than the spectrum of those with Lyme disease.
Finally, although this study included data from close to 4000 participants in MyLymeData and has significant implications for research design and health policy, observational samples are not suitable for determining cause and effect.

5. Conclusions

The focus of this study is to demonstrate the usefulness of the GROC scale to unmask heterogeneous treatment effects in Lyme disease. In Lyme disease, researchers have emphasized average treatment effects when assessing the effectiveness of treatment interventions [16,17,18]. However, averages may mask heterogeneity of treatment response by failing to distinguish between patients who respond better or worse than average [59]. The purpose of examining treatment response heterogeneity is to improve individualized care and health outcomes that are most meaningful to patients and clinicians [37,70].
The analysis discussed in this paper, using a Likert scale to provide a more granular view of treatment response, demonstrates the benefit of going beyond examination of average treatment effects. With the increased availability of big data—both patient-generated and clinician-generated—a highly granular approach is becoming more feasible.
Determining the best clinical treatment approach for an individual is fundamentally different from determining the average treatment effect in a randomized clinical trial, although the two are often conflated [37]. With the increase in the number of cases of chronic Lyme disease, there is a growing urgency to discover how best to treat these patients. The limitations of “one variable at a time” sequential clinical trials include slow progress at a time when patients who are profoundly sick today do not have the luxury of waiting for tomorrow’s research. Fortunately, current technologies and large data sources provide us with the opportunity to develop newer, more robust, pragmatic trial designs to augment the rigor of randomized clinical trials and can accelerate the pace of research.
With the availability of larger databases, including patient registries such as MyLymeData, researchers have the opportunity to employ techniques to discover small to moderate treatment effects and then further examine the characteristics of the patients that are most responsive to treatment. This approach paves the way for analyzing patient characteristics to understand which patients are at higher risk of developing chronic disease and why a treatment is more effective with certain patients.
These factors, in turn, may be used to predict treatment responses in individual cases and to help address and prevent these differences from developing, for example, through early diagnosis and treatment. Possible approaches to teasing out treatment effects may include examination of the following: disease duration, disease severity at diagnosis, number of coinfections, patient demographics, and treatment delivery mechanism as well as treatment regimen and duration. A deeper understanding of treatment response may also help identify potential biomarkers for this disease and aid in developing more targeted and innovative treatments for those with chronic Lyme disease.

Supplementary Materials

The following are available online at, Table S1: data used in the preparation of this article, obtained from the patient registry, MyLymeData, Phase 1 27 April 2017.

Author Contributions

L.J. conceived the study, recruited patients, and wrote and edited the manuscript. J.M. and M.S. analyzed the data and edited the manuscript.


Funding for this work is supported in part by grants from the National Science Foundation (NSF BIGDATA DMS #1740325), the Bruce Lindorf Memorial Foundation, and the BMC Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.


Most importantly, we thank the patients participating in MyLymeData, who have the power to accelerate research in Lyme disease and without whom this research could not have been done. The authors also thank Brian Fallon, John Aucott, Deanna Needell, Steve Wygant, Elizabeth Maloney, Phyllis Mervine, Chris Green, and Amanda Elan for their thoughtful comments and research insights.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A. MyLymeData Patient Registry Survey Overview

Medical HistoryPatient Characteristics
  • Onset/duration
  • Severity
  • Diagnostic history/exposure
  • Treatment history
  • Medications
  • Diagnostic tests and results
  • Comorbidities
  • Reproductive history
  • Family history
  • Mortality (cause and date)
  • Functional status, quality of life, symptoms
  • Marital status
  • Work status
  • Disability, work attendance (days lost from work), or absenteeism/presenteeism
  • Economic status
  • Healthcare resource utilization
  • Genetic information
  • Sources of care
Treatment Risks and BenefitsChanges over Time
  • Medication, modes, duration of treatment
  • Adverse events
  • Treatment effectiveness
  • Medications/medical status
  • Patient characteristics
  • Changes in financial status

Appendix B. Potential Clinical Research Criteria for Pragmatic Big Data Trials

This appendix is derived from a paper by Stricker et al. [34] See Table 2 for diagnostic characteristics of sample.
  • Required
    Physician diagnosed
    Clinical symptoms consistent with Lyme disease
    Late stage Lyme disease (untreated for ≥6 months after symptom onset) or chronic Lyme disease (persistent symptoms for ≥6 months after treatment with <30 days of antibiotics)
  • Strongly supportive
    Positive serological testing
    Fulfills CDC surveillance Western blot criteria
    Fulfills Ma/Engstrom criteria
    Seropositive for other tick-borne conditions
  • Supportive
    History of EM rash
    Known or possible tick bite
    Risk of exposure

Appendix C. Data Sources and Limitations

Data SourceStrengthsWeaknesses
Claims DataUbiquitous
Insurance covered care only
Designed for billing purposes
No “results”
Not timely
Academic case dataWell characterized samples
Rigorous methodology
Lab measures
Assessments (Results)
EHR siloed
Physician time constraints
Practice constraints limit treatments
Convenience sample
Community clinical dataReal-world clinical care
N-of-1 practices
Lab measures
EHR siloed or paper/nonexistent
Physician time constraints
Convenience sample
Patient registriesReal-world clinical care
Labor burden on patient
Emerging innovation
Patient generated
Self-selected sample


  1. Centers for Disease Control and Prevention. CDC Provides Estimate of Americans Diagnosed with Lyme Disease Each Year. 2013. Available online: (accessed on 30 September 2018).
  2. Aucott, J.N.; Rebman, A.W.; Crowder, L.A.; Kortte, K.B. Post-treatment Lyme disease syndrome symptomatology and the impact on life functioning: Is there something here? Qual. Life Res. 2013, 22, 75–84. [Google Scholar] [CrossRef] [PubMed]
  3. Hirsch, A.G.; Herman, R.J.; Rebman, A.; Moon, K.A.; Aucott, J.; Heaney, C.; Schwartz, B.S. Obstacles to diagnosis and treatment of Lyme disease in the USA: A qualitative study. BMJ Open 2018, 8, e021367. [Google Scholar] [CrossRef] [PubMed]
  4. Johnson, L.; Aylward, A.; Stricker, R.B. Healthcare access and burden of care for patients with Lyme disease: A large united states survey. Health Policy 2011, 102, 64–71. [Google Scholar] [CrossRef] [PubMed]
  5. Johnson, L.; Wilcox, S.; Mankoff, J.; Stricker, R.B. Severity of chronic Lyme disease compared to other chronic conditions: A quality of life survey. PeerJ 2014, 2, e322. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Asch, E.S.; Bujak, D.I.; Weiss, M.; Peterson, M.G.; Weinstein, A. Lyme disease: An infectious and postinfectious syndrome. J. Rheumatol. 1994, 21, 454–461. [Google Scholar] [PubMed]
  7. Shadick, N.A.; Phillips, C.B.; Logigian, E.L.; Steere, A.C.; Kaplan, R.F.; Berardi, V.P.; Duray, P.H.; Larson, M.G.; Wright, E.A.; Ginsburg, K.S.; et al. The long-term clinical outcomes of Lyme disease. A population-based retrospective cohort study. Ann. Intern. Med. 1994, 121, 560–567. [Google Scholar] [CrossRef] [PubMed]
  8. Shadick, N.A.; Phillips, C.B.; Sangha, O.; Logigian, E.L.; Kaplan, R.F.; Wright, E.A.; Fossel, A.H.; Fossel, K.; Berardi, V.; Lew, R.A.; et al. Musculoskeletal and neurologic outcomes in patients with previously treated Lyme disease. Ann. Intern. Med. 1999, 131, 919–926. [Google Scholar] [CrossRef] [PubMed]
  9. Treib, J.; Fernandez, A.; Haass, A.; Grauer, M.T.; Holzer, G.; Woessner, R. Clinical and serologic follow-up in patients with neuroborreliosis. Neurology 1998, 51, 1489–1491. [Google Scholar] [CrossRef] [PubMed]
  10. Goswami, N.D.; Pfeiffer, C.D.; Horton, J.R.; Chiswell, K.; Tasneem, A.; Tsalik, E.L. The state of infectious diseases clinical trials: A systematic review of clinicaltrials.Gov. PLoS ONE 2013, 8, e77086. [Google Scholar] [CrossRef] [PubMed]
  11. Report of the Pathogenesis, Transmission, and Treatment Subcommittee to the Tick-Borne Disease Working Group. 4 May 2018. Available online: Https://www.Hhs.Gov/ash/advisory-committees/tickbornedisease/reports/pathogenesis-transmission-2018-5-9/index.Html (accessed on 30 September 2018).
  12. Zhang, X.; Meltzer, M.I.; Pena, C.A.; Hopkins, A.B.; Wroth, L.; Fix, A.D. Economic impact of Lyme disease. Emerg. Infect. Dis. 2006, 12, 653–660. [Google Scholar] [CrossRef] [PubMed]
  13. Hook, S.A.; Nelson, C.A.; Mead, P.S. U.S. Public’s experience with ticks and tick-borne diseases: Results from national healthstyles surveys. Ticks Tick Borne Dis. 2015, 6, 483–488. [Google Scholar] [CrossRef] [PubMed]
  14. Mead, P. Statement by Paul Mead, M.D., M.P.H., Medical Epidemiologist, Division of Vector-Borne Infectious Diseases, National Center for Infectious Diseases, Center for Disease Control and Prevention, U.S. Department of Health and Human Services at Hearing: Cdc’s Lyme Disease Prevention and Control Activities before the Connecticut Department of Public Health and the Connecticut Attorney General’s Office On 29 January 2004. Available online: (accessed on 30 September 2018).
  15. Wormser, G.P.; Dattwyler, R.J.; Shapiro, E.D.; Halperin, J.J.; Steere, A.C.; Klempner, M.S.; Krause, P.J.; Bakken, J.S.; Strle, F.; Stanek, G.; et al. The clinical assessment, treatment, and prevention of Lyme disease, human granulocytic anaplasmosis, and babesiosis: Clinical practice guidelines by the infectious diseases society of america. Clin. Infect. Dis. 2006, 43, 1089–1134. [Google Scholar] [CrossRef] [PubMed]
  16. Fallon, B.A.; Keilp, J.G.; Corbera, K.M.; Petkova, E.; Britton, C.B.; Dwyer, E.; Slavov, I.; Cheng, J.; Dobkin, J.; Nelson, D.R.; et al. A randomized, placebo-controlled trial of repeated iv antibiotic therapy for Lyme encephalopathy. Neurology 2008, 70, 992–1003. [Google Scholar] [CrossRef] [PubMed]
  17. Klempner, M.; Hu, L.; Evans, J.; Schmid, C.; Johnson, G.; Trevino, R.; Norton, D.; Levy, L.; Wall, D.; McCall, J.; et al. Two controlled trials of antibiotic treatment in patients with persistent symptoms and a history of Lyme disease. N. Engl. J. Med. 2001, 345, 85–92. [Google Scholar] [CrossRef] [PubMed]
  18. Krupp, L.B.; Hyman, L.G.; Grimson, R.; Coyle, P.K.; Melville, P.; Ahnn, S.; Dattwyler, R.; Chandler, B. Study and treatment of post Lyme disease (Stop-LD): A randomized double masked clinical trial. Neurology 2003, 60, 1923–1930. [Google Scholar] [CrossRef] [PubMed]
  19. Wormser, G.P. Clinical practice. Early Lyme disease. N. Engl. J. Med. 2006, 354, 2794–2801. [Google Scholar] [CrossRef] [PubMed]
  20. Horwitz, R.I.; Hayes-Conroy, A.; Caricchio, R.; Singer, B.H. From evidence based medicine to medicine based evidence. Am. J. Med. 2017, 130, 1246–1250. [Google Scholar] [CrossRef] [PubMed]
  21. RAND Health. 36-Item Short Form Survey (SF-36). Available online: (accessed on 30 September 2018).
  22. Gliklich, R.D.N.; Leavy, M. (Eds.) Registries for Evaluating Patient Outcomes: A User’s Guide, 3rd ed.; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2014.
  23. Centers for Disease Control and Prevention. Hospital Utilization (in Non-Federal Short-Stay Hospitals). Available online: (accessed on 30 September 2018).
  24. Centers for Disease Control and Prevention. Measuring Healthy Days; CDC Atlanta: Atlanta, GA, USA, 2000.
  25. Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance Survey Data. Available online: (accessed on 30 September 2018).
  26. Centers for Disease Control and Prevention. National Ambulatory Medical Care Survey: 2010 Summary Tables. Available online: (accessed on 30 September 2018).
  27. Centers for Disease Control and Prevention. National Ambulatory Medical Care Survey: 2010 Outpatient Department Summary Tables. Available online: (accessed on 30 September 2018).
  28. Centers for Disease Control and Prevention. National Ambulatory Medical Care Survey: 2010 Emergency Department Summary Tables. Available online: (accessed on 30 September 2018).
  29. Agency for Healthcare Research and Quality. Medical Expenditure Panel Survey. Available online: (accessed on 19 March 2014).
  30. Junghard, O.; Halling, K. Responsiveness of measures of heartburn improvement in non-erosive reflux disease. Health Qual. Life Outcomes 2007, 5, 32. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Kamper, S.J.; Maher, C.G.; Mackay, G. Global rating of change scales: A review of strengths and weaknesses and considerations for design. J. Man. Manip. Ther. 2009, 17, 163–170. [Google Scholar] [CrossRef] [PubMed]
  32. Wang, Y.C.; Hart, D.L.; Stratford, P.W.; Mioduski, J.E. Baseline dependency of minimal clinically important improvement. Phys. Ther. 2011, 91, 675–688. [Google Scholar] [CrossRef] [PubMed]
  33. Gliklich, R.D.N.; Leavy, M.; Christian, J.B. (Eds.) 21st Century Patient Registries. In Ebook Addendum to Registries for Evaluating Patient Outcomes: A User’s Guide, 3rd ed.; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2018. [Google Scholar]
  34. Stricker, R.B.; Fessler, M. Chronic lyme disease: A working case definition. Am. J. Infect. Dis. 2018, 14, 1–44. [Google Scholar] [CrossRef]
  35. Wang, Y.C.; Sindhu, B.S.; Kapellusch, J.; Yen, S.C.; Lehman, L. Global rating of change: Perspectives of patients with lumbar impairments and of their physical therapists. Physiother. Theory Pract. 2018, 2, 1–9. [Google Scholar] [CrossRef] [PubMed]
  36. Jaeschke, R.; Singer, J.; Guyatt, G. Measurement of health status: Ascertaining the minimal clinically important difference. Control. Clin. Trials 1989, 10, 407–415. [Google Scholar] [CrossRef]
  37. Dahabreh, I.J.; Hayward, R.; Kent, D.M. Using group data to treat individuals: Understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence. Int. J. Epidemiol. 2016, 45, 2184–2193. [Google Scholar] [CrossRef] [PubMed]
  38. Wright, A.; Hannon, J.; Hegedus, E.J.; Kavchak, A.E. Clinimetrics corner: A closer look at the minimal clinically important difference (MCID). J. Man. Manip. Ther. 2012, 20, 160–166. [Google Scholar] [CrossRef] [PubMed]
  39. National Center for Health Statistics. Health, United States, 2016: With Chartbook on Long-Term Trends in Health; National Center for Health Statistics: Hyattsville, MD, USA, 2017.
  40. Chomel, B.B.; Kasten, R.W. Bartonellosis, an increasingly recognized zoonosis. J. Appl. Microbiol. 2010, 109, 743–750. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Lantos, P.M.; Maggi, R.G.; Ferguson, B.; Varkey, J.; Park, L.P.; Breitschwerdt, E.B.; Woods, C.W. Detection of bartonella species in the blood of veterinarians and veterinary technicians: A newly recognized occupational hazard? Vector Borne Zoonotic Dis. 2014, 14, 563–570. [Google Scholar] [CrossRef] [PubMed]
  42. Workman, T.A. Agency for Healthcare Research and Quality. Engaging Patients in Information Sharing and Data Collection: The Role of Patient-Powered Registries and Research Networks. 2013. Available online: (accessed on 30 September 2018).
  43. Mader, L.B.; Harris, T.; Kläger, S.; Wilkinson, I.B.; Hiemstra, T.F. Inverting the patient involvement paradigm: Defining patient led research. Res. Involv. Engagem. 2018, 4, 21. [Google Scholar] [CrossRef] [PubMed]
  44. Howie, L.; Hirsch, B.; Locklear, T.; Abernethy, A.P. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff. 2014, 33, 1220–1228. [Google Scholar] [CrossRef] [PubMed]
  45. Centers for Disease Control and Prevention. National Amyotrophic Lateral Sclerosis (ALS) Registry. Available online: (accessed on 30 September 2018).
  46. PatientsLikeMe. Available online: (accessed on 30 September 2018).
  47. Stanford University and Volunteers. Cancerbase. Available online: (accessed on 30 September 2018).
  48. The Duchenne Registry. Available online: (accessed on 30 September 2018).
  49. Fleurence, R.L.; Beal, A.C.; Sheridan, S.E.; Johnson, L.B.; Selby, J.V. Patient-powered research networks aim to improve patient care and health research. Health Aff. 2014, 33, 1212–1219. [Google Scholar] [CrossRef] [PubMed]
  50. Groft, S. Patient Registries As a Prelude to Clinical Trials and Post-Approval Studies; Pediatric Devices for Rare Diseases Food and Drug Administration: White Oak, MD, USA, 8 January 2014.
  51. Chawla, N.V.; Davis, D.A. Bringing big data to personalized healthcare: A patient-centered framework. J. Gen. Intern. Med. 2013, 28 (Suppl. 3), S660–S665. [Google Scholar] [CrossRef]
  52. Centers for Disease Control and Prevention. Hiv Surveillance Report. 2016; Volume 28. Available online: (accessed on 30 September 2018).
  53. Centers for Disease Control and Prevention. Lyme and Other Tickborne Diseases. Available online: (accessed on 30 September 2018).
  54. National Institute of Allergy and Infectious Diseases. Hiv Infections Drop 18 Percent in Six Years. 14 February 2017. Available online: (accessed on 30 September 2018).
  55. Health Resources and Services Administration. National Hansen’s Disease (Leprosy) Program Caring and Curing Since 1894. Available online: (accessed on 30 September 2018).
  56. Biau, D.J.; Kernéis, S.; Porcher, R. Statistics in brief: The importance of sample size in the planning and interpretation of medical research. Clin. Orthop. Relat. Res. 2008, 466, 2282–2288. [Google Scholar] [CrossRef] [PubMed]
  57. Guyatt, G.H.; Mills, E.J.; Elbourne, D. In the era of systematic reviews, does the size of an individual trial still matter. PLoS Med. 2008, 5, e4. [Google Scholar] [CrossRef] [PubMed]
  58. Delong, A.K.; Blossom, B.; Maloney, E.; Phillips, S.E. Antibiotic retreatment of Lyme disease in patients with persistent symptoms: A biostatistical review of randomized, placebo-controlled, clinical trials. Contemp. Clin. Trials 2012, 33, 1132–1142. [Google Scholar] [CrossRef] [PubMed]
  59. Kravitz, R.L.; Duan, N.; Braslow, J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004, 82, 661–687. [Google Scholar] [CrossRef] [PubMed]
  60. Aucott, J.N.; Crowder, L.A.; Kortte, K.B. Development of a foundation for a case definition of post-treatment Lyme disease syndrome. Int. J. Infect. Dis. 2013, 17, e443–e449. [Google Scholar] [CrossRef] [PubMed]
  61. Johnson, L.; Mervine, P.; Potter, M. New patient-powered research tool can be used to answer important questions about Lyme disease. In Proceedings of the Poster Presentation at the LDA/Columbia Conference, St. Paul, MN, USA, 16 October 2016. [Google Scholar]
  62. Aucott, J.; Morrison, C.; Munoz, B.; Rowe, P.C.; Schwarzwalder, A.; West, S.K. Diagnostic challenges of early Lyme disease: Lessons from a community case series. BMC Infect. Dis. 2009, 9, 79. [Google Scholar] [CrossRef] [PubMed]
  63. Aucott, J.N.; Seifter, A. Misdiagnosis of early Lyme disease as the summer flu. Orthop. Rev. 2011, 3, e14. [Google Scholar] [CrossRef] [PubMed]
  64. Stricker, R.B.; Johnson, L. Gender bias in chronic Lyme disease. J. Womens Health 2009, 18, 1717–1718. [Google Scholar] [CrossRef] [PubMed]
  65. Schwarzwalder, A.; Schneider, M.F.; Lydecker, A.; Aucott, J.N. Sex differences in the clinical and serologic presentation of early Lyme disease: Results from a retrospective review. Gend. Med. 2010, 7, 320–329. [Google Scholar] [CrossRef] [PubMed]
  66. Adrion, E.R.; Aucott, J.; Lemke, K.W.; Weiner, J.P. Health care costs, utilization and patterns of care following Lyme disease. PLoS ONE 2015, 10, e0116767. [Google Scholar] [CrossRef] [PubMed]
  67. Kent, D.M.; Hayward, R.A. Limitations of applying summary results of clinical trials to individual patients: The need for risk stratification. JAMA 2007, 298, 1209–1212. [Google Scholar] [CrossRef] [PubMed]
  68. Connor, S. Glaxo Chief: Our Drugs Do Not Work on Most Patients. The Independent, 8 December 2003. [Google Scholar]
  69. Kent, D.M.; Nelson, J.; Dahabreh, I.J.; Rothwell, P.M.; Altman, D.G.; Hayward, R.A. Risk and treatment effect heterogeneity: Re-analysis of individual participant data from 32 large clinical trials. Int. J. Epidemiol. 2016, 45, 2075–2088. [Google Scholar] [CrossRef] [PubMed]
  70. Dreyer, N.; Franklin, P.; Haynes, K. Direct-to-patient registry and other patient-centric designs. In 21st Century Patient Registries: Registries for Evaluating Patient Outcomes: A User’s Guide, 3rd ed.; Gliklich, R.E., Dreyer, N.A., Leavy, M.B., Eds.; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2018. [Google Scholar]
  71. Cohen, D.J.; Keller, S.R.; Hayes, G.R.; Dorr, D.A.; Ash, J.S.; Sittig, D.F. Integrating patient-generated health data into clinical care settings or clinical decision-making: Lessons learned from project healthdesign. JMIR Hum. Factors 2016, 3, e26. [Google Scholar] [CrossRef] [PubMed]
  72. Wood, W.A.; Bennett, A.V.; Basch, E. Emerging uses of patient generated health data in clinical research. Mol. Oncol. 2015, 9, 1018–1024. [Google Scholar] [CrossRef] [PubMed]
  73. Bayliss, M.; Rendas-Baum, R.; White, M.K.; Maruish, M.; Bjorner, J.; Tunis, S.L. Health-related quality of life (HRQL) for individuals with self-reported chronic physical and/or mental health conditions: Panel survey of an adult sample in the united states. Health Qual. Life Outcomes 2012, 10, 154. [Google Scholar] [CrossRef] [PubMed]
  74. Bytzer, P. Assessment of reflux symptom severity: Methodological options and their attributes. Gut 2004, 53 (Suppl. 4), iv28–iv34. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Preliminary sample, exclusions, and final sample size determination.
Figure 1. Preliminary sample, exclusions, and final sample size determination.
Healthcare 06 00124 g001
Figure 2. The majority of participants (51%) reported some improvement in their condition after treatment with antibiotics, with High Treatment Responders constituting 34% of participants. Approximately 37% reported their condition as unchanged. Only 12% reported their condition as worse. Slight deviation in the percentages in the figure from the text reflect rounding errors.
Figure 2. The majority of participants (51%) reported some improvement in their condition after treatment with antibiotics, with High Treatment Responders constituting 34% of participants. Approximately 37% reported their condition as unchanged. Only 12% reported their condition as worse. Slight deviation in the percentages in the figure from the text reflect rounding errors.
Healthcare 06 00124 g002
Figure 3. Average treatment effect, high responders, and nonresponders on global rating of change (GROC) scale shows heterogeneous treatment response among participants that average treatment effect masks.
Figure 3. Average treatment effect, high responders, and nonresponders on global rating of change (GROC) scale shows heterogeneous treatment response among participants that average treatment effect masks.
Healthcare 06 00124 g003
Figure 4. Research in Lyme disease is sparse compared to other infectious diseases. (Derived from Goswami 2013 [10]).
Figure 4. Research in Lyme disease is sparse compared to other infectious diseases. (Derived from Goswami 2013 [10]).
Healthcare 06 00124 g004
Figure 5. (left) Treatment effects of nonrepresentative samples cannot be generalized to the full spectrum of disease; (right) In addition, within any given sample, individual patient treatment response varies from mean. (Derived from Kravitz 2004).
Figure 5. (left) Treatment effects of nonrepresentative samples cannot be generalized to the full spectrum of disease; (right) In addition, within any given sample, individual patient treatment response varies from mean. (Derived from Kravitz 2004).
Healthcare 06 00124 g005
Table 1. Demographic characteristics of respondents.
Table 1. Demographic characteristics of respondents.
VariableCount (% of Working Sample)
 Female3250 (83%)
 Mean age49
Education a
 High school or less340 (9%)
 Some college or associate degree1265 (34%)
 Bachelor degree1139 (31%)
 Graduate school degree945 (25%)
Family income b
 <$25k485 (14%)
$25–50k542 (15%)
$50–75k547 (16%)
$75–100k408 (12%)
 >$100k1025 (29%)
Geography c
 East1274 (33%)
 Midwest571 (15%)
 South1021 (26%)
 West1004 (26%)
a 214 skipped or selected “prefer not to answer”. b 896 skipped or selected “prefer not to answer”. c Excludes 33 from a US territory or who did not indicate a state.
Table 2. Current stage of illness and diagnostic characteristics of sample.
Table 2. Current stage of illness and diagnostic characteristics of sample.
VariableCount (% of Working Sample)
Current stage of illness
 Chronic LD a61%
 Late untreated LD b18%
 Early Lyme disease c6%
 Don’t know/Other15%
Stage when diagnosed
 Late untreated LD b70%
 Early Lyme disease c22%
 Don’t know/Other8%
Key diagnostic factors
 Clinician diagnosed (entry criteria for registry) d100%
 Recollection of tick bite41%
 Recollection of EM rash e34%
 With supportive lab tests78%
 1 or more coinfection60%
 Self-reported health status as fair or poor65%
 Disabled (with or without disability benefits)32%
a Remained ill for six months or more after treatment with antibiotics for 10–21 days. b Diagnosed and untreated for six months or more after symptom onset. c “Within days to weeks after my tick bite or exposure, I experienced symptoms associated with Lyme disease”. d To be enrolled, patients must have self-reported US residency and diagnosis by a healthcare provider. e Because of a branching error in the initial survey, patients were re-asked this question. This data includes the 1190 who responded to the revised question.
Table 3. Degree of change reported on global rating of change scale.
Table 3. Degree of change reported on global rating of change scale.
Better/Worse/UnchangedDegree of ChangeLikert Scoren (% of Total)Assigned Group
BetterHardly better at all143 (1.22)Low Responders
A little better2269 (7.61)Low Responders
Somewhat better3298 (8.43)Low Responders
Total 17.26%Low Responders
Moderately better4295 (8.34)High Responders
A good deal better5450 (12.73)High Responders
A great deal better6289 (8.17)High Responders
A very great deal better7191 (5.40)High Responders
Total 34.64%High Responders
Total Better 51.9%
Unchanged a 01293 (36.57)Nonresponders
WorseA very great deal worse−764 (1.81)Nonresponders
A great deal worse−664 (1.81)Nonresponders
A good deal worse−585 (2.40)Nonresponders
Moderately worse−471 (2.01)Nonresponders
Somewhat worse−366 (1.87)Nonresponders
A little worse−235 (0.99)Nonresponders
Hardly worse at all−123 (0.65)Nonresponders
Total Worse 11.54%Nonresponders
Total 100%
a Includes “almost the same” (better/worse).
Table 4. Research trials for chronic Lyme disease are small and highly selective compared to patient-generated big data studies.
Table 4. Research trials for chronic Lyme disease are small and highly selective compared to patient-generated big data studies.
Study TypeTrialScreenedEnrolledYieldTime to Recruit
RCT *Klempner (2001)19961297%3.3 years
RCTKrupp (2003)5125611%2.5 years
RCTFallon (2008)3368371%4 years
Big DataJohnson (2014)5357309058%6 months
* Randomized controlled trial [5,16,17,18].

Share and Cite

MDPI and ACS Style

Johnson, L.; Shapiro, M.; Mankoff, J. Removing the Mask of Average Treatment Effects in Chronic Lyme Disease Research Using Big Data and Subgroup Analysis. Healthcare 2018, 6, 124.

AMA Style

Johnson L, Shapiro M, Mankoff J. Removing the Mask of Average Treatment Effects in Chronic Lyme Disease Research Using Big Data and Subgroup Analysis. Healthcare. 2018; 6(4):124.

Chicago/Turabian Style

Johnson, Lorraine, Mira Shapiro, and Jennifer Mankoff. 2018. "Removing the Mask of Average Treatment Effects in Chronic Lyme Disease Research Using Big Data and Subgroup Analysis" Healthcare 6, no. 4: 124.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop