1. Introduction
Fragile X syndrome (FXS) is the most common cause of inherited intellectual disability (ID) and single-gene disorder associated with autism spectrum disorder (ASD) [
1,
2,
3,
4]. FXS is caused by a CGG-repeat expansion in the 5’ untranslated region of the Fragile X Mental Retardation 1 (
FMR1) gene (full mutation-FM, >200 CGGs). The FM triggers epigenetic silencing (hyper-methylation) of the gene, which results in marked reduction of the
FMR1 product (Fragile X Mental Retardation Protein, FMRP). FMRP is an RNA-binding protein that regulates the translation of ~4% of the brain’s mRNA, including many transcripts from genes implicated in ASD [
5,
6], and serves as a critical regulatory protein for developmental and ongoing synaptic plasticity. The FMRP expression in the brain is the ultimate factor determining the severity of the neurobehavioral phenotype. Data integration that enables clarification of the relationships between the
FMR1 genotype and FMRP and the neurobehavioral phenotype, including ASD, continue to grow [
7,
8].
Individuals with FXS present with cognitive impairment and a wide range of problem behaviors, such as ASD, general and social anxiety, attentional network deficits, repetitive and perseverative behaviors, and sensory over-reactivity [
1,
2,
3,
4]. About 40–50% of males and 20% of females meet the criteria for ASD [
2]. In addition to adequate educational placements and behavioral interventions, these problem behaviors often require a combination of complex, symptom-based pharmacological interventions [
1,
2,
3,
4]. To date, no target-symptom approaches or disease-modifying treatments for FXS have received regulatory approval [
7].
Animal and in vitro FXS models have led to great progress in drug identification and clinical trials in humans with FXS. Nevertheless, translating preclinical successes targeting core phenotypes associated with FXS into clinical trials has been a challenge [
7,
8]. At present, no definitive large-scale placebo-controlled trial has met primary endpoints [
8,
9]. Lacking precedent, the ‘first-wave’ of FXS clinical trials was felt to reflect the problems inherent in “building the bridge while crossing it.” All these large-scale and well-powered trials failed, suggesting issues with the predictive value of animal models, adequacy of trial design, age of treated cohorts, and, to some extent, outcome measure selection [
7,
9]. The ‘lessons learned’ have, therefore, prompted efforts at developing and validating improved outcome measures [
7,
8,
9,
10]. However, less attention has been given to other critical factors that could have negatively impacted these failed trials. One of the most notable is the placebo effect, which likely played a role due to the trials heavily depending on parent/caregiver reporting [
11,
12].
Placebo response has steadily increased over time in neurologic and, in particular, psychiatric randomized controlled trials (RCTs), potentially obscuring any improvements due to drug treatment [
12,
13,
14,
15,
16]. Preliminary studies in pediatric populations with idiopathic ASD have found increased placebo response associated with the use of clinician- over caregiver-rated outcome measures [
16]. RCTs in FXS have mainly relied on subjective, apparently “placebo-sensitive” clinician- and caregiver-rated outcome measures due to the lack of well-validated, reliable, linked to a clinical outcome, and sensitive to change biomarkers [
9]. The use of direct performance-rated cognitive and achievement testing in clinical trials has proven difficult in FXS. While such tests are used for clinical assessments, there are substantial floor effects in many tests not designed for or validated in populations with ID [
17]. These tests lack short-term sensitivity and, in some cases, there is a need to administer different levels of tests to capture the entire range of function in the individual or disorder. This is an area of ongoing intensive work, and tests that can circumvent floor effects are being validated [
18]. These tests show good test–retest reproducibility in ID populations, including FXS, covering a broad range of function; tests such as expressive language sampling (ELS) [
19] and NIH Toolbox [
20], that are expected to be sensitive to short-term change, are in development for trials in FXS [
8,
9]. Recent advances in molecular-phenotype relationships underscore links between
FMR1 expansion, gene methylation, FMRP deficit and overall severity of neurobehavioral phenotypes [
21,
22]. Despite these concerns, the extent to which the placebo effect has impacted FXS trial outcomes in general or specific outcome measures is unknown [
9].
Here, we present the first meta-analysis of RCTs in FXS with a focus on the placebo effect response. Rather than evaluating the impact of the placebo effect on the overall outcome of these trials, we have focused on examining whether the placebo groups showed significant improvements in eight RCTs of individuals with FXS conducted between 2006 and 2018. We aimed at answering the following questions:
Did placebo groups show significant and clinically meaningful responses?
Were clinician- and caregiver-rated outcome measures equally affected by placebo response?
Did age influence response in placebo groups?
4. Discussion
The present study is, to our knowledge, the first meta-analysis demonstrating positive responses in participants with FXS receiving placebo in multiple drug trials. We identified significant improvements in several caregiver-rated outcome measures, but not in the single clinician-rated endpoint under evaluation. No performance-based measure was used in more than one study, making this type of endpoint not suitable for meta-analyses. Improvements in caregiver-rated outcome measures were detected in studies involving either children or adolescents/adults, with a larger proportion of the latter showing a positive outcome. In terms of the magnitude of improvements, as measured by effect size or reference values, two out of three outcome measures showed functionally significant changes. These data are in line with reports of increasing placebo responses in RCTs of neurologic and psychiatric disorders, and raise concerns about study design and conduct of drug trials in FXS and other neurodevelopmental disorders [
11,
12,
14,
16,
42].
The main goals of this meta-analysis were to determine whether placebo-treated patients with FXS show improvements and, if so, whether positive responses were influenced by age of the participant and type of outcome measure. Multiple endpoints demonstrated improvements in placebo groups even in the face of effect heterogeneity in some trials (i.e., significant I
2) and were strong amidst assumptions regarding pre-post correlations (i.e., 0, 0.5, 0.7). As the studies used predominantly caregiver-rated endpoints, we could not evaluate the effect of the type of measure on outcomes. The fact that the CGI-I did not reach statistical significance, contrary to a study reporting on placebo response in pediatric ASD [
15], may be a random finding or reflect greater weight of caregiver input into CGI scores in autism trials. However, the CGI-1 meta-analysis data here is likely too limited to be informative as many of the prior trials in FXS appears to have shown a substantial placebo effect in the CGI-I scores at the end of the study (
Table 2) [
23,
24,
25,
26,
27,
28,
29,
30,
38,
43].
We could not derive conclusions from performance-based measures because they were not included in the meta-analysis due to their single study use. Nonetheless, the studies included in
Table 1 indicate that these endpoints do not change substantially in response to placebo administration. Overall, a larger proportion of caregiver-rated scales demonstrated improvement in older individuals with FXS treated with placebo. This is in line with the abovementioned meta-analysis of placebo effect in pediatric ASD, which reported an improvement in 60% of caregiver-rated outcome measures in adults versus approximately 25% in children [
16]. Nevertheless, meta-analyses of RCTs in other treatment areas (e.g., genetically determined intellectual disability, drug resistant partial epilepsy) demonstrate a different profile with greater placebo responses in younger participants [
42,
44]. A recent review of best practices in FXS clinical trials also concluded that inclusion of cohorts of younger participants would reduce the placebo effect [
8].
It is important to note not only the range of caregiver-rated outcome measures showing placebo responses (e.g., Vineland-II Comp, VAS Comp, and ABC-CFX subscales), but also the magnitude of the change in the reported values of these outcome measures. Our analyses of effect size showed that in children with FXS, the VAS Comp score improved at a very large level (i.e., 1.05–1.06 SD). As a reference, a minimal clinically important difference (MCID) is a metrics equivalent to ≥0.5 SD [
39,
40], which has been recently used to determine meaningful changes in adaptive behavior in ASD [
45]. Although the effect size of Vineland-II Comp score changes in children was negligible, the small increases represent an opposite direction to the naturally occurring decrease in standard scores [
41]. These data emphasize the potential large magnitude of responses to placebo administration in individuals with FXS and the importance of identifying and minimizing them.
The present meta-analysis shows that patients with FXS receiving placebo in drug trials can demonstrate statistically significant and clinically relevant improvements. The reporter must believe there are actual improvements in behavioral symptoms or cognition of the participant for these improvements to be captured in the outcome measures. This is important, as positive changes subjectively reported by caregivers could be due to caregiver expectations of treatment efficacy. As such, even measures like the CGI, which is clinician-rated but by and large caregiver informed in behavioral studies, appears to have captured a substantial placebo response in the recently failed clinical trials in FXS [
23,
24,
25,
26,
27,
28,
29,
30,
38,
43]. Thus, while our clinician-rated CGI-I meta-analysis data did not reach statistical significance due to limited available data, visual inspection of CGI-I data across multiple trials reveals the notable placebo effect on this measure. Our data also raises specific concerns about the use of caregiver-rated measures, in particular the VAS, in future FXS studies. Our findings are consistent with the suggestion that placebo responses have a greater impact on trials of older individuals with FXS [
8]. Our results have other implications, including the need for improvement of currently available endpoints, decreased dependence on caregiver-rated instruments, enhanced rater training to mitigate placebo response, enrollment of younger participants and utilization of study designs directly addressing the placebo effect (e.g., placebo run-ins, adaptive designs with enrichment in non-placebo responders) [
8,
46].
6. Limitations and Future Directions
The present study of a heterogeneous group of clinical trials in FXS focused on improvements in groups assigned to placebo, rather than a direct assessment of placebo (versus drug) responses in these trials. Moreover, it could only arrive to conclusions regarding caregiver-rated measures because of the small number of studies using other types of endpoints. Future analyses should also attempt to differentiate objective from subjective improvements in individuals with FXS. It is also important to appreciate differences between actual placebo-related improvements in behavior/cognition due to increased attention during the trial from positive changes subjectively reported by caregivers (e.g., due to their expectations of treatment efficacy). Considering these and other limitations of this study, larger follow up meta-analyses are needed. Developmental placebo response in longer (≥6 month) studies where the child grows and improves need to be considered as well [
28]. Moreover, the developmental gain of the “placebo” effect observed at this young age may also enable an additive synergy to the drug effect. Thus, the placebo effect is a major and increasing challenge for trials of neurologic and psychiatric disorders of all ages [
11,
12]. The issues with using caregiver-rated endpoints emphasize a compelling need to study biomarkers in FXS. Neurobiology in FXS is known to be impacted early in infancy. For example, visual perceptual problems, a common deficit observed in FXS that appears amenable to the drug treatment [
28], can be detected as early as the first year of life through eye-tracking studies of babies with FXS [
47], throughout the lifespan. MRI can be used over a large age range in FXS, and while MRI changes will be long term, it may not be sensitive enough to capture change in shorter trials [
9]. This is of relevance as the effects of fragile X gene expression on the early development of white matter structural connectivity are well established at six months of age [
48]. Electroencephalography (EEG) as a potential neural biomarker of changes due to treatment has the ability to examine features of fragile X due to scalability and reproducibility and translation from mouse model to clinical intervention such as neural hyperexcitability in FXS [
8,
49,
50]. Fragile X gene–protein biomarker development has also advanced as recent molecular phenotype studies underscore link between
FMR1 expansion, gene methylation, and FMRP deficit, and overall severity of neurobehavioral phenotype [
7,
8]. Newer performance-rated measures such as ELS and NIH Toolbox may be able to capture real change, controlling for placebo effect given the lack of caregiver rating [
8,
19,
20,
51]. As noted above, there are also newer trial designs that aim to limit or account for placebo response, including placebo lead-ins and adaptive trials designs [
8,
46]. Overall, the development of valid, sensitive-to-treatment biomarkers is necessary to reliably track treatment changes in the unfolding wave of clinical trials in FXS [
8,
52], which will substantially reduce the placebo effect. The examination of factors contributing to responses in placebo groups should be a continuous process [
35,
53] parallel to efforts at improving the measurement properties of endpoints in drug trials, including performance-rated measures [
9].