Reward Behavior Disengagement, a Neuroeconomic Model-Based Objective Measure of Reward Pathology in Depression: Findings from the EMBARC Trial

The probabilistic reward task (PRT) has identified reward learning impairments in those with major depressive disorder (MDD), as well as anhedonia-specific reward learning impairments. However, attempts to validate the anhedonia-specific impairments have produced inconsistent findings. Thus, we seek to determine whether the Reward Behavior Disengagement (RBD), our proposed economic augmentation of PRT, differs between MDD participants and controls, and whether there is a level at which RBD is high enough for depressed participants to be considered objectively disengaged. Data were gathered as part of the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, a double-blind, placebo-controlled clinical trial of antidepressant response. Participants included 195 individuals with moderate to severe MDD (Quick Inventory of Depressive Symptomatology (QIDS–SR) score ≥ 15), not in treatment for depression, and with complete PRT data. Healthy controls (n = 40) had no history of psychiatric illness, a QIDS–SR score < 8, and complete PRT data. Participants with MDD were treated with sertraline or placebo for 8 weeks (stage I of the EMBARC trial). RBD was applied to PRT data using discriminant analysis, and classified MDD participants as reward task engaged (n = 137) or reward task disengaged (n = 58), relative to controls. Reward task engaged/disengaged groups were compared on sociodemographic features, reward–behavior, and sertraline/placebo response (Hamilton Depression Rating Scale scores). Reward task disengaged MDD participants responded only to sertraline, whereas those who were reward task engaged responded to sertraline and placebo (F(1293) = 4.33, p = 0.038). Reward task engaged/disengaged groups did not differ otherwise. RBD was predictive of reward impairment in depressed patients and may have clinical utility in identifying patients who will benefit from antidepressants.


Introduction
Anhedonia, the loss of pleasure and/or reduced desire to pursue normally enjoyable activities, is a core symptom of major depressive disorder (MDD). Evidence for an anhedonic phenotype in patients with MDD is well supported. Notably, the probabilistic reward task (PRT), which measures implicit reward learning (i.e., the bias towards rewarding stimulus), has successfully measured reward learning impairments in MDD participants [1][2][3] as well as anhedonia-specific reward learning impairments [2,4,5]. However, attempts to validate the quantification of this anhedonic trait have not produced consistent findings [3,[6][7][8]. Such shortcomings may, at least in part, be due to the omission of an empiric cost/benefit model from measures of reward impairment [9]. These economic models imply that the extent to which a person engages or disengages in the task at hand, and therefore, the magnitude of reward learning, is predicated on a favorable cost/benefit analysis of the reward being offered. Thus, reward task engagement is thought to be an important, yet overlooked component of the reward learning impairments seen in MDD.

Conceptualization of Reward Behavior Disengagement (RBD)
The current study aims to characterize the behavioral and clinical features of a novel economic index of reward task engagement called Reward Behavior Disengagement (RBD). RBD is built upon an economic conceptualization of hedonic behavior, namely, that the decision to engage in any activity is contingent on a favorable cost/benefit analysis, wherein the costs include, at minimum, the effort required to perform the activity well (see Supplementary Materials for RBD derivation and computational details). Clinically significant reward impairment, as examined from this perspective, can thus arise anytime the effort associated with a normally rewarding activity is overly penalized, and ultimately results in task disengagement. For example, in clinically healthy individuals, the dollar dropped onto the floor of one's home is worth picking up, while the dollar left on the other side of the airport security checkpoint is not. However, in individuals with severe anhedonic impairment, as indexed by higher than normal RBD, neither dollar is worth the effort required to collect it.

Measuring RBD
It is worth noting that although this is the first work involving RBD, functionally equivalent measures can be independently derived by simply adapting the Integrated Signals and Economics (ISE) framework, formerly proposed by Lynn et al., to the reward task or behavior of interest (see Supplementary Materials for details on how this can be conducted). Thus, this work can be thought of as a real-world examination of the a priori ISE framework.
For this study, the ISE framework was applied to the PRT to produce our new measure, RBD. Briefly, the PRT involves two blocks. In block 1, participants experience asymmetric reward reinforcement among the two task stimuli; in block 2, participants process this experience to more effectively select rewarding stimuli. We hypothesized that RBD would differ between MDD and healthy control (HC) participants, specifically in block 2 after participants had been given the opportunity to process the asymmetric reinforcement schedule. Critically, and consistent with prior findings that blunted reward learning in the PRT is driven by abnormalities in anhedonic [2] MDD participants, we hypothesized that this difference would be driven by the presence of two distinct phenotypes within our MDD cohort: a reward task disengaged group (i.e., significantly elevated RBD relative to HCs), and an HC-like reward task engaged group.
Finally, we tested the clinical utility of the reward task engaged and reward task disengaged classifiers by assessing their putative redundancy with sociodemographic and clinical features, as well as their prognostic significance on treatment outcomes. We hypothesized that the reward task engaged and reward task disengaged groups would differ in their response to an 8-week course of antidepressant treatment (sertraline) or placebo, as measured by the 17-item clinician-rated Hamilton Depression Rating Scale (HAMD-17) [10].
This report details our implementation of RBD in HC and MDD participants who were enrolled in the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study [11]. To test our hypotheses, we addressed these specific questions: 1.
Does RBD in block 2 differ between HC and MDD participants? 2.
Among MDD participants, is there a level at which RBD is high enough to be considered objectively disengaged when compared to HC participants? 3.
Do reward task engaged and reward task disengaged MDD participants differ in sociodemographic or clinical features? 4.
Do reward task engaged and reward task disengaged MDD participants respond differently to sertraline versus placebo?

Design and Participants
The multisite two-phase, double-blind, EMBARC trial randomized 296 participants with MDD to sertraline or placebo (stage I) with the aim of identifying biosignatures of antidepressant response in MDD [11]. Additionally, the study included 40 HC participants who-as with the MDD group-were assessed at baseline and 1 week post baseline. This secondary analysis included 195 MDD and 40 HC participants with complete PRT data following the EMBARC protocol for model development using a randomly selected twothirds of the total depressed cohort, with validation on the remaining one-third to occur at a later time. All participants were 18-65 years old. The study was approved by the Institutional Review Board at each site. All participants signed informed consent. See Table 1 for demographics. Note: Major depressive disorder (MDD); standard deviation (SD); * the probabilistic reward task data from one of these participants was suggestive of a lab error and thus was omitted from model development; ** eight participants had too many MDD episodes to count and were not included in this entry.
MDD participants were diagnosed using the Structured Clinical Interview for DSM-IV (SCID) [12] and were not undergoing treatment for MDD. Participants were excluded if they scored <14 on the Quick Inventory of Depressive Symptomatology-Self Report (QIDS-SR) [13], if any other mental disorder was primary to MDD, if they had a lifetime history of an Axis-I mood disorder, psychotic disorder, or eating disorder, or did not score within the normal IQ range on the Wechsler Abbreviated Scale of Intelligence. For other exclusion criteria, refer to Trivedi et al. [11]. HC participants had no history of psychiatric diagnoses based on the SCID, and scored ≤8 on the QIDS-SR. Scores from the HAMD-17 [10] and the Snaith-Hamilton Anhedonia Scale [14] (SHAPS) were also collected to track MDD severity and measure anhedonia, respectively.

Procedures
All data were collected at four sites: Columbia University (CU), Massachusetts General Hospital/McLean Hospital (MG), University of Michigan (UM), and University of Texas Southwestern Medical Center (TX). These data can be accessed on the National Institute of Mental Health Data Archive website (https://nda.nih.gov/edit_collection.html?id=2199, accessed on 30 April 2015).

Probabilistic Reward Task (PRT)
PRT methods for EMBARC followed the methods of Pizzagalli et al. [3]. Briefly, participants viewed schematic faces with either a long or short mouth (Supplementary Figure S1). For each face presented, participants made one of two responses: (1) "long" when they judged the mouth to be the longer of the two stimuli, or (2) "short" when they judged the mouth to be the shorter one. The potential reward for correct identification was 20 cents. Correct identification of mouth length was not always rewarded, with one mouth length (rich stimulus) rewarded three times more frequently than the other (lean stimulus). Among HCs, this asymmetrical reinforcement schedule induces a response bias towards the rich stimulus (i.e., reward learning) [2,3,[15][16][17]. Participants completed a practice phase, followed by 200 trials divided evenly into two blocks.

RBD Subgrouping of MDD Participants
Participants were categorized as either reward task engaged or reward task disengaged based on their block 2 RBD measurement (i.e., after they had been given the opportunity to process the asymmetric reinforcement schedule). In order to objectively specify an RBD score indicative of clinical impairment, quadratic discriminant analysis [18] was implemented to obtain a block 2 RBD cutoff value that most effectively distinguished the MDD group from the HC group (p < 0.05). This cutoff value was compared to the threshold of HC participants, which has been used previously to define the normal range of functioning in MDD participants [19].

Statistical Analyses
RBD scores between HC and MDD participants during block 2 were compared using a t-test with Cochran-Cox approximation to account for unequal variance. MDD participants were then classified into reward task engaged and reward task disengaged groups based on a threshold identified using HCs (described above).
Baseline clinical and sociodemographic features were compared using chi-squared or t-tests. The features selected for comparison included age, sex, race, marital status, education level, employment status, number of depressive episodes (divided into three quantiles), length of current depressive episode (≤6 months, 7-24 months, >24 months), monthly income in USD (<2000, 2000-4000, >4000), number of medical comorbidities (divided into quartiles), as well as HAMD-17, QIDS-SR, and SHAPS scores. A 6 (time: 1, 2, 3, 4, 6, and 8 weeks) × 2 (treatment group: sertraline or placebo) × 2 (MDD RBD-subgroup: reward task engaged, reward task disengaged) repeated-measures linear mixed-model was conducted, and all interactions were used to test the moderator effect (differential prediction of treatment outcome with sertraline versus placebo) of RBD status as indicated by HAMD-17 scores across time. HAMD-17 scores were measured at baseline and weeks 1, 2, 3, 4, 6, and 8. Time was log-transformed, a random effect was assigned to each participant (with unstructured variance-covariance matrix), and a spatial power variance-covariance structure for the repeated measures error term was used. To control for baseline depression severity, baseline HAMD-17 scores were dropped from the model and entered as a covariate. A random intercept was used.

Among MDD Participants, Is
There a Level at Which RBD Is High Enough to Be Considered Objectively Impaired (or "Disengaged") When Compared to HC Participants?
Yes. Quadratic discriminant analysis yielded a cutoff of 5.08 for block 2 RBD. Of the 195 participants with MDD, 137 (70.25%) had block 2 RBD < 5.08 (classified as reward task engaged), while 58 (29.75%) had block 2 RBD ≥ 5.08 (classified as reward task disengaged). A chi-square test revealed that the block 2 RBD cutoff criterion significantly differentiated the MDD and HC groups, (χ 2 (1, n = 235) = 5.03, p = 0.025). Of the 40 HCs examined, 35 were reward task engaged and only 5 were reward task disengaged. Sensitivity and specificity for meeting DSM-IV MDD criteria were 58.7%, (95% CI [53.5-64.1%]) and 88.9% (95% CI [76.0-96.3%]), respectively. See Supplementary Materials for the comparisons of RBD to traditional PRT measures. Please note however, that traditional PRT measures such as Response Bias and Discriminability share many closely-related computational variables with RBD, and these comparisons between these measures are difficult to interpret.

Do Reward Task Engaged and Reward Task Disengaged MDD Participants Differ in Sociodemographic or Clinical Features?
No. Reward task engaged and reward task disengaged MDD participants did not differ on any sociodemographic or clinical feature (Table 2), which indicates that the RBD metric provides incremental information. Notably, reward task engaged and reward task disengaged MDD participants did not differ in self-reported anhedonia (SHAPS:

Do Reward Task Engaged and Reward Task Disengaged MDD Participants Respond
Differently to Sertraline versus Placebo?
Yes. There was a significant 3-way interaction, 7 (time) × 2 (treatment) × 2 (RBD group); (F(1293) = 4.33 p = 0.038). To visualize, we plotted the changes in HAMD-17 scores with sertraline and placebo separately for the reward task engaged and reward task disengaged groups. As shown in Figure 1B, HAMD-17 scores of the reward task engaged group did not differ between placebo and sertraline treatments over time. However, in the reward task disengaged group ( Figure 1A), the rate of change in HAMD-17 scores differed significantly between the sertraline and placebo treatment groups, with marked separation observed by week 6. Specifically, those MDD patients who were reward task disengaged had greater reductions in their HAMD-17 scores as treatment progressed when on sertraline compared to placebo.

Discussion
In this study, we describe for the first time a newly developed novel measure of reward impairment, Reward Behavior Disengagement or RBD, and demonstrate its clinical significance. To our knowledge, this is the first study to utilize a neuroeconomic approach to identify an objective, clinically relevant measure of reward impairment, especially in the context of a placebo-controlled clinical trial of MDD. Unlike current symptom measures such as the SHAPS, RBD is an objective measure of reward impairment, thereby

Discussion
In this study, we describe for the first time a newly developed novel measure of reward impairment, Reward Behavior Disengagement or RBD, and demonstrate its clinical significance. To our knowledge, this is the first study to utilize a neuroeconomic approach to identify an objective, clinically relevant measure of reward impairment, especially in the context of a placebo-controlled clinical trial of MDD. Unlike current symptom measures such as the SHAPS, RBD is an objective measure of reward impairment, thereby directly addressing the issue of the heterogeneity of anhedonic pathology seen in depression [6,8]. RBD differed significantly between HC and MDD participants, and discriminant analysis identified a subgroup of MDD participants for whom RBD was elevated beyond that seen in HCs. Finally, the clinical importance of RBD is highlighted by the moderating effect of the RBD depression classifiers reward task disengaged and reward task engaged on acute-phase treatment outcomes, an effect which was independent of socioeconomic or clinical features.
Reward impairment as described by the SHAPS and RBD differ fundamentally from each other, and SHAPS scores were not predictive of RBD classification. Anhedonia, characterized in an economic framework, can arise when the effort associated with a normally rewarding activity is overly penalized, and ultimately results in the decision to not engage in the activity. This over-penalization of costs is fairly easy to objectively quantify (e.g., RBD). In contrast, the SHAPS simply asks patients if they enjoy items that are presumed to be universally indicative of hedonic capacity, for example, "reading a book, magazine, or newspaper" or "the smell of a fresh sea breeze". This methodology does not account for the fact that the desirability of these activities is not necessarily related to disease status or anhedonic impairment. It thus may be unsurprising that SHAPS scores were not predictive of RBD classification. Although RBD is unlikely to be the perfect index of hedonic impairment, its ability to objectively assess patients is a strength that measures such as the SHAPS do not possess.
Given the nascence of the RBD measure, it is difficult to assess its construct validity at this time. However, some degree of construct validity may be implied by the work of Lawlor et al. which was performed on the same EMBARC PRT data [20]. This work showed that the impaired PRT performance of depressed individuals was likely due to an altered evidence accumulation process. Although examining the relationship of these prior findings with RBD was not a goal of this manuscript, doing so may be worthwhile as one would expect high RBD to negatively affect the quality of an individual's evidence accumulation process (or vice versa).
RBD was not reflected in common sociodemographic features or subjective clinical severity scales, further highlighting the importance of adopting objective symptom measures that have incremental validity into psychiatric care. Interestingly, response rates of sertraline and placebo were similar among reward task engaged participants. However, among reward task disengaged participants, sertraline was more effective than a placebo even after controlling for baseline depression severity, thereby identifying a subgroup of patients with MDD who would uniquely benefit from antidepressant treatment. This has significant clinical implications. High rates of placebo response have resulted in the failure of multiple phase II and III medication trials [21]. RBD may enrich future clinical trials by identifying patients who are less likely to respond to a placebo. Additionally, RBD can be utilized in clinical practice to identify instances of MDD likely to respond equally well to treatments with lower side effects as compared to antidepressant medications, regardless of depression severity.

Limitations and Future Directions
This work only assessed the predictive power of RBD regarding response to treatment in an unmedicated, moderate-to-severely depressed sample of MDD patients who were able to tolerate rigorous testing sessions across various days; thus, RBD findings may not extend to other MDD samples. Although promising, validation within this cohort and in future studies in separate cohorts is needed. An interesting direction for future research would be the assessment of RBD as a predictor of differential treatment outcome across several additional treatment paradigms (e.g., different antidepressant medications or medication versus psychotherapy). Additionally, we did not examine the test-retest reliability of RBD for behavioral or outcomes data, which should be a key goal of any future RBD study. This additional validation testing would be reassuring given that RBD did not trend with the SHAPS or QIDS-SR.

Conclusions
Reward Behavior Disengagement is a novel, objective measure that identifies a new, prognostically significant category of reward impairment in depression. RBD-based classifiers were independent of common sociodemographic and clinical measures, yet uniquely identified differences in acute-phase treatment response to sertraline versus placebo. RBD may represent one of the first clinically relevant neuroeconomic biomarkers of anhedonic impairment.