Next Article in Journal
Improving Outcomes by Implementing a Pressure Ulcer Prevention Program (PUPP): Going beyond the Basics
Previous Article in Journal
Comparison of a Restricted and Unrestricted Vegan Diet Plan with a Restricted Omnivorous Diet Plan on Health-Specific Measures
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 3
Issue 3
10.3390/healthcare3030556
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

The Promises and Challenges of Ecological Momentary Assessment in Schizophrenia: Development of an Initial Experimental Protocol

by
Brandon A. Gaudiano
1,*,
Ethan Moitra
2,
Stacy Ellenberg
1 and
Michael F. Armey
1
1
Butler Hospital and Warren Alpert Medical School of Brown University, 345 Blackstone Blvd., Providence, RI 02906, USA
2
Warren Alpert Medical School of Brown University, Box G-BH, Providence, RI 02912, USA
*
Author to whom correspondence should be addressed.
Healthcare 2015, 3(3), 556-573; https://doi.org/10.3390/healthcare3030556
Submission received: 27 March 2015 / Revised: 19 May 2015 / Accepted: 9 July 2015 / Published: 15 July 2015
Browse Figure
Versions Notes

Abstract

:
Severe mental illnesses, including schizophrenia and other psychotic-spectrum disorders, are a major cause of disability worldwide. Although efficacious pharmacological and psychosocial interventions have been developed for treating patients with schizophrenia, relapse rates are high and long-term recovery remains elusive for many individuals. Furthermore, little is still known about the underlying mechanisms of these illnesses. Thus, there is an urgent need to better understand the contextual factors that contribute to psychosis so that they can be better targeted in future interventions. Ecological Momentary Assessment (EMA) is a dynamic procedure that permits the measurement of variables in natural settings in real-time through the use of brief assessments delivered via mobile electronic devices (i.e., smartphones). One advantage of EMA is that it is less subject to retrospective memory biases and highly sensitive to fluctuating environmental factors. In the current article, we describe the research-to-date using EMA to better understand fluctuating symptoms and functioning in patients with schizophrenia and other psychotic disorders and potential applications to treatment. In addition, we describe a novel EMA protocol that we have been employing to study the outcomes of patients with schizophrenia following a hospital discharge. We also report the lessons we have learned thus far using EMA methods in this challenging clinical population.

1. Introduction and Background

1.1. Treatment Nonadherence Is a Major Public Health Problem in Schizophrenia

Patients with psychosis are at an increased risk for treatment nonadherence, which is a costly problem. Schizophrenia is among the top ten causes of disability worldwide [1], with estimated treatment costs of $62.7 billion per year [2]. Furthermore, research indicates that 50% of patients with schizophrenia are medication nonadherent [3]. Nonadherence is predictive of poorer outcomes, including relapse, rehospitalization, self/other harm, and homelessness [4]. It is estimated that improved adherence could save over $100 million per year in inpatient costs for Medicare alone [5]. In addition to medication nonadherence, poor behavioral adherence (e.g., missed appointments, drop out) to other treatments (e.g., psychotherapy, case management) also puts patients at high risk for negative outcomes such as relapse and hospitalization [6].
The post-hospital period is a critical time to study adherence because it is particularly prevalent and problematic after discharge. Medication nonadherence, session nonattendance, and drop out increase risk for rehospitalizations. In addition, recent hospitalization predicts reduced adherence [7,8,9] and the transition from inpatient to outpatient treatment produces the highest risk of nonadherence and drop out [6,10]. Recently hospitalized patients are adjusting to new medications, dosages, and new outpatient treatments. Cost, time, and other barriers limit the ability of clinicians to monitor successful transition from inpatient to outpatient care, leading to adherence problems.

1.2. Our Knowledge of Adherence Predictors and Interventions Is Limited

Research has identified a number of different adherence predictors [11]. However, little is known about the relative contribution of these variables when combined, and even less is known about how these barriers fluctuate in the patient’s natural environment. Prior studies are mostly cross-sectional/retrospective in nature [11], and few have examined factors that influence day-to-day adherence. Retrospective assessment is problematic because memory bias may influence self-reporting [12]. In addition, many adherence barriers fluctuate according to environmental circumstances and mood states, making them difficult to assess out of context [13,14]. Previous retrospective research suggests that treatment complexity [15,16]; patient-provider alliance [17,18]; treatment attitudes/beliefs [19,20]; cost/access [21]; and medication side effects [19,22] are important to consider in adherence to treatments. Situationally-cued factors such as self-stigma [17,23], environmental stressors and inadequate support system [24], forgetfulness [25], and reduced insight [26,27,28] predispose patients to nonadherence. Lastly, psychological symptoms and their expression are predictive of adherence, with situationally varying emotional states [11], substance use [11], and positive symptoms [29] predicting nonadherence.
Traditional psychosocial adherence interventions for schizophrenia have produced mixed results to date [4,30,31]. Psychoeducational interventions alone generally have been found to be ineffective for improving adherence in schizophrenia [31]. Compliance therapy (with cognitive-behavioral and motivational strategies) showed promise for reducing nonadherence in an initial trial [32], but there was a subsequent failure to replicate findings [33]. A recently developed treatment that combines motivational and behavioral strategies tailored to target different problems related to nonadherence produced significantly improved service engagement and medication adherence compared with treatment as usual in patients with psychotic disorders without substance abuse [3]. Other behavioral and family approaches have shown promise in initial studies [4,30,31]. However, the most recent Patient Outcomes Research Team guidelines concluded: “there is insufficient evidence to recommend any specific intervention to promote adherence to antipsychotic medications among persons with schizophrenia” [34] p. 61. Without advancing the field’s knowledge of adherence barriers, there is little hope for improving the effectiveness of adherence interventions. Furthermore, few adherence studies have addressed the issue of behavioral adherence, as it differs from medication adherence [35]. Behavioral forms of adherence (e.g., appointment attendance) are critical in maintaining treatment gains, preventing relapse [36], and improving medication adherence [37]. Research is needed to examine adherence predictors in vivo to identify dynamic fluctuations and the influences of situational/contextual factors to improve prediction of adherence behaviors.

1.3. Ecological Momentary Assessment (EMA) Can Improve Our Understanding of Nonadherence

Newer technologies can be used to develop a more ecologically-valid understanding of adherence. The measurement of adherence barriers, including social support, stressors, substance use, affect, and psychotic symptoms, is best suited to an approach that is sensitive to dynamic changes in adherence behaviors in context. This can be provided through real-time assessment using experience sampling or ecological momentary assessment (EMA) [38]. Mobile devices and other personal digital assistants have become the ideal technological platform for collecting in vivo data. EMA has been shown to be reliable and valid for measuring adherence in several studies, but, to date, mostly in pediatric populations (e.g., cystic fibrosis) [39,40].
Preliminary research supports the short-term feasibility and acceptability of EMA in psychosis via mobile devices [41,42]. Various studies show that about 90% of people with schizophrenia accept and can be trained to use EMA and that compliance is comparable and sometimes superior to that of nonclinical populations (completing at least 75% of assessments) [25,42,43,44,45]. Most importantly, EMA has demonstrated incremental validity over traditional retrospective reports in assessing symptoms associated with schizophrenia, with one study finding that spikes in daily paranoia were captured via EMA in patients who reported low paranoia in traditional, retrospective assessments [46]. Another study demonstrated the incremental validity of EMA compared to retrospective symptom reports in individuals with schizophrenia compared to healthy controls [43]. In addition, Oorschot et al. [47] found inconsistent reporting of symptomatology between ecological assessments and clinical assessments on measures of delusions and auditory hallucinations, but not on measures of suspiciousness and visual hallucinations, highlighting the importance of capturing a moment-to-moment assessment of specific symptoms.
Furthermore, EMA can be used to identify factors that could lead to refinements in psychosocial interventions for schizophrenia. Ecological momentary interventions provide moment-to-moment self-management techniques to patients, cued by triggers from ecological assessments. One pilot study showed the feasibility and acceptability of using ecological momentary intervention via mobile devices with schizophrenic patients, and demonstrated low dropout rates and high satisfaction [48]. Montes et al. [49] used short message service (SMS) to provide outpatients with schizophrenia with daily medication reminders for three months. As a result, adherence and attitudes towards medications were both significantly improved as compared with the control group. Furthermore, improvements were seen in cognitive, global, and negative symptoms. The current project builds on this work by expanding and improving assessment of adherence barriers and using data to identify contextual modifiable predictors to target in future interventions.

1.4. Rationale and Hypotheses for Current Project

Our study aims to advance the field’s understanding of factors that can limit the impact of treatments due to nonadherence via the assessment of adherence barriers in the patient’s natural environment. Patients with psychosis often require long-term adherence to medications and other outpatient therapies, most critically upon discharge from the hospital. Reduced adherence to these treatments and premature drop-out both put individuals at high risk for acute hospitalization and other negative outcomes. Previous research has examined a number of risk factors for nonadherence, but little is known about how these variables fluctuate in the moment that can be better targeted by future self-management interventions. The use of new technologies can lead to an improved understanding of the factors that place patients at high risk for nonadherence and acute hospitalization. EMA is the ideal approach to fill this research gap. Initial steps have been taken to demonstrate that EMA is a feasible and acceptable assessment procedure for community-dwelling patients. However, we are not aware of previous EMA studies in patients with schizophrenia during the critical period from inpatient to outpatient treatment, which may have unique risk factors. The next step is to further refine EMA specifically focused on post-hospital adherence to guide future technology-assisted intervention development.
In the present study, our primary outcomes of interest will be (a) medication adherence, based on a multi-modal assessment using electronic monitoring, pill counts, and self-report; and, (b) behavioral adherence to outpatient treatments, measured by self-report of treatment utilization. We hypothesize that nonadherence to either outcome will be predicted by several dynamic ecologically assessed variables: (a) occurrence of positive psychotic symptoms; (b) negative affect; (c) occurrence of stressful life events; (d) lack of social support; (e) experience of medication side effects; (f) poor working alliance with the treatment provider(s); (g) negative medication/treatment beliefs; (h) substance use; and (i) avoidance-based psychological coping.

2. EMA Study Procedures

2.1. Sample Selection

Participants in this study will consist of 60 patients recruited during a psychiatric hospitalization. We used Zhang and Wang’s simulation procedure to estimate our needed sample size based on our main outcomes of medication and behavioral adherence [50]. Our simulation assumed seven repeated assessments for EMA data, a moderate correlation between slopes and intercepts (r = 0.30), a total missing data rate of ~20%, and 2500 simulation replications per sample size. The proposed sample of participants should be adequate (Power = 0.80) to detect a medium (clinically significant) effect (Cohen’s d = 0.425) for the full sample. Based on our previous experience with this population, we anticipate <15% study attrition. We plan to recruit additional patients to compensate for potential drop-out where possible. Inclusion criteria are: (1) currently hospitalized; (2) DSM-5 criteria for a psychotic-spectrum disorder (i.e., schizophrenia, schizoaffective disorder, delusional disorder, schizophreniform disorder, psychosis NOS) or a mood disorder (major depression or bipolar disorder) with psychotic features based on diagnostic interview (Structured Clinical Interview for DSM); (3) 18 years or older; (4) prescribed oral antipsychotic medication; and (5) ability to speak and read English sufficiently to complete the assessments. Exclusion criteria are: (1) planned discharge to supervised living settings or participation in outpatient adherence programs (e.g., medication packaging/monitoring); (2) pregnancy or medical condition contraindicating use of antipsychotics (e.g., dementia); or (3) homelessness.
We chose to expand our recruitment to include patients with psychotic mood disorders for several reasons. First, we were interested more broadly in studying the interactions among positive psychotic symptoms, mood symptoms, and treatment adherence across different levels of severity and comorbidities. Second, we were recruiting patients from an inpatient setting where initial diagnosis can be unclear and patients often transition from psychotic mood disorders to primary psychotic disorders over time. Thus, we were hoping to be able to “catch” this process as it develops in some patients. Third, EMA has widely been studied in patients with primary mood disorders [51], but not those with mood disorders with psychotic features. However, we believe that the predictors of nonadherence may be very similar across groups. Furthermore, in the current study, it is important to note that current or past risk for self-harm/suicidality and harm to others are assessed at intake but are not exclusion criteria. As needed, licensed study clinicians conduct safety assessments and provide appropriate intervention (e.g., referral to emergency services, community treatments resources) for patients indicating risk at the baseline and follow-up assessments.

2.2. Feasibility of EMA

We assessed the acceptability/feasibility of our EMA protocol in five inpatients with schizophrenia at our recruitment site. Patients were trained to use the device and complete the EMA assessment. We collected data from four patients who reported willingness to use the device. They reported (scale = 1–5) that they liked using the device (M = 4; SD = 1) and it was understandable (M = 5; SD = 0), interesting (M = 4; SD = 1.4), and respectful (M = 4.75; SD = 0.5). We obtained feedback from one patient with delusions about technology unwilling to use device but willing to answer the assessment questions in another form (e.g., diaries) and found them acceptable. We used this information to further refine our protocol.

2.3. Choice of EMA Variables

Treatment adherence is multi-determined. We chose to employ a “risk reduction” model similar to those utilized in heart disease and diabetes [52,53,54,55]. Modifying risk factors is hypothesized to decrease the incidence of unwanted outcomes. Some known predictors of nonadherence are relatively static and difficult or impossible to change (e.g., demographics, treatment access/costs). These variables are not assessed via EMA. For EMA, we selected targets that are: (a) potentially malleable, (b) related to adherence, and (c) could be impacted by future intervention strategies. Data suggest the following variables are relevant to EMA, given their dynamic nature and relationship with adherence: positive psychotic symptoms [29]; affect [11]; daily stressors [24]; social support [24]; side effects [25]; working alliance with the treatment provider [17,18]; negative medication/treatment beliefs [20,56]; substance use [11]; and psychological coping [57]. See Table 1 for description of EMA protocol and sample questions. When possible, we selected items used successfully in previous EMA research (see Table 1 for references). When this was not possible, we adapted EMA items from reliable and valid measures that were otherwise too long for EMA administration. This is a common strategy in EMA research, which often requires modifications to psychometric assumptions given the frequency and brevity of these assessments compared with traditional paper-and-pencil measures [58,59]. In other words, the primary aim of EMA involves a tailored measurement approach that is adapted specifically to the target population and goals of the assessment.
Table
Table 1. Ecological Momentary Assessment (EMA) Protocol.
Table 1. Ecological Momentary Assessment (EMA) Protocol.
ScheduleDynamic VariablesSample Items
Random AssessmentContextWhere are you right now? [42]
Substance UseHave you used any of the following substances?
AffectRate the following words based on how you feel right now [60].
Psychotic SymptomsWas someone spying on you or plotting against you? [42]
Psychological CopingHow did you deal with any difficulties you were having? [61]
Psychological AcceptanceI simply noticed my feelings and continued with what I was doing [62].
StressorsHas something stressful or bad happened in your life? [63]
Social SupportI am getting the emotional help and support I need from my family or friends [64].
End of Day AssessmentBehavioral AdherenceDid you attend any treatment appointments today?
Medication AdherenceI’ve taken my psychiatric medications as prescribed today.
Therapeutic AllianceDo you and your treatment provider understand each other? [65]
Reasons for NonadherenceMy medications make me feel strange or “doped up.” [66,67,68,69]
Medication Side EffectsHow bothered were you by these side effects? [70]
We propose to understand adherence behaviors within a contextual behavioral science (CBS) framework, which is the basis of Acceptance and Commitment Therapy (ACT), a newer cognitive-behavioral approach that has empirical support for treating psychosis [57,71,72]. CBS is an inductive, bottom up approach that emphasizes intervention development that is tightly linked to principles of behavior change derived from basic research [73]. A contextual approach “views psychological events as ongoing actions of the whole organism interacting in and with historically and situationally defined contexts” (p. 4) [74]. This framework suggests that an individual’s behavior change is dependent upon the dynamically changing internal and external environment and that detrimental behaviors (i.e., nonadherence) occur most often when individuals are attempting to avoid discomfort, despite how avoidance can negatively impact them. EMA is the ideal way to model this framework. We believe that our proposed EMA variables would be potentially modifiable based on future mobile interventions [75] based on CBS and ACT. A mobile intervention is a “therapist in your pocket” and can provide self-management skills by delivering algorithm-based, personalized coping strategies to target adherence barriers.

2.4. EMA Procedures

We have been piloting a version of our EMA protocol using the open source, MyExperience Tool [76] (http://myexperience.sourceforge.net/) software package to be administered via Windows-compatible mobile devices. EMA consists of: (1) Random schedule: allows for daytime hours (9 am–9 pm) to be split into three, 4-h segments, scheduling a random sample in each of these three segments. Each assessment takes 5–10 min. Each assessment is at least 30 min removed from a prior/subsequent assessment. Random assessments focus on variables that are likely to fluctuate throughout the day, including substance use, affect, psychotic symptoms, coping with symptoms, and experiences of environmental stressors. (2) End of Day schedule: Appointment attendance with mental health providers (i.e., psychiatrist, nurse, social worker, and therapist), working alliance, and reasons for nonadherence are assessed here. We also specifically assess medication adherence and experiences of medication side effects at the end of each day. All participants undergo an initial EMA training with research staff, in which they practice responding to EMA and familiarize themselves with the device. To ensure that participants who are highly symptomatic or cognitively impaired are able to successfully complete EMA, participants are instructed to contact us if questions arise during their use of the mobile device. On a weekly basis, a study staff person calls the patient and offers to answer questions or respond to any technical issues that might be interfering with his/her ability to respond to the EMA assessments. The contacts are kept as brief as possible (i.e., typically <5 min). As a backup plan, it is possible to consider replacing EMA with paper/pencil diaries in rare instances that remedial training does not resolve these issues.

2.5. Traditional (Non-EMA) Assessments

In addition to EMA-administered questions, we also use retrospective study measures completed via traditional pen-and-pencil methods. These are administered at baseline, 1- (i.e., the conclusion of the EMA phase), two-, and four-month follow-ups and are intended to complement our EMA battery. We also assess acceptability and potential burden of our assessment strategy. Additional study measures are listed in Table 2.
We use the Medication Event Monitoring System (MEMS) to assess adherence to the primary antipsychotic medication post-hospital discharge. MEMS uses electronic pill caps that record bottle openings/closings and the corresponding time/date. Electronic monitoring is recommended for objective assessment [77]; MEMS has been used in past schizophrenia studies with success [78]. We follow published guidelines for training patients to use MEMS and for analyzing data ([79,80]). Patients are instructed to bring their prescribed antipsychotic medication to our clinic and are trained by study staff to use MEMS at the start of the study. The research assistant demonstrates the use of MEMS in front of the patient and answers any questions the patient may have. Then, the patient is asked to transfer his/her medication to MEMS and label it according to the instructions on the original prescription. The research assistant supervises the process to ensure that the patient is using MEMS correctly. In the event that the patient is unable to return to our clinic to complete the MEMS (e.g., medication refills occurring between assessments or difficulty returning due to problems accessing transportation), the staff person assists the patient in using the MEMS system by providing additional instruction over the telephone as needed. Using MEMS is similar to patients using other pill organizer systems. Patients return to our clinic at one-, two-, and four-month assessments so that we can download the MEMS data from the cap. The patients are instructed to call the staff person on the number provided in the event of medication changes between visits, problems using MEMS, or any questions that may arise about using the caps during the study.
Table
Table 2. Traditional (Non-EMA) Study Measures.
Table 2. Traditional (Non-EMA) Study Measures.
MeasureTopicFormatTime Point
Treatment History Interview-4 (THI-4) [81]Treatment Utilization/
Behavioral Adherence		InterviewBL, 1, 2, 4
Medication Event Monitoring System (MEMS) [77]Medication AdherenceElectronicBL, 1, 2, 4
Pill Counts [82]Medication AdherenceBehavioralBL, 1, 2, 4
Brief Adherence Rating Scale (BARS) [78]Medication AdherenceInterviewBL, 1, 2, 4
Ratings of Medication Influences Scale (ROMI) [69]Reasons for NonadherenceInterviewBL, 1, 2, 4
Drug Attitude Inventory-10 (DAI-10) [68]Medication AttitudesSelf-ReportBL, 1, 2, 4
Antipsychotic Side Effect Checklist (ASC) [83]Medication Side EffectsInterviewBL, 1, 2, 4
Working Alliance Inventory-Short Version (WAI-S) [65]Doctor-Patient
Therapeutic Alliance		Self-ReportBL, 1, 2, 4
Brief Psychiatric Rating Scale (BPRS) [84]Psychiatric SymptomsInterviewBL, 1, 2, 4
Positive and Negative Affect Schedule-Expanded (State) (PANAS) [60]Mood StateSelf-ReportBL, 1, 2, 4
Consumer Experiences of Stigma Questionnaire (CESQ) [67]Stigma/DiscriminationSelf-ReportBL, 1, 2, 4
Life Events Assessment (LEA) [63]Stressful Life EventsInterviewBL, 1, 2, 4
Multidimensional Scale of Perceived Social Support
(MSPSS) [64]		Perceived Social SupportSelf-ReportBL, 1, 2, 4
Emotional Regulation Questionnaire (ERQ) [85]Psychological CopingSelf-ReportBL, 1, 2, 4
Acceptance and Action Questionnaire-II (AAQ) [86]Psychological Flexibility/ Experiential AvoidanceSelf-ReportBL, 1, 2, 4
World Health Organization Disability Assessment Schedule 2.0–12 Item Version (WHODAS 2.0) [87]Psychosocial FunctioningSelf-ReportBL, 1, 2, 4
Addiction Severity Index (ASI) [88]Illicit Drug UseInterviewBL, 1, 2, 4
Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) [89]Hazardous DrinkingSelf-ReportBL, 1, 2, 4
Note. Assessment time points: baseline (BL), 1, 2, and 4 months post-hospitalization.
Some patients do not find MEMS acceptable and refuse to use it. In addition, patients may be unable to use MEMS depending on the method used to take medications (e.g., use of pill containers or blister packs). Therefore, we use pill counts as an alternative objective measure of adherence when MEMS is not available. We follow similar procedures as those described by Kalichman et al. [82]. In brief, patients are instructed to: (a) bring their primary antipsychotic medication being assessed for adherence to the assessment visit, (b) provide the medication bottle with prescription information for the research assistant to record, (c) report lost or gained pills since the previous count, (d) place the pills in the bottle on a disposable plate for counting, (e) count the pills on the plate, (f) double count the pills for accuracy, and (g) then replace the pills in the original bottle.

2.6. EMA Data Analysis Strategy

EMA produces large amounts of data and requires different strategies for analysis compared with traditional (less frequent) longitudinal assessment. We will initially explore the utility of EMA measures by identifying episodes of nonadherence. Using that point as an anchor, we will identify three ecological assessments prior to (E-3, E-2, E-1) and immediately following (E + 1, E + 2, E + 3) the nonadherence event (E). We will also randomly identify an adherence event and select three ecological assessment points before and after this adherence event as a comparison. We will then directly compare incidents of adherence to nonadherence using the dynamic adherence constructs identified above. Using latent growth curve modeling (LGCM; see Figure 1), we will estimate intercept (i.e., mean) and slope (i.e., linear rate of increase/decrease) for each construct, with the option to explore nonlinear outcomes (i.e., quadratic) should linear slope provide a poor fit to the data. We will fix our initial two time points, and estimate remaining time parameters. We will initially fit unconditional models to determine optimal change trajectories for each outcome, with useful measures of nonadherence evidencing a differing slope or intercept than adherence events. Following specification of LGCM, we then fit a series of iterative conditional growth models in which our parameters are regressed on selected predictor variables, allowing for the retention/rejection of each variable and the incremental utility of adding ecological adherence measures to traditional assessments.
Healthcare 03 00556 g001 1024
Figure 1. Model for one ecological assessment (EA) variable (e.g., psychotic symptoms) as a dynamic predictor of treatment adherence. “E” reflects nonadherence event, with + and − denoting assessments before or after event.
Figure 1. Model for one ecological assessment (EA) variable (e.g., psychotic symptoms) as a dynamic predictor of treatment adherence. “E” reflects nonadherence event, with + and − denoting assessments before or after event.
Healthcare 03 00556 g001

2.7. Future Ecological Momentary Intervention Development

After identifying the relative importance of our adherence predictors and the degree to which these variables change, we will be well-poised to draw upon our treatment expertise to develop a mobile device-based adherence intervention for a future project using an ecological momentary intervention approach. We will use our EMA protocol as a model, appending assessment of variables found to be salient in this study with brief interventions matched to modify their influence on adherence. For instance, if we find that psychological coping with stressors is particularly impactful on daily adherence, we will use EMA to probe for stressors when they occur, and automatically respond with a brief intervention to prompt appropriate coping strategies (e.g., cognitive, behavioral, motivational). Depending on device connectivity, it would be possible to electronically communicate nonadherence to providers in (or near) real-time, perhaps triggering telephone outreach by clinical staff to address patients’ concerns or other variables undermining adherence. An immediate response (and potential resolution) could prevent decompensation and reduce the risk of rehospitalization. Thus, our future intervention will be personalized to the individual and his/her daily experiences. Also, ecological momentary interventions have the potential to be highly disseminable once developed because they can have greater reach than traditional intervention approaches relying on individual therapists for delivery [90]. Once our mobile intervention is initially developed, we plan to test it out in community mental health settings to explore how it can be integrated into existing treatment programs to complement standard treatments and provide greater support to patients at home. We plan to solicit feedback from patients and providers about the acceptability and usability of our mobile intervention and to continue to develop it in an iterative fashion. This will ensure that our mobile intervention is being developed in ways that will prove ultimately usable by consumers and workable within current systems of care, following similar models used by other investigators [91].

3. Lessons Learned Thus Far

3.1. Choice of Mobile Devices and Programming

Successful EMA research depends on reliable hardware and software. We used the MyExperience Tool [76] to develop our EMA paradigm. MyExperience is an open-source, freely available software package that facilitates real-time data collection on mobile devices using XML coding. It runs on the Windows Mobile operating system, meaning that we were limited to Windows devices, rather than iOS (Apple, Cupertino, CA, USA) or Android-driven operating systems. We chose MyExperience because it is free, reliable, and simple. However, it should be noted that from the development of this study two years ago to the present, MyExperience has become somewhat outdated. This is common in the software industry as innovations in technology and programming move at a rapid pace. Newer software, such as Mobile EMA (mEMA; Rensen St. Lansing, MI, USA) or Apple’s new ResearchKit (Apple, Inc., Cupertino, CA, USA), can improve programming capabilities using more advanced features and provide more flexible data collection opportunities.
A recent Pew Center study showed that 88% of Americans over the age of 18 have a cellular phone, smartphone or otherwise [92]. Although we expected that most of our participants would have a mobile device, we could not guarantee their uniformity or reliability. Therefore, we chose to provide devices to our participants to standardize the delivery platform and ensure control over the software. Because of concerns that participants might damage or lose the devices we provided, plus potential risk of participants selling the devices, we sought devices that were affordable and somewhat “outdated,” meaning that their resale value was less than the potential monetary incentives we would provide for completion of EMA and returning the device to us.
As such, all participants were given older-generation Palm mobile phones. These were somewhat less user-friendly than newer smartphones (e.g., less responsive touch screen), but their benefits outweighed the costs of a more expensive, newer device. Another issue we ran into is that many mobile devices will only boot with a SIM card; because we did not enable the Palm devices with telephone or wireless service, we used “dummy” SIM cards to facilitate successful operation. The Palm devices and SIMs were purchased from the secondary market, making their availability more variable. The devices were enabled to contact emergency services (911). Although it is possible to transmit data from mobile devices to a research database, we chose not to enable wireless access as our funds were limited and our paradigm did not require real-time communication between patients and researchers. Instead, we incentivized patients $50 to return the devices to us, further reducing the risk of loss or resale.

3.2. Recruitment Strategies

We plan to recruit 60 adult patients referred from psychiatric units at our hospital. Initially we targeted inpatients only. However, over the course of our study, we expanded recruitment to the partial hospitalization program to ensure that we met recruitment goals and also to assess a wider range of functioning and symptoms in our sample. Our experience showed that partial hospital patients were generally better functioning than inpatients despite often having clinically significant psychotic symptoms.
A research assistant reviews new admissions via an electronic medical records system and obtains physician permission to approach the potential participant about the study. We learned that successful recruitment depends on using a collaborative recruitment approach and being sensitive to the patient’s clinical status at the time of recruitment. Because our study is assessment-only, we do not offer treatment to potential participants. In addition to being motivated to participate because of the compensation provided, we realized patients would be more willing to participate if they believed that their participation could contribute to the greater good. When approaching potential participants, the research assistant describes the study and emphasizes how disruptive and unpleasant the hospitalization process can be for many patients. The research assistant highlights the need to better understand what leads to hospitalization, noting our goal of helping people stay out of the hospital when possible. The priority is to engage the patient by acknowledging how he/she can help other patients. Because EMA is novel to most patients, the research assistant demonstrates how the study works on a device during the recruitment conversation. This shows how our study is different and perhaps more enjoyable than traditional pencil-and-paper assessment studies.
Given the nature of our target population, it is common for patients to be too sedated due to medication or too symptomatic to be fully attentive and receptive to the recruitment conversation at first. The research assistant works closely with hospital staff to determine the appropriate time to make first contact with patients, to recognize when they cannot make an informed consent decision, and to determine when to re-approach the patient. Every patient has the right to refuse participation, but we do not want them to make this decision without their full cognitive capacity.

3.3. Retention Strategies

We compensate participants for completion of the one-month EMA phase and returning the device ($50) and each two- and four-month in person follow-up assessment ($25). Patients are also incentivized $0.50 for each of the 120 EMA assessments completed for up to an additional $60 in compensation. Further, we maintain flexibility in scheduling and accommodating patients to encourage retention. As needed, we provide transportation to and from our research site and if needed, will conduct home visits to recover the device and complete assessments. Long before the potential loss of any participant via refusal or difficulty locating the person, the research assistant plays a significant role in the individual’s continued participation. Consistent with our recruitment strategies, we treat participation as a collaborative endeavor. We are courteous, empathetic, and professional with our participants and adapt our assessment procedures as needed to maintain good rapport.

3.4. Technical Troubleshooting with Patients

All participants undergo EMA training with research staff prior to going home with the device. In this training, participants practice responding to EMA and familiarize themselves with the device. We also provide a short EMA manual that answers frequently asked questions and troubleshoots common device issues. As mentioned, we call patients weekly during the EMA phase to troubleshoot any problems. These calls are not intended to remind the patient to complete assessments, but instead are focused on addressing technical issues as needed. Before issuing the devices to patients, all study staff field-tested the EMA protocol to ensure that the devices would function properly during the one-month EMA phase. In doing so, technical issues that were idiosyncratic to the devices were identified and anticipated when troubleshooting with patients. We were able to identify internal and external issues that would compromise our device’s functionality over time, including hardware issues such as SD cards being misplaced, or internal issues, such as having the clock fail to keep the correct time. Some patients reported that the phone failed to issue all four surveys over the course of a day. However, some patients suggested that this may be attributed to not always carrying the phone on their person. We found that restarting the system daily allowed for optimal performance, preventing complications that could arise from older generation phones.

4. Conclusions

In this article, we reviewed the promises and challenges of using EMA to better understand post-hospital outcomes in patients diagnosed with psychotic-spectrum disorders. These patients are at high risk for relapse, rehospitalization, and many other negative outcomes such as suicide. Most previous studies assessing factors leading to rehospitalization in this population are limited by use of retrospective reporting that may be biased or fail to capture the most important time-varying and immediate contextual variables predicting these negative outcomes. EMA is a feasible tool for better understanding treatment-relevant behaviors, symptoms, and functioning in patients with psychosis. Our initial experience using EMA with this population suggests that there are many potential challenges using mobile devices with patients with severe mental illness. However, these challenges are manageable with reasonable modifications to the protocol and, thus far, patients appear to enjoy using the devices and find them acceptable as a way to share their experiences. Our hope is that the EMA data that we collect will prove useful for designing future personalized, ecological self-management interventions using these same mobile devices to improve treatment adherence and engagement in patients with psychosis following discharge from a psychiatric hospitalization.

Acknowledgments

This work was supported by a grant from the National Institute of Mental Health (R21 MH102000) awarded to Brandon A. Gaudiano and Ethan Moitra.

Author Contributions

All authors contributed to the writing, reviewing, and revising of this manuscript in the order listed.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Murray, C.; Lopez, A. The Global Burden of Disease; World Health Organization (WHO): Geneva, Swizerland, 1996. [Google Scholar]
  2. Wu, E.Q.; Birnbaum, H.G.; Shi, L.; Ball, D.E.; Kessler, R.C.; Moulis, M.; Aggarwal, J. The economic burden of schizophrenia in the United States in 2002. J. Clin. Psychiatry 2005, 66, 1122–1129. [Google Scholar] [CrossRef] [PubMed]
  3. Staring, A.B.; van der Gaag, M.; Koopmans, G.T.; Selten, J.P.; van Beveren, J.M.; Hengeveld, M.W.; Loonen, A.J.; Mulder, C.L. Treatment adherence therapy in people with psychotic disorders: Randomised controlled trial. Br. J. Psychiatry 2010, 197, 448–455. [Google Scholar] [CrossRef] [PubMed]
  4. Byerly, M.J.; Nakonezny, P.A.; Lescouflair, E. Antipsychotic medication adherence in schizophrenia. Psychiatry Clin. N. Am. 2007, 30, 437–452. [Google Scholar] [CrossRef] [PubMed]
  5. Marcus, S.C.; Olfson, M. Outpatient antipsychotic treatment and inpatient costs of schizophrenia. Schizophr. Bull. 2008, 34, 173–180. [Google Scholar] [CrossRef] [PubMed]
  6. Ruscher, S.M.; de Wit, R.; Mazmanian, D. Psychiatric patients’ attitudes about medication and factors affecting noncompliance. Psychiatry Serv. 1997, 48, 82–85. [Google Scholar]
  7. Kreyenbuhl, J.; Slade, E.P.; Medoff, D.R.; Brown, C.H.; Ehrenreich, B.; Afful, J.; Dixon, L.B. Time to discontinuation of first- and second-generation antipsychotic medications in the treatment of schizophrenia. Schizophr. Res. 2011, 131, 127–132. [Google Scholar] [CrossRef] [PubMed]
  8. Mullins, C.D.; Obeidat, N.A.; Cuffel, B.J.; Naradzay, J.; Loebel, A.D. Risk of discontinuation of atypical antipsychotic agents in the treatment of schizophrenia. Schizophr. Res. 2008, 98, 8–15. [Google Scholar] [CrossRef] [PubMed]
  9. Novick, D.; Haro, J.M.; Suarez, D.; Perez, V.; Dittmann, R.W.; Haddad, P.M. Predictors and clinical consequences of non-adherence with antipsychotic medication in the outpatient treatment of schizophrenia. Psychiatry Res. 2010, 176, 109–113. [Google Scholar] [CrossRef] [PubMed]
  10. Bergen, J.; Hunt, G.; Armitage, P.; Bashir, M. Six-month outcome following a relapse of schizophrenia. Aust. N. Z. J. Psychiatry 1998, 32, 815–822. [Google Scholar] [CrossRef] [PubMed]
  11. Lacro, J.P.; Dunn, L.B.; Dolder, C.R.; Leckband, S.G.; Jeste, D.V. Prevalence of and risk factors for medication nonadherence in patients with schizophrenia: A comprehensive review of recent literature. J. Clin. Psychiatry 2002, 63, 892–909. [Google Scholar] [CrossRef] [PubMed]
  12. Bradburn, N.M.; Rips, L.J.; Shevell, S.K. Answering autobiographical questions: The impact of memory and inference on surveys. Science 1987, 236, 157–161. [Google Scholar] [CrossRef] [PubMed]
  13. Csikszentmihalyi, M.; Larson, R. Validity and Reliability of the Experience-Sampling Method. J. Nerv. Ment. Dis. 1987, 175, 526–536. [Google Scholar] [CrossRef]
  14. Ebner-Priemer, U.W.; Trull, T.J. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol. Assess. 2009, 21, 463–475. [Google Scholar] [CrossRef] [PubMed]
  15. Corrigan, P.W.; Liberman, R.P.; Engel, J.D. From noncompliance to collaboration in the treatment of schizophrenia. Hosp. Community Psychiatry 1990, 41, 1203–1211. [Google Scholar] [CrossRef] [PubMed]
  16. Davis, J.M.; Kane, J.M.; Marder, S.R.; Brauzer, B.; Gierl, B.; Schooler, N.; Casey, D.E.; Hassan, M. Dose response of prophylactic antipsychotics. J. Clin Psychiatry 1993, 54, 24–30. [Google Scholar] [PubMed]
  17. Fenton, W.S.; Blyler, C.R.; Heinssen, R.K. Determinants of medication compliance in schizophrenia: Empirical and clinical findings. Schizophr. Bull. 1997, 23, 637–651. [Google Scholar] [CrossRef] [PubMed]
  18. Olfson, M.; Mechanic, D.; Hansell, S.; Boyer, C.A.; Walkup, J.; Weiden, P.J. Predicting medication noncompliance after hospital discharge among patients with schizophrenia. Psychiatry Serv. 2000, 51, 216–222. [Google Scholar] [CrossRef]
  19. Meier, J.; Becker, T.; Patel, A.; Robson, D.; Schene, A.; Kikkert, M.; Barbui, C.; Burti, L.; Puschner, B. Effect of medication-related factors on adherence in people with schizophrenia: A European multi-centre study. Epidemiol. Psichiatry Soc. 2010, 19, 251–259. [Google Scholar] [CrossRef]
  20. Perkins, D.O.; Johnson, J.L.; Hamer, R.M.; Zipursky, R.B.; Keefe, R.S.; Centorrhino, F.; Green, A.I.; Glick, I.B.; Kahn, R.S.; Sharma, T.; et al. Predictors of antipsychotic medication adherence in patients recovering from a first psychotic episode. Schizophr. Res. 2006, 83, 53–63. [Google Scholar] [CrossRef] [PubMed]
  21. Mutsatsa, S.H.; Joyce, E.M.; Hutton, S.B.; Webb, E.; Gibbins, H.; Paul, S.; Barnes, T.R. Clinical correlates of early medication adherence: West London first episode schizophrenia study. Acta Psychiatry Scand. 2003, 108, 439–446. [Google Scholar] [CrossRef]
  22. Schennach-Wolff, R.; Jager, M.; Seemuller, F.; Obermeier, M.; Messer, T.; Laux, G.; Pfeiffer, H.; Naber, D.; Schmidt, L.G.; Gaebel, W.; et al. Attitude towards adherence in patients with schizophrenia at discharge. J. Psychiatry Res. 2009, 43, 1294–1301. [Google Scholar] [CrossRef] [PubMed]
  23. Fung, K.M.T.; Tsang, H.W.H.; Corrigan, P.W. Self-stigma of people with schizophrenia as predictor of their adherence to psychosocial treatment. Psychiatry Rehabil. J. 2008, 32, 95–104. [Google Scholar] [CrossRef] [PubMed]
  24. Hudson, T.J.; Owen, R.R.; Thrush, C.R.; Han, X.; Pyne, J.M.; Thapa, P.; Sullivan, G. A pilot study of barriers to medication adherence in schizophrenia. J. Clin. Psychiatry 2004, 65, 211–216. [Google Scholar] [CrossRef] [PubMed]
  25. Granholm, E.; Ben-Zeev, D.; Link, P.C.; Bradshaw, K.R.; Holden, J.L. Mobile Assessment and Treatment for Schizophrenia (MATS): A pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr. Bull. 2012, 38, 414–425. [Google Scholar] [CrossRef] [PubMed]
  26. Acosta, F.J.; Bosch, E.; Sarmiento, G.; Juanes, N.; Caballero-Hidalgo, A.; Mayans, T. Evaluation of noncompliance in schizophrenia patients using electronic monitoring (MEMS (R)) and its relationship to sociodemographic, clinical and psychopathological variables. Schizophr. Res. 2009, 107, 213–217. [Google Scholar] [CrossRef] [PubMed]
  27. Cuffel, B.J.; Alford, J.; Fischer, E.P.; Owen, R.R. Awareness of illness in schizophrenia and outpatient treatment adherence. J. Nerv. Ment. Dis. 1996, 184, 653–659. [Google Scholar] [CrossRef] [PubMed]
  28. Rittmannsberger, H.; Pachinger, L.; Keppelmuller, P.; Wancata, J. Medication adherence among psychotic patients before admission to inpatient treatment. Psychiatry Serv. 2004, 55, 174–179. [Google Scholar] [CrossRef]
  29. Hofer, A.; Kemmler, G.; Eder, U.; Honeder, M.; Hummer, M.; Fleischhacker, W.W. Attitudes toward antipsychotics among outpatient clinic attendees with schizophrenia. J. Clin. Psychiatry 2002, 63, 49–53. [Google Scholar] [CrossRef] [PubMed]
  30. Dolder, C.R.; Lacro, J.P.; Leckband, S.; Jeste, D.V. Interventions to improve antipsychotic medication adherence: Review of recent literature. J. Clin. Psychopharm. 2003, 23, 389–399. [Google Scholar] [CrossRef] [PubMed]
  31. Zygmunt, A.; Olfson, M.; Boyer, C.A.; Mechanic, D. Interventions to improve medication adherence in schizophrenia. Am. J. Psychiatry 2002, 159, 1653–1664. [Google Scholar] [CrossRef] [PubMed]
  32. Kemp, R.; Hayward, P.; Applewhaite, G.; Everitt, B.; David, A. Compliance therapy in psychotic patients: Randomised controlled trial. Br. Med. J. 1996, 312, 345–349. [Google Scholar] [CrossRef]
  33. O’Donnell, C.; Donohoe, G.; Sharkey, L.; Owens, N.; Migone, M.; Harries, R.; Kinsella, A.; Larkin, C.; O’Callaghan, E. Compliance therapy: A randomised controlled trial in schizophrenia. Br. Med. J. 2003. [Google Scholar] [CrossRef] [PubMed]
  34. Dixon, L.B.; Dickerson, F.; Bellack, A.S.; Bennett, M.; Dickinson, D.; Goldberg, R.W.; Lehman, A.; Tenhula, W.N.; Calmes, C.; Pasillas, R.M.; et al. The 2009 schizophrenia PORT psychosocial treatment recommendations and summary statements. Schizophr. Bull. 2010, 36, 48–70. [Google Scholar] [CrossRef] [PubMed]
  35. Gaudiano, B.A.; Weinstock, L.M.; Miller, I.W. Improving treatment adherence in bipolar disorder: A review of current psychosocial treatment efficacy and recommendations for future treatment development. Behav. Modif. 2008, 32, 267–301. [Google Scholar] [CrossRef] [PubMed]
  36. Colom, F.; Vieta, E.; Tacchi, M.J.; Sanchez-Moreno, J.; Scott, J. Identifying and improving non-adherence in bipolar disorders. Bipolar Disord. 2005, 7, 24–31. [Google Scholar] [CrossRef] [PubMed]
  37. Swartz, M.S.; Swanson, J.W.; Wagner, H.R.; Burns, B.J.; Hiday, V.A. Effects of involuntary outpatient commitment and depot antipsychotics on treatment adherence in persons with severe mental illness. J. Nerv. Ment. Dis 2001, 189, 583–592. [Google Scholar] [CrossRef] [PubMed]
  38. Bolger, N.; Davis, A.; Rafaeli, E. Diary methods: Capturing life as it is lived. Ann. Rev. Psychol. 2003, 54, 579–616. [Google Scholar] [CrossRef] [PubMed]
  39. Modi, A.C.; Lim, C.S.; Yu, N.; Geller, D.; Wagner, M.H.; Quittner, A.L. A multi-method assessment of treatment adherence for children with cystic fibrosis. J. Cystic Fibrosis 2006, 5, 177–185. [Google Scholar] [CrossRef] [PubMed]
  40. Quittner, A.L.; Modi, A.C.; Lemanek, K.L.; Ievers-Landis, C.E.; Rapoff, M.A. Evidence-based assessment of adherence to medical treatments in pediatric psychology. J. Pediatry Psychol. 2008, 33, 916–936. [Google Scholar] [CrossRef] [PubMed]
  41. Ben-Zeev, D.; Ellington, K.; Swendsen, J.; Granholm, E. Examining a cognitive model of persecutory ideation in the daily life of people with schizophrenia: A computerized experience sampling study. Schizophr. Bull. 2011, 37, 1248–1256. [Google Scholar] [CrossRef] [PubMed]
  42. Granholm, E.; Loh, C.; Swendsen, J. Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophr. Bull. 2008, 34, 507–514. [Google Scholar] [CrossRef] [PubMed]
  43. Ben-Zeev, D.; McHugo, G.J.; Xie, H.; Dobbins, K.; Young, M.A. Comparing retrospective reports to real-time/real-place mobile assessments in individuals with schizophrenia and a nonclinical comparison group. Schizophr. Bull. 2012. [Google Scholar] [CrossRef] [PubMed]
  44. Kimhy, D.; Delespaul, P.; Ahn, H.; Cai, S.; Shikhman, M.; Lieberman, J.A.; Malaspina, D.; Sloan, R.P. Concurrent measurement of “real-world” stress and arousal in individuals with psychosis: Assessing the feasibility and validity of a novel methodology. Schizophr. Bull. 2010, 36, 1131–1139. [Google Scholar] [CrossRef] [PubMed]
  45. Swendsen, J.; Ben-Zeev, D.; Granholm, E. Real-time electronic ambulatory monitoring of substance use and symptom expression in schizophrenia. Am. J. Psychiatry 2011, 168, 202–209. [Google Scholar] [CrossRef] [PubMed]
  46. Oorschot, M.; Lataster, T.; Thewissen, V.; Wichers, M.; Myin-Germeys, I. Mobile assessment in schizophrenia: A data-driven momentary approach. Schizophr. Bull. 2012. [Google Scholar] [CrossRef] [PubMed]
  47. So, S.H.; Peters, E.R.; Swendsen, J.; Garety, P.A.; Kapur, S. Detecting improvements in acute psychotic symptoms using experience sampling methodology. Psychiatry Res. 2013, 210, 82–88. [Google Scholar] [CrossRef] [PubMed]
  48. Depp, C.A.; Mausbach, B.; Granholm, E.; Cardenas, V.; Ben-Zeev, D.; Patterson, T.L.; Lebowitz, B.D.; Jeste, D.V. Mobile interventions for severe mental illness: Design and preliminary data from three approaches. J. Nerv. Ment. Dis. 2010, 198, 715–721. [Google Scholar] [CrossRef] [PubMed]
  49. Montes, J.M.; Medina, E.; Gomez-Beneyto, M.; Maurino, J. A short message service (SMS)-based strategy for enhancing adherence to antipsychotic medication in schizophrenia. Psychiatry Res. 2012, 200, 89–95. [Google Scholar] [CrossRef] [PubMed]
  50. Zhang, Z.Y.; Wang, L.J. Statistical power analysis for growth curve models using SAS. Behav. Res. Methods 2009, 41, 1083–1094. [Google Scholar] [CrossRef] [PubMed]
  51. Armey, M.F.; Schatten, H.T.; Haradhvala, N.; Miller, I.W. Ecological momentary assessment (EMA) of depression-related phenomena. Curr. Opin. Psychol. 2015, 4, 21–25. [Google Scholar] [CrossRef] [PubMed]
  52. Cockcroft, J.R.; Wilkinson, I.B.; Yki-Jarvinen, H. Multiple risk factor intervention in type 2 diabetes: An opportunity not to be missed. Diabetes Obes. Metab. 2001, 3, 1–8. [Google Scholar] [CrossRef] [PubMed]
  53. Hjermann, I. Multiple risk factor intervention in coronary heart disease. Ciba Found. Symp. 1985, 110, 110–125. [Google Scholar] [PubMed]
  54. Kuller, L.; Neaton, L.; Caggiula, A.; Falvo-Gerard, L. Primary prevention of heart attacks: The multiple risk factor intervention trial. Am. J. Epideiol. 1980, 112, 185–199. [Google Scholar]
  55. Stamler, J.; Vaccaro, O.; Neaton, J.D.; Wentworth, D. Diabetes, other risk factors, and 12-yr cardiovascular mortality for men screened in the Multiple Risk Factor Intervention Trial. Diabetes Care 1993, 16, 434–444. [Google Scholar] [CrossRef] [PubMed]
  56. Loffler, W.; Kilian, R.; Toumi, M.; Angermeyer, M.C. Schizophrenic patients’ subjective reasons for compliance and noncompliance with neuroleptic treatment. Pharmacopsychiatry 2003, 36, 105–112. [Google Scholar] [PubMed]
  57. Moitra, E.; Herbert, J.D.; Forman, E.M. Acceptance-based behavior therapy to promote HIV medication adherence. AIDS Care 2011, 23, 1660–1667. [Google Scholar] [CrossRef] [PubMed]
  58. Vilardaga, R.; McDonell, M.; Leickly, E.; Ries, R. Ecological momentary assessments in psychosis: A contextual behavioral approach to studying mindfulness and acceptance. In Incorporating Mindfulness and Acceptance into the Treatment of Psychosis: Current Trends and Future Directions; Gaudiano, B.A., Ed.; Oxford: New York, NY, USA, 2015; pp. 25–56. [Google Scholar]
  59. Hektner, J.M.; Schmidt, J.A.; Csíkszentmihályi, M. Experience Sampling Method: Measuring the Quality of Everyday Life; SAGE: London, UK, 2007. [Google Scholar]
  60. Watson, D.; Clark, L. The PANAS-X: Manual for the Positive and Negative Affect Schedule-Expanded Form; The University of Iowa: Iowa City, IA, USA, 1994. [Google Scholar]
  61. Vilardaga, R.; Hayes, S.C.; Atkins, D.C.; Bresee, C.; Kambiz, A. Comparing experiential acceptance and cognitive reappraisal as predictors of functional outcome in individuals with serious mental illness. Behav. Res. Ther. 2013, 51, 425–433. [Google Scholar] [CrossRef] [PubMed]
  62. Shawyer, F.; Ratcliff, K.; Mackinnon, A.; Farhall, J.; Hayes, S.C.; Copolov, D. The voices acceptance and action scale (VAAS): Pilot data. J. Clin. Psychol. 2007, 63, 593–606. [Google Scholar] [CrossRef] [PubMed]
  63. Goldberger, L.; Breznitz, S. Handbook of Stress: Theoretical and Clinical Aspects; The Free Press: New York, NY, USA, 1993; Volume 2. [Google Scholar]
  64. Zimet, G.D.; Dahlem, N.W.; Zimet, S.G.; Farley, G.K. The multidimensional scale of perceived social support. J. Pers. Assess. 1988, 52, 30–41. [Google Scholar] [CrossRef]
  65. Munder, T.; Wilmers, F.; Leonhart, R.; Linster, H.W.; Barth, J. Working Alliance Inventory-Short Revised (WAI-SR): Psychometric properties in outpatients and inpatients. Clin. Psychol. Psychother. 2010, 17, 231–239. [Google Scholar] [CrossRef] [PubMed]
  66. Birchwood, M.; Smith, J.; Drury, V.; Healy, J.; Macmillan, F.; Slade, M. A self-report insight scale for psychosis: Reliability, validity and sensitivity to change. Acta Psychiatry Scand. 1994, 89, 62–67. [Google Scholar] [CrossRef]
  67. Dickerson, F.B.; Sommerville, J.; Origoni, A.E.; Ringel, N.B.; Parente, F. Experiences of stigma among outpatients with schizophrenia. Schizophr. Bull. 2002, 28, 143–155. [Google Scholar] [CrossRef] [PubMed]
  68. Nielsen, R.E.; Lindstrom, E.; Nielsen, J.; Levander, S. DAI-10 is as good as DAI-30 in schizophrenia. Eur. Neuropsychopharmacol. 2012, 22, 747–750. [Google Scholar] [CrossRef] [PubMed]
  69. Weiden, P.; Rapkin, B.; Mott, T.; Zygmunt, A.; Goldman, D.; Horvitz-Lennon, M.; Frances, A. Rating of medication influences (ROMI) scale in schizophrenia. Schizophr. Bull. 1994, 20, 297–310. [Google Scholar] [CrossRef] [PubMed]
  70. Dibonaventura, M.; Gabriel, S.; Dupclay, L.; Gupta, S.; Kim, E. A patient perspective of the impact of medication side effects on adherence: Results of a cross-sectional nationwide survey of patients with schizophrenia. BMC Psychiatry 2012. [Google Scholar] [CrossRef] [PubMed]
  71. Gaudiano, B.A.; Herbert, J.D. Acute treatment of inpatients with psychotic symptoms using Acceptance and Commitment Therapy: Pilot results. Behav. Res. Ther. 2006, 44, 415–437. [Google Scholar] [CrossRef] [PubMed]
  72. Hayes, S.C.; Strosahl, K.D.; Wilson, K.G. Acceptance and Commitment Therapy: The Process and Practice of Mindful Change, 2nd ed.; Guilford: New York, NY, USA, 2012. [Google Scholar]
  73. Hayes, S.; Villatte, M.; Levin, M.; Hildebrandt, M. Open, aware, and active: Contextual approaches as an emerging trend in the behavioral and cognitive therapies. Ann. Rev. Clin. Psychol. 2011, 7, 141–168. [Google Scholar] [CrossRef] [PubMed]
  74. Hayes, S.C.; Luoma, J.B.; Bond, F.W.; Masuda, A.; Lillis, J. Acceptance and commitment therapy: Model, processes and outcomes. Behav. Res. Ther. 2006, 44, 1–25. [Google Scholar] [CrossRef] [PubMed]
  75. Shiffman, S.; Stone, A.A.; Hufford, M.R. Ecological momentary assessment. Ann. Rev. Clin. Psychol. 2008, 4, 1–32. [Google Scholar] [CrossRef]
  76. Froehlich, J. The Myexperience Tool. Avaliable online: http://myexperience.sourceforge.net/ (accessed on 20 June 2015).
  77. Osterberg, L.; Blaschke, T. Adherence to medication. N. Engl. J. Med. 2005, 353, 487–497. [Google Scholar] [CrossRef] [PubMed]
  78. Byerly, M.J.; Nakonezny, P.A.; Rush, A.J. The brief adherence rating scale (bars) validated against electronic monitoring in assessing the antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. Schizophrenia Res. 2008, 100, 60–69. [Google Scholar] [CrossRef] [PubMed]
  79. Ailinger, R.L.; Black, P.L.; Lima-Garcia, N. Use of electronic monitoring in clinical nursing research. Clin. Nurs. Res. 2008, 17, 89–97. [Google Scholar] [CrossRef] [PubMed]
  80. O’Carroll, R.; Dennis, M.; Johnston, M.; Sudlow, C. Improving adherence to medication in stroke survivors (IAMSS): A randomised controlled trial: Study protocol. BMC neurology 2010. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  81. Linehan, M.M.; Heard, H.L. Treatment History Interview (THI); University of Washington: Seattle, WA, USA, 1987. [Google Scholar]
  82. Kalichman, S.C.; Amaral, C.M.; Cherry, C.; Flanagan, J.; Pope, H.; Eaton, L.; Kalichman, M.O.; Cain, D.; Detorio, M.; Caliendo, A.; et al. Monitoring medication adherence by unannounced pill counts conducted by telephone: Reliability and criterion-related validity. HIV Clin. Trials 2008, 9, 298–308. [Google Scholar] [CrossRef] [PubMed]
  83. Weiden, P.J.; Miller, A.L. Which side effects really matter? Screening for common and distressing side effects of antipsychotic medications. J. Psychiatry Pract. 2001, 7, 41–47. [Google Scholar] [CrossRef]
  84. Overall, J.; Gorham, D. The brief psychiatric rating scale. Psychol. Rep. 1962, 10, 799–812. [Google Scholar] [CrossRef]
  85. Gross, J.J.; John, O.P. Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. J. Pers. Soc. Psychol. 2003, 85, 348–362. [Google Scholar] [CrossRef] [PubMed]
  86. Bond, F.W.; Hayes, S.C.; Baer, R.A.; Carpenter, K.M.; Guenole, N.; Orcutt, H.K.; Waltz, T.; Zettle, R.D. Preliminary psychometric properties of the Acceptance and Action Questionnaire-II: A revised measure of psychological inflexibility and experiential avoidance. Behav. Ther. 2011, 42, 676–688. [Google Scholar] [CrossRef] [PubMed]
  87. Federici, S.; Meloni, F.; Mancini, A.; Lauriola, M.; Olivetti Belardinelli, M. World Health Organisation disability assessment schedule II: Contribution to the Italian validation. Disabil. Rehabil. 2009, 31, 553–564. [Google Scholar] [CrossRef] [PubMed]
  88. McLellan, A.T.; Luborsky, L.; Woody, G.E.; O’Brien, C.P. An improved diagnostic evaluation instrument for substance abuse patients. The Addiction Severity Index. J. Nerv. Ment. Dis. 1980, 168, 26–33. [Google Scholar] [CrossRef] [PubMed]
  89. Bush, K.; Kivlahan, D.R.; McDonell, M.B.; Fihn, S.D.; Bradley, K.A. The AUDIT alcohol consumption questions (AUDIT-C): An effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch. Intern. Med. 1998, 158, 1789–1795. [Google Scholar] [CrossRef] [PubMed]
  90. Heron, K.E.; Smyth, J.M. Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. Br. J. Health Psychol. 2010, 15, 1–39. [Google Scholar] [CrossRef] [PubMed]
  91. Ben-Zeev, D.; Kaiser, S.M.; Brenner, C.J.; Begale, M.; Duffecy, J.; Mohr, D.C. Development and usability testing of FOCUS: A smartphone system for self-management of schizophrenia. Psychiatry Rehabil. J. 2013, 36, 289–296. [Google Scholar] [CrossRef] [PubMed]
  92. Zickuhr, K.; Smith, A. Digital Differences. 2012. Avilable online: http://www.pewinternet.org/2012/04/13/digital-differences/ (accessed on 20 June 2015).

Share and Cite

MDPI and ACS Style

Gaudiano, B.A.; Moitra, E.; Ellenberg, S.; Armey, M.F. The Promises and Challenges of Ecological Momentary Assessment in Schizophrenia: Development of an Initial Experimental Protocol. Healthcare 2015, 3, 556-573. https://doi.org/10.3390/healthcare3030556

AMA Style

Gaudiano BA, Moitra E, Ellenberg S, Armey MF. The Promises and Challenges of Ecological Momentary Assessment in Schizophrenia: Development of an Initial Experimental Protocol. Healthcare. 2015; 3(3):556-573. https://doi.org/10.3390/healthcare3030556

Chicago/Turabian Style

Gaudiano, Brandon A., Ethan Moitra, Stacy Ellenberg, and Michael F. Armey. 2015. "The Promises and Challenges of Ecological Momentary Assessment in Schizophrenia: Development of an Initial Experimental Protocol" Healthcare 3, no. 3: 556-573. https://doi.org/10.3390/healthcare3030556

Article Metrics

Back to TopTop