Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder characterized by impairments in social interaction and restricted, repetitive patterns of behavior that is now estimated to affect one in 68 children [1
]. Co-occurring developmental, psychiatric, and medical conditions are present in more than 70% of individuals with ASD, and include intellectual disability, language disorders, attention-deficit hyperactivity disorder (ADHD), anxiety, depression, obsessive-compulsive disorder, epilepsy, sleep disorders, and gastrointestinal problems [2
]. Caring for children with ASD can be challenging and is estimated to cost $2.4 million over a lifespan [5
]. Costs can include the direct costs of caring for an individual and the loss of work and productivity for caregivers. Children with ASD generally receive care from multiple providers in different domains, including behavioral therapy, speech and language therapy, physical therapy, occupational therapy, primary care, and medical specialties. The majority of children with ASD use three or more of these service providers and take at least one medication [6
]. In addition, 88% of families report currently or recently treating their children with ASD with at least one complementary and alternative therapy [7
Given the complexity of caring for a child with ASD, many experts have advocated for the use of a “medical home” or “family-centered” coordinated care model [8
]. A medical home has been described as care that emphasizes “comprehensive services that are family centered, continuous, culturally sensitive, compassionate, and coordinated with other providers in the community” [11
]. Prior studies have shown that children with special health care needs who receive care in the medical home have a number of improved outcomes including greater satisfaction, access to care, communication, fewer unmet needs, and reduced costs and negative impacts [12
Unfortunately, most families of children with ASD do not receive coordinated care. The National Survey of Children found that 62% of children with ASD have unmet needs for care coordination [14
]. In a study reporting the results of a focus group of parents of children with ASD, most parents felt that their physicians were too busy to provide coordinated care and assumed the role of care coordination themselves [8
]. A separate study found that over half of families of children with ASD reported having a parent stop or reduce work because of their child’s needs, and over 25% spent greater than 10 h a week coordinating care [15
There is clearly a largely unmet need to improve care coordination for children with ASD. Interestingly, the aforementioned studies that describe care coordination have limited or no description of coordinating care with schools or teachers. One study found that parents perceived little or no coordination between physicians and schools, and pediatricians reported never having been contacted by schools to discuss a child with ASD [8
]. A recent review of studies examining academic performance of children with ASD found that these children often have specific areas of strength and weakness, suggesting that individualized assessment would allow for targeted educational programming [16
]. The same review highlighted the lack of studies using measures that involved teachers and their observations within the classroom. This is a potentially missed opportunity for many reasons. Children who spend 7–10 h per day at school will have a variety of social interactions that may vary markedly from home and clinic settings. Also, their teachers can observe and monitor their changes over time. Therefore, care decisions made by parents or medical providers without input from the school environment may lack the rich information about symptom changes that take place at school. We therefore sought to expand the current care model—which separates care between caregivers, clinicians, and teachers—to a “three-component” model that emphasizes the need to have fluid communication between parents/caregivers, clinicians and teachers—the “three components” that typically comprise care for a child with ASD.
The underlying purpose of this pilot study was to investigate the feasibility of monitoring outcome measures of children with ASD using validated questionnaires that were administered to parents and teachers online. In collaboration with the teachers, we additionally developed two novel measures, the Teacher Autism Progress Scale (TAPS) and the Academic Progress Rating Scales (APRS), which we hoped to implement and test for initial utility at a school for children with ASD. Part of this effort involved developing a web-based, user-friendly platform that the participants are able to use to record regular changes in the child’s behavior, social responsiveness and progress in both the school and home environment. This process also involved the development of a novel outcome measure based on a collaborative process with teachers and designed to measure autism-specific changes that are observed in an educational environment.
Additionally, we sought to understand the factors affecting parent and teacher participation and attrition, an important part of assessing feasibility of our platform and program. Previous studies have indicated several potential factors that may be involved in study attrition, such as gender, race, and the number of children in the household [17
]. To determine participation and response rates in our study and examine feasibility of an outcomes tracking database, such as the one assessed here, we analyzed several variables that may have contributed to missing data, including, but not limited to, these demographic factors.
2.1. Preparation and Collaboration
This study was carried out in accordance with the Declaration of Helsinki, and all of the procedures were approved by UCSF Committee on Human Research (#13–11086) prior to contacting any families, teachers, or students to participate in the study. Informed consent was obtained from all parents or guardians of subjects prior to their participation in the study. A partnership was formed with a local school specializing in the education of children with autism and other neurodevelopmental conditions affecting social interactions and behavior (Oak Hill School, San Anselmo, CA, USA, http://www.theoakhillschool.org/
). The school is certified by the California Department of Education as a non-public, non-sectarian special education school (NPS), which allows it to contract with public school districts to provide educational and clinical services to students through their individualized education programs (IEPs). As of May 2017, more than 90% of students are publicly funded to receive services at the school.
2.2. Study Population
Student population: Oak Hill serves a heterogeneous population of children, adolescents, and young adults, all of whom have autism spectrum disorder (ASD) or other neurologically-based disorders of relating and communicating. A large number of co-morbidities are present in the population, which include seizure and tic disorders, intellectual disability, ADHD, obsessive-compulsive and anxiety disorders, learning disorders, cerebral palsy, language disorders, and other medical conditions (i.e., gastrointestinal disorders, fibromyalgia, leukodystrophy, etc.). The school has separate classrooms for children in grades K through 12 and class placements are affected by a number of factors, including verbal ability, age, and a careful behavioral and academic assessment.
2.3. Eligibility Criteria and Enrollment Process
Students were recruited through an e-mail announcement to the entire parent body of the school and through parent information sessions given at the school by the study investigators in 2013 and 2014. Researchers were careful to emphasize, both verbally and in writing, that study participation was not required and declining to participate would not affect the child’s educational program. All of the students with ASD enrolled in the school were considered as eligible. Our goal was to enroll as many children as possible in the pilot phase. ASD was defined as being present if the child had a diagnosis from a medical professional or if the student was determined by school staff and the study psychiatrist to meet Diagnostic and Statistical Manual of Mental Disorders
, Fifth Edition (DSM–5
) criteria for ASD [18
]. School enrollment is currently 36 students ranging from six to 24 years of age. The only exclusion criterion was being unable or unwilling to complete study forms. Demographics of the enrolled study population, including measures of baseline severity, are described in the Results section and in Table 1
School-based interventions: Students receive special education instruction and customized on-site clinical programs which may include speech/language pathology, occupational therapy, and group and individual psychotherapy. A portion of students have received recent functional behavior analyses (FBAs), with school staff implementing positive behavior intervention programs. The school also offers arts-based therapies and adaptive arts instruction.
Case conferences: For each enrolled child, a case-conference session was scheduled by the school senior teacher coordinator with the study Principal Investigator, who is an expert in clinical evaluation and the care of children with ASD, and members of child’s team (special education teacher, psychotherapist, speech/language pathologist, occupational therapist, school administrator, research coordinator). These school consultation sessions included a review of the progress of the outcome measures in the web-based platform (parent and teacher) and verbal feedback from the teachers regarding the child’s behavior and social interactions. During each session, target behaviors/barriers/challenges were identified and the group discussed and developed an action plan to address these problem behaviors. Two case conferences were conducted each week except during holidays or other scheduling conflicts.
Development of web-based platform:
A user friendly, web-based platform was developed for this study to allow for on-line data entry by parents and teachers and some management functions for the study research team. The development and implementation of this online platform is detailed in a previous study of children with ASD attending integrative medicine clinics [19
]. The web-based platform was developed on 23 August 2013 and pilot tested during the 5-month period of September 2013 to January 2014. All of the outcome measures were loaded for system deployment on 16 May 2014, which was the beginning of formal recruitment and enrollment.
Development and selection of outcome measures:
A literature review was conducted to identify the most appropriate outcome measures for the study. The outcomes were selected based on their validity, reliability, relevance, and sensitivity to the degree of change expected from any intervention for children with ASD [20
]. The primary outcomes being measured were aberrant behavior, social responsiveness, and quality of life, which were assessed via the Aberrant Behavior Checklist (ABC), Social Responsiveness Scale (SRS), and Pediatric Quality of Life Scale (PEDS), respectively.
In addition, two internal scales, labeled the Teacher Autism Progress Scale (TAPS) and the Academic Progress Rating Scales (APRS), were developed by both the teaching staff and the research team. TAPS was developed to measure the weekly behavioral, social and functional progress in target areas of school-related child behavior and skills (Table S1 Supplementary
), whereas APRS was designed to measure other key academic and functional domains on a quarterly basis. The content for both TAPS and APRS was generated in a series of focus groups combining UCSF research staff and Oak Hill School teaching staff, held in the fall of 2013. In this article, the weekly measure of TAPS will be analyzed and assessed for initial validity, whereas the quarterly APRS assessment will be implemented and analyzed in future studies.
Detailed past medical history for each child (including birth and family history, medical problems, prior medications, CIM, and other treatments) was also obtained via a parent intake questionnaire at baseline.
Timeline: Subjects were enrolled continuously between May 2014 and June 2016. Parents completed baseline measures upon enrollment, including the ABC, SRS, PEDS, and a medical history and demographic intake form. Parents and teachers completed the ABC and SRS at quarterly time points for the duration of the study. Parents additionally completed PEDS at these time points, and teachers additionally completed the APRS at these time points. Teachers also completed the TAPS on a weekly basis for all of the students enrolled in the study.
2.5. Statistical Analyses
The analyses determined a priori were to: (1) measure the change in outcome measures over time; (2) assess correlations between the TAPS and outcome measures, ABC and SRS scores; and (3) examine associations between baseline variables and missing data (termed “missingness”). All of the statistical analyses were performed using STATA 14 Statistical Software Package.
Sensitivity analysis: Multiple imputation and last-observation-carried forward were used as sensitivity analyses for missing data in all longitudinal analyses. Linear models were tested for linearity, normality of residuals, constant variance/heteroscedasticity and influential points using component-plus-residual (CPR) plots, qnorm and Kernel density plots, residual vs. fitted plots and DFBETAs, respectively. Influential points were tested using DFBETA and were excluded from the analysis if deemed to have high leverage and discrepancy.
Longitudinal and correlation analyses. Change in outcome measures over time was assessed via a mixed effects regression model and locally weighted scatterplot smoothing (LOWESS) curve. The magnitude and direction of association between TAPS and the outcome measures, aberrant behavior, and social responsiveness, was measured using Pearson’s correlation. Influential points following sensitivity analysis were excluded in the correlation analysis. The results for change of average total outcome scores per unit time by respondent type were adjusted for sex of the subjects, caregiver, teacher and grade of subjects.
To examine variables that might affect missing data, an odds ratio was derived to measure the association between the baseline covariates and the binary outcome of “missingness” (0 = not missing, 1 = missing). Several baseline variables (i.e., summer quarter, neurological disorders, and hospital visits) were included in a multivariate mixed effects model. Forward-stepwise selection was performed to select each additional variable based on a significance level of p
< 0.2. For the baseline variables that had missing data, multiple imputation iterative chained equation (MICE) was used to replace the missing values [21
Analysis of respondent participation. Form completion was calculated as the percentage of forms that were completed over the total number of forms assigned per quarter for caregivers/parents (total forms = 5) and teachers (total forms = 3). Response rate was calculated as the percentage of participants who answered any survey over the total number of participants enrolled at baseline for each time period a survey was filled out.
The overarching goal of this study was to facilitate communication between caregivers, clinicians, and educators, in order to promote coordination of efficacious intervention programs for children with autism spectrum disorders. A coordinated care model such as ours has been advocated for in the literature [8
], but we are unaware of any study assessing such a model in schools. In this study, we demonstrate the feasibility of implementing and tracking outcomes of this kind of model in a school. We were able to visually observe a trend of an overall improvement over time in validated outcome measures, including aberrant behavior, social responsiveness, and pediatric quality of life (Figure 1
). We were also able to demonstrate a statistically significant improvement in the TAPS (Figure 1
D), which was developed to measure the progress of the participants in a school setting in areas that are directly meaningful to the teachers, staff and clinicians at the school. These results were further supported by the correlational analysis, which showed a strong and statistically significant correlation between improvement on the TAPS and on the ABC and SRS (Figure 2
). Prospectively, the observed results suggest that this novel internal instrument, TAPS, is a potentially reliable and sensitive measure for tracking the behavioral progress of children with autism at schools.
There were several limitations in our study that we hope to address with future research. Declining response rates are prevalent and unavoidable in longitudinal studies, and can increase the likelihood of nonresponse bias [17
]. This pilot study showed that surveys assigned during the summer and perception of high survey burden (i.e., length, number and frequency of surveys) were the primary contributors to study dropout. Therefore, for future studies, the response rates could be improved with the allocation of additional resources to study retention. One likely solution would be to implement more effective community-engaged research strategies, such as including parents and teachers of children with ASD from outside the study population in the initial design of the data collection methods to help determine a feasible assessment schedule. Other methods will involve more consistent and frequent engagement (i.e., reminders, encouragement, help sessions, etc.) of the participants throughout the study.
An additional limitation of our study is the lack of standardization analysis of the TAPS. While we found statistically significant improvements in the TAPS over time and found statistically significant correlations between TAPS scores and ABC and SRS scores, we felt that a full validity and reliability analysis of the TAPS would be better done with higher response rates and a larger population than those that we experienced in this study. Because of this, we only intend the TAPS’s correlations with the ABC and SRS to be used as a preliminary indicator of its potential utility in ASD research. We hope to perform stronger analysis on this novel measure in future research.
Other limitations in our study include a small sample size, a wide age range in the subjects, a lack of exclusion criteria, and no control group, which can hopefully be controlled for in future studies. Due to the small population at the school, it was determined that the presence of exclusion criteria and a control group would result in a sample size too small to conduct the desired research. It is also possible that the results reported here might have been due to external factors outside of school interventions, a possibility that our methods do not address. Additionally, the significant improvement in the TAPS could have resulted from teachers’ potentially increased focus on TAPS-related domains during case conferences, causing them to focus more on brainstorming those specific TAPS-related challenges in their students and resulting in more improved scores on the TAPS than on the other measures assessed. We hope to expand upon this pilot in future studies with a large enough sample to be able to address these methodological issues.
Despite the limitations, results collected from this pilot study demonstrate proof-of-concept for systematic monitoring of the progress of children with autism in both the school and home environment. Children from this study population showed trends of improvement in aberrant behavior, social responsiveness, and pediatric quality of life over the study period, highlighting the value and utility of monitoring these outcomes as a way of measuring the effectiveness of school-based interventions. For example, it would be possible to examine changes in the entire school population or on an individual child level after the introduction of a new curriculum, a new program (e.g., gardening or music), a change of classrooms, or a change in medical therapy for a given child. This is the next step in our “medical home” outcomes research. Furthermore, we provide initial evidence that a novel outcome measure (TAPS) developed by teachers is sensitive to changes in child functioning in a school environment. The TAPS was strongly correlated with both the ABC and SRS and the only measure to show statistically significant changes over time, suggesting that it may be more sensitive for detecting meaningful behavioral and social change in a school environment than the ABC or SRS. We also identified areas that will likely improve retention and completion rates, including reduced participant burden and activities to promote community engagement. In conclusion, the data collected from this pilot study will improve collaborative care and potentially provide clinicians, caregivers, and educators with the tools and insight to collectively implement the most efficacious treatment plans for each individual child.