Next Article in Journal
Compression Tests of ABS Specimens for UAV Components Produced via the FDM Technique
Previous Article in Journal
A Transportable Photovoltaic Power Generation System Utilizing a SiC Inverter and Spherical Si Solar Cells
Previous Article in Special Issue
Capturing the True Value of Assistive Technologies to Consumers in Routine Outcome Measurement
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Measurement of Assistive Technology Outcomes Associated with Computer-Based Writing Interventions for Children and Youth with Disabilities

by
Stephen E. Ryan
1,2,* and
Mary-Beth Sophianopoulos
2
1
Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON M4G 1R8, Canada
2
Department of Occupational Science and Occupational Therapy, University of Toronto, Toronto, ON M5G 1V7, Canada
*
Author to whom correspondence should be addressed.
Technologies 2017, 5(2), 19; https://doi.org/10.3390/technologies5020019
Submission received: 30 January 2017 / Revised: 22 March 2017 / Accepted: 19 April 2017 / Published: 22 April 2017

Abstract

:
Young people who have chronic disabilities use computer technologies and receive rehabilitation services to overcome functional limitations associated with writing activities. However, the functional impact of these specialized assistive technologies on the everyday lives of children is not clearly understood; in part due to the lack of targeted outcome measures. This article describes the development and evaluation of the Family Impact of Assistive Technology Scale for Writing Interventions (FIATS-WI). The FIATS-WI is a multi-dimensional, parent-report questionnaire designed to measure child functioning and outcomes associated with computer-based writing interventions for children and youth aged 5–18 years. Participants included parents of children with writing-related disabilities who completed the questionnaire at home during one of two study phases. In the first phase, 121 eligible parents, out of 364 invited, completed a single administration of the questionnaire. In the second phase, 28 out of 33 eligible parents completed the FIATS-WI twice to assess its stability. Item and subscale correlations informed an item reduction plan, and Cronbach’s alpha and intraclass correlation coefficients provided acceptable estimates for internal consistency and test–retest reliability, respectively. Correlations between FIATS-WI scores and scores from a standardized home participation measure tested its convergent validity. The study provides emerging evidence for the FIATS-WI as a sound measure of computer-based writing technology outcomes for children and youth with disabilities.

1. Introduction

1.1. Background

An estimated 65,000 young Canadians between the ages of 5 and 19 years have mobility impairments due to childhood-onset disabilities such as cerebral palsy, acquired brain injuries, and muscular dystrophy [1]. More than half of these children and youth (hereafter ‘children’) also have fine motor impairments and communication limitations that further curb their ability to participate fully in everyday activities [1]. Children with these disabilities see allied health professionals to acquire assistive technology (AT) devices and learn strategies to use them to optimize their functional performance in home, school, and community settings. Occupational therapists and speech language pathologists assess the functional and contextual needs of children to support the selection of appropriate AT devices, software, and training programs to overcome specific barriers associated with writing access and the planning, organization, and translation of ideas into written text [2,3]. AT devices and services can have a positive effect on a child’s ability to complete written work and influence the child’s ability to access the education curriculum, share ideas and opinions, and engage in enjoyable leisure activities [2,4].
Given the ubiquity of personal computers, tablets, and smart phones, compatible AT devices and software can provide the necessary adaptation required for individuals to achieve their writing goals [5]. AT devices for writing can range in complexity from oversized keyboards to eye tracking technologies that translate eye movement into letter selection on an on-screen keyboard. Children who have difficulty mastering spelling, punctuation, and the organization of ideas may also use AT software, such as word prediction programs for word processing and screen readers for proofing written text. The remedial technologies and methods adopted vary depending upon the child’s health condition and a host of contextual factors including the child’s attitude toward writing activities [6,7] and the capacity of family members to implement and support the writing intervention at home [8]. In this article, ‘computer-based writing interventions’ means AT devices, software, training programs, and associated services intended to support a child’s access to and use of computer-based tools for written language expression and/or remediation.
A small body of research provides empirical support for the efficacy of different computer-based writing interventions on child behaviour [9], writing performance [10,11,12,13], spelling accuracy [14,15,16,17], typing rate [17,18], literacy [2,19,20], computer usage [21], and device satisfaction [22,23,24]. Yet, less is known about the functional impact and sustainability of these interventions as well as the influence of factors that support improvements in writing performance [3,9,25]. The measurement of these outcomes is important as Canadian children with communication-related disabilities reportedly have the highest proportion of unmet needs for AT devices compared to the device needs of children with other disabilities [26]. Neglecting technology needs for written communication can impact literacy skill development [27], limit peer interactions [27], and lead to social isolation [28,29]. Understanding more about the functioning of children with disabilities and factors that support/hinder writing activities will help to inform the planning, development, and implementation of supportive writing technologies and approaches.
The assessment of writing needs and measurement of related outcomes for children with disabilities typically begins at the initial meeting of the AT practitioner, child, and parents/caregivers. As Scherer and others posit in modelling factors associated with the selection of AT devices for children [30,31], the writing needs assessment commences with the collection of information about the child and family (resources, knowledge, expectations, personal preferences/priorities) and the environment (cultural/financial priorities, related legislation/policy, and attitudes of child and key others). Armed with this information, those present make shared decisions about a writing intervention that meets both objective and subjective needs. The objective need is determined by the practitioner’s assessment of the child’s functional abilities. Whereas, the subjective need takes into account the child’s/family’s predisposition (perceived need) for a particular writing technology. The assessment of subjective/objective needs informs goal setting pre-intervention and goal achievement post-intervention.
Usually, a practitioner introduces the computer-based writing technology and provides necessary training for both the user and others who support the user in different settings (home, school). When training is complete, the child uses the device and applies learned strategies to support writing needs. Short-term evaluation of writing outcomes typically takes place weeks after AT provision using individualized measures such as Goal Attainment Scaling [32] or the Canadian Occupational Performance Measure [33]. This approach to outcome measurement helps to understand the child’s and family’s perspectives, progress toward specific goal achievement, and changes in activity performance. However, such targeted performance measurement does not capture the broader functional impact of writing interventions across multiple dimensions and users during training and beyond the end of clinical involvement. Consequently, factors that influence the continued use of the AT device, and intervention adjustments that ensure continued benefit for the child, may be missed [34]. In addition, goal-based approaches to outcome measurement may not reflect how a writing intervention may change the experiences and attitudes of other family members who may influence its use. It is necessary to use outcome measures that can shed light on how computer-based writing interventions affect the everyday lives of the child and other caregivers along the continuum of use.
Assessing writing needs, preferences, and outcomes by self-report is ideal, but challenges may exist due to age or cognitive limitations [35]. While discrepancies between self- and proxy-reports in other child health domains exist [36,37,38], parents’ perspectives are crucial as they routinely provide important counsel and may influence/be influenced by both the nature and outcomes of AT interventions [39,40,41]. Parents are uniquely positioned to observe, generalize, and report their child’s functional performance in everyday settings, and perceive family functioning and other contextual factors that may be linked to child-related outcomes.
Fuhrer and colleagues propose a useful conceptual framework to underpin modelling processes for the measurement of outcomes for specific assistive technologies such as writing technologies for children [42]. Informed by the World Health Organization’s biopsychosocial view of functioning, disability and health [43], and applied within a context of child- and family-centred care in pediatric rehabilitation [44], the AT outcomes framework considers functional impact as outcomes of the dynamic interactions among characteristics of the device, the child who uses the technology, and other moderating co-factors in the environment.
Short-term and long-term outcomes may include device effectiveness, efficiency, and satisfaction, as well as psychological functioning and subjective well-being of both the child who uses the writing device and family/other community members who support its implementation in different environments [42]. The identification and measurement of these and other factors that affect functional outcomes may predict AT device use/abandonment following the provision of the device. However, modelling AT outcomes and identifying predictive factors for successful writing interventions are predicated on the availability and application of rigorously-developed outcome measures that detect relevant functional and contextual effects over time.

1.2. Development of the Family Impact of Asssistive Technology Scale for Writing Interventions (FIATS-WI)

Members of our research team developed the FIATS-WI to address this measurement need. The FIATS-WI is a parent-report questionnaire designed to measure child functioning, environmental factors, and personal factors in domains that may be influenced by the introduction of computer-based writing interventions for children aged 5 to 18 years. It was conceived as a standardized measure that could be used by researchers and clinicians to detect pre- and post-intervention levels and change (outcomes) globally and within each of its dimensions.
Candidate dimensions for the FIATS-WI came from a systematic review of peer-reviewed studies that reported on AT effects on child, caregiver, and family functioning [45] and a review of other questionnaires that measured health outcomes in children with disabilities [46]. Team members reviewed AT outcomes and considered other content areas based on clinical experience. The team agreed upon the inclusion of eight writing-related dimensions and an additional eight domains from the original Family Impact of Assistive Technology Scale (FIATS) [47]. The original FIATS measures child functioning and contextual factors associated with other types of AT interventions for children with mobility-related limitations, so it was postulated that its dimensions may also be relevant for writing technologies.
Nine content experts assessed the relevance of all 16 dimensions as content areas that could interact with child functioning following introduction of a computer-based writing intervention. Experts included parents of children with writing technology needs, and AT professionals with extensive experience in writing technology services and research for children with disabilities. This expert screening process resulted in the selection of 12 dimensions for the new FIATS-WI (Table 1). Of these, four dimensions were shared with the original FIATS. The dimensions chosen generally related to child functioning (doing activities, independence, quality of writing, social adaptability, writing activities) environmental factors (caregiver relief, establishing routines, exertion, parent well-being, supervision) and personal factors (attitudes toward writing, contentment).
An interdisciplinary team-including a senior AT practitioner, senior child health researcher, and occupational therapy trainee-developed items for each of the new dimensions [48]. Five parents of children with writing needs individually reviewed and flagged items that were unclear or not relevant to understanding factors associated with writing outcomes. The same parents then met as a group to reword, accept, or eliminate items identified as unclear/irrelevant.
The revised FIATS-WI had 88 items, included a 7-point Likert rating scale so parents could indicate their degree of agreement with item statements (strongly disagree to strongly agree), and used a scoring strategy adopted from the original FIATS [49]. This earlier pilot work provided an essential prefatory grounding for ongoing development and evaluation of the FIATS-WI as a multidimensional, parent-report questionnaire that measures AT outcomes linked to computer-based writing interventions for children with disabilities aged 5 to 18 years.
The aim of the present study was to advance the development, estimate the reliability, and obtain support for the validity of the FIATS-WI. The research study had two phases to meet this aim. Both phases were conducted in accordance with Canada’s Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans and approved by the research ethics board at the authors’ academic teaching hospital (REB 09-030/14-435). All participants consented to take part in the research study.

2. Materials and Methods

2.1. Phase 1: Item Reduction and Internal Reliability

The objectives of phase 1 were to: (a) improve the scoring precision and reduce the length of the 88-item FIATS-WI; and, (b) estimate the internal consistencies of the total scale and its subscales.
The research design was a hospital-based, mail-out survey to parents whose children with disabilities were between the ages of 5 and 18 years and had received services within the previous three years through the writing technologies service at the authors’ institution. The institution’s electronic health records department identified an initial sampling frame of 400 parents.
The survey strategy used for this phase was adapted from Dillman’s Tailored Design Method [50]. Each parent received a notification letter from the manager of the writing technologies service. The letter introduced the purpose, importance, and voluntary nature of the study, and advised that the survey package would follow by mail a few days later. The survey package included a cover letter, a demographics form, the 88-item FIATS-WI questionnaire, a self-addressed stamped return envelope, and a writing resource compact disc. The mailing included a hyperlink to allow secure access to an online version for parents who chose to do the survey electronically rather than complete it by paper and pencil. The compact disc created by the research team included writing resources and activities for children and hyperlinks to other online resources. The free disc served as a form of social exchange to encourage participation [50].
Parents were asked to complete the questionnaire within two weeks after receiving the survey package. The basic demographic information requested included the relationship of respondent to child, age/diagnosis of the child, and general type of writing technology used (if any). No linking information was used to assure anonymity of participants. All families received a final reminder postcard 3 weeks after the initial survey mailing.
The initial survey went to 400 households, but 36 (9%) were returned as undeliverable. Of the 364 questionnaires remaining, the research team received 139 responses (38% response rate). Of these, 18 questionnaires were removed from the analysis for three reasons: the child was older than 18 years of age (n = 15); the child’s age was not reported (n = 2); and one questionnaire had fewer than 80% of the ratings assigned. Data were extracted from the remaining 121 questionnaires. Participants returned their questionnaires by mail (n = 106 (88%)) or on-line (n = 15 (12%)).
A Classical Test Theory approach [34,51] informed the development of an analytic plan. The general plan was to use data collected to identify and eliminate items that did not correlate well with the other items of the subscale to which they were assigned. Also considered for elimination were dimensions that highly aligned with the construct measured by another dimension. Specifically, decisions about item/dimension elimination generally followed these criteria:
  • Discard items where less than 20% or 25 respondents provide no rating.
  • Discard items where over 80% of respondents selected the same rating for a particular item.
  • Identify items with low item-total subscale correlations (Pearson’s |r| < 0.2). Correlate these items with ‘other’ subscales. If the item–other subscale correlation was greater than 0.2, conditionally, reassign the item to the ‘other’ subscale if it related conceptually to that subscale. Otherwise, eliminate the item.
  • Calculate item–total other subscale correlations for the remaining items to confirm that they were assigned to the correct subscales. Reassign an item if it had a higher item–subscale total correlation with another subscale, and was conceptually related to it.
  • Identify within-scale items with high inter-item correlations (r > 0.9). Eliminate the item that has the lower item-subscale correlation.
  • Calculate inter-dimensional correlations to flag possible redundancy in measured constructs. Review constructs that have high correlations (r > 0.9) to judge whether one dimension should be eliminated from the FIATS-WI.
Cronbach’s alpha provided estimates of internal consistency for the total FIATS-WI scale and each subscale. Homogeneous scales should have an alpha between 0.7 and 0.9 [34]. If the statistic did not meet this criterion, then items with the lowest item-subscale correlations were considered for elimination until alpha was within the range of acceptable values. All analyses were conducted using Statistical Program for Social Sciences (SPSS) Version 16.0. (SPSS Inc. Released 2007. SPSS for Windows, Version 16.0. Chicago, SPSS Inc.)
Measurement authorities suggest that 5–10 participants per dimension be used for this type of analysis [34], so the target sample size was 120 completed questionnaires. Forms not meeting eligibility criteria or returned with less than 80% of the FIATS-WI ratings were considered incomplete and not included in the analysis.

2.2. Phase 2: Test–Retest Reliability and Convergent Construct Validity

The objectives of phase 2 were to: (a) estimate the test–retest reliabilities of the FIATS-WI scale and subscales; and (b) obtain support for the convergent construct validity of the FIATS-WI.
The study used a hospital-based, repeated measures survey design. Parents who were eligible to participate had a child reported to: (a) be 5 to 18 years of age inclusive; (b) have difficulty writing due to their disability; (c) be able to compose ideas in writing; (d) have writing needs at home (as a condition of being seen for services); and (e) have received writing technology services within the previous 3 years. Further, parents needed to understand written English to complete the forms provided.
A random sample of 100 parents of children who met the eligibility criteria received a letter from the clinical services manager to introduce the study and inform them that a researcher may contact them to determine their interest in study participation. Two weeks after the mailing, a researcher called families on the contact list in random order. The first 30 interested families received a package containing a study information letter, the FIATS-WI questionnaire, and a pre-paid return envelope. Parents received a phone call one week later to confirm their participation. Consent to participate in the study was implied upon return of completed questionnaires. Upon receipt of the first questionnaire, a second FIATS-WI questionnaire and the Participation and Environment Measure for Children and Youth (PEM-CY) measure were mailed. Families who participated received a $20 gift card as a token of appreciation.
Participants completed the 74-item FIATS-WI twice to estimate its test–retest reliability. A two-to-three week interval between questionnaire administrations was chosen as it was assumed participants would not recall their initial ratings and expected there would be no substantive changes in the lives of children and their families. Basic demographic data and the number of days between measurement administrations were also collected.
Participation is defined by the World Health Organization’s International Classification of Functioning, Disability and Health: Children and Youth Version (ICF-CY) as ‘involvement in life situations’ [43]. Within the ICF-CY framework, participation may be influenced by its dynamic interactions with body function and structures, activities, and contextual factors including both environmental factors (e.g., family functioning) and personal factors (e.g., child behaviour). The FIATS-WI aligns with the ICF-CY framework by incorporating interrelated dimensions that measure participation and other aspects of child functioning (doing activities, independence, quality of writing, social adaptability, writing activities), environmental factors (caregiver relief, exertion, supervision), and personal factors (attitudes toward writing, contentment). Since these dimensions may interact with levels of child involvement in everyday social settings, the research team hypothesized that child and family functioning as measured globally by the FIATS-WI would be associated with both the frequency and intensity of the child’s home participation as measured by the PEM-CY.
The PEM-CY is a parent-report questionnaire that probes child participation and environmental supportiveness in activities of children aged 5–17 years in home, school, and community settings [52]. The PEM-CY has adequate psychometric properties, with moderate to good internal consistencies (Cronbach’s alpha > 0.59) and test–retest reliabilities over a 1–4 week period for home participation frequency (intra-class correlation coefficients (ICC) > 0.58) and involvement (ICC > 0.76) [52]. The home participation section of the PEM-CY includes 10 generic types of home activities such as indoor play, household chores, personal care management, and socializing using technology, and can be administered as a setting-specific measure of participation. Further, the PEM-CY scale is most relevant for parents as they are well positioned to observe their child’s participation at home. Higher scores suggest higher frequency (i.e., how often dimension) and intensity of involvement (i.e., how involved dimension) in select home activities.
Paired data from 30 participants could detect a correlation as large as 0.50 assuming a power of 80% and two-tailed alpha level of .05. Collected data were entered from all questionnaires into IBM SPSS version 19.0 (IBM Corp. Released 2010. IBM SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp.) for analysis. Test–retest reliability estimates were ICCs using a mixed model for absolute agreement. A lower 95% confidence limit of ICC = 0.70 was considered to be an indicator of good test–retest reliability for the total FIATS-WI score and each of its subscales.
Support for convergent construct validity was determined by calculating correlations among the FIATS-WI and how often and how involved in specific home activities as measured by the PEM-CY. These analyses did not adjust alpha for multiple testing as they were intended to flag possible associations among the constructs measured. Pearson’s correlation coefficient (r) and Spearman’s rank correlation coefficient (rs) provided the statistics of association for data demonstrating normal and non-normal distributions, respectively. Significant positive correlations within the low to moderate range of 0.3 < r/rs < 0.7 were viewed as support for convergent construct validity.

3. Results

3.1. Phase 1: Item Reduction and Internal Reliability

Most survey respondents were mothers, and the rest were a mix of fathers and other primary or unspecified caregivers (Table 2). The mean age of the child who received writing technology services was 13.0 years (SD = 3.2) and nearly two-thirds of the children were boys. The four most common diagnoses were cerebral palsy, developmental delay, acquired brain injury, and autism spectrum disorders. Most respondents reported that their child used some form of standard computer hardware/software, and special writing software was the most common type of adaptive writing technology used.
No FIATS-WI items were eliminated for having either too few ratings or too many of the same rating as required in the analytic plan. Nine items were eliminated from the scale as they had low subscale correlations and either did not relate well or conceptually to the items assigned to other subscales. A further three items were reassigned to another subscale as they correlated more closely and were conceptually related to other subscales. This elimination/reassignment process removed all five items from the establishing routines subscale thereby removing this dimension from the measure.
The inter-dimensional correlation was very high (r = 0.9) between two similar constructs—caregiver relief and parent well-being. The parent well-being dimension was subsequently removed since caregiver relief was shared with the original FIATS and had additional empirical evidence of acceptable reliability and validity in earlier research [47,49].
Alphas for the total scale and 8/10 subscales met recommended levels of homogeneity. The two remaining dimensions (contentment, doing activities) had alphas marginally below the recommended threshold (Table 3). Eliminating one or more items for these two dimensions did not raise the alphas to the lower threshold of 0.70.

3.2. Phase 2: Test–Retest Reliability and Convergent Construct Validity

Initially 30 participants were recruited, but this increased to 33 participants to replace parents who for unknown reasons did not return a second FIATS-WI questionnaire. By the study’s end, 28 parents completed both administrations of the FIATS-WI and the PEM-CY. The median ages of children who received writing services was 15.5 years (min = 10 years; max = 18 years) (n = 32; 1 missing). The mean interval between the two administrations was 30 days (min = 14 days; max = 90 days; SD = 15.8 days). Table 4 provides the main descriptive statistics for the two administrations of the FIATS-WI, the test–retest reliability point estimates, and 95% CIs for these statistics. Test–retest reliability statistics for all scales exceeded ICC = 0.87 and the lower confidence limits for the scales exceeded the recommended threshold of ICC = 0.70.
The PEM-CY exhibited a non-normal scoring distribution, so Spearman’s rank correlations were selected for the correlational analyses. The total FIATS-WI and how often and how involved dimensions of the PEM-CY both showed positive, moderate correlations at the p < 0.05 level (Table 5). Only the supervision subscale showed a positive, moderate correlation with the how often dimension of the PEM-CY. All other subscale correlations were not significant.

4. Discussion

4.1. Phase 1: Item Reduction and Internal Reliability

The 10 subscale scores for each participant were calculated by the mean rating of the items assigned to the dimension so the possible score for each was between 1 and 7. Whereas, the FIATS-WI total score was calculated by the sum of the subscale means yielding a possible score between 10 and 70. The total FIATS-WI score in this phase was 43.4 (SD = 8.1) or 55.7% of the total possible score with no indication of floor or ceiling effects.
The survey response rate was below that expected using Dillman’s survey method. However, the research team chose to be less assertive in the invitations to participate, by including only one reminder notice. It may be that those who did not respond were systematically different than those who did respond. Further, the sampling pool was situated in a very diverse, multicultural urban area with a rich mix of primary languages other than English. Consequently, requiring English literacy may have further limited the representativeness of the study sample.
A similar item reduction strategy used for the original FIATS resulted in the development of a related AT outcomes measure with acceptable internal consistencies [49]. In the present study, the precision of the FIATS-WI improved and the length of the measure reduced by 16% (14/88 original items). While content experts in previous work indicated that establishing family routines and parent well-being were constructs that may be impacted by the introduction of writing technologies, the research team elected to improve measurement precision and reasoned that doing so would reduce respondent burden without severely compromising scale validity
The total FIATS-WI and the subscales showed at least moderate levels of homogeneity as indicated by their alpha statistics. While two subscales were somewhat lower than the recommended threshold, the research team elected to retain these dimensions to avoid further erosion of the content validity of the measure. In sum, the reduced FIATS-WI had 74 items assigned to one of 10 dimensions.

4.2. Phase 2: Test–Retest Reliability and Convergent Construct Validity

The test–retest reliability levels reported here are similar to those reported for original FIATS [49] and the FIATS-AAC—a companion parent-report measure of child and family functioning for face-to-face rather than written communication interventions [53]. This is the third study to estimate acceptable test–retest reliabilities for the four subscales shared with the original FIATS and FIATS-AAC measures [49,53].
An analysis of correlations between the FIATS-WI total scale and measures of home participation frequency and intensity provides support for convergent construct validity. Higher scores on the total FIATS-WI tended to be associated with a greater frequency and degree of involvement in home activities as scored by the PEM-CY. However, only 17% of the variance (R2) is explained by each indicator of home participation. This is consistent with the research team’s belief that the FIATS-WI measures participation as well as other aspects of child and family functioning not reflected in the PEM-CY. Moderate associations were expected for individual dimensions, but the sample size was not large enough in the present study to support testing these hypotheses. Additional studies designed to test the strength of subscale associations with levels of participation and other aspects of child functioning, contextual factors, and health-related quality of life will provide additional empirical support for the validity of the FIATS-WI.
Phase 2 also had study limitations. Although parents of children who were as young as 5 were eligible, the random sampling strategy resulted in the recruitment of parents whose children were at least 5 years older and had a mean age of 16. While the inclusion of parents of younger children would likely result in the same conclusion regarding the adequacy of the test–retest reliability, researchers and practitioners should be cautious when assuming the stability of the FIATS-WI for young school-aged children. It is recommended that future reliability studies use a stratified sampling strategy to include parents of children who represent the full range of ages. Further, although participants were asked to complete their second questionnaires 2–3 weeks after completing the first one, five parents dropped out after completing the first questionnaire and more than half of the families exceeded the 3-week limit. Of concern was that the lives of children and their families may have changed during this longer period. Regardless, the test–retest reliabilities for the total FIATS-WI and all 10 subscales were acceptable.

5. Conclusions

The present study provides empirical support for the FIATS-WI as an emerging measure of AT outcomes in children with disabilities who need computer-based writing interventions. The FIATS-WI and its constituent subscales have acceptable levels of reliability, and the total scale has initial evidence to support its convergent construct validity. Since no gold standard measure exists for measuring AT outcomes for writing interventions, further empirical study of the construct validity of the FIATS-WI is indicated. Future investigations should include demonstration of the ability of the measure to detect clinically important change following the provision of writing technologies and along the continuum of device use. Future studies should also explore the utility of the FIATS-WI including the amount and acceptability of time required to complete the measure by parents. Further, the development of other measurement scales that capture the perspectives of young people who use computer-based writing technologies and those who read their narratives (peers, educators) would help to provide a balanced view of the effects of computer-based writing interventions.
Child- and family-centred practice is core to the delivery of AT devices and services, including the computer-based writing interventions considered in this article. Yet, AT outcome measurement that focuses solely on activity performance and the attainment of important goals may be limiting. Individualized approaches to outcome measurement do not provide a broad view of the impact of writing interventions and factors that may influence their short- and long-term use. The FIATS-WI shows promise as a complementary strategy for AT practitioners and researchers by offering a measure of key functional and contextual factors that may influence and be influenced by the selection, training, and ongoing use of writing technologies.
How reliant a child with a disability is on writing technologies may inform shared decision making about the relative importance and need to consider outcomes across all domains of the FIATS-WI. Future research may also be directed to study the flexible use of dimensions important for individual children and their families. For example, simple writing technologies such as the use of enlarged keyboards may not require consideration of constructs related to need for parent respite (caregiver relief). On the other hand, an eye tracking writing system that enables a child to connect independently with peers via social media may have a profound impact on both child development and family functioning. In such cases, monitoring change across all dimensions of the FIATS-WI may be important to understanding AT outcomes.
The measurement of AT outcomes for children with disabilities is complex as they are multidimensional and influenced by both known and unknown moderating factors. The availability of sound tools designed to measure key functional and contextual factors will support the development and evaluation of new technologies intended to address unmet needs and, ultimately, improve the everyday lives of children with disabilities and their families.

Acknowledgments

The authors gratefully acknowledge funding support from the Canadian Institutes of Health Research. The authors also wish to thank the parents who participated and recognize the study contributions of the following research colleagues: Allyson Schonberger, Rebecca Yeoman, Anushi Sivarajah, Lisa Bennett, Rachel Vizcarra, Cynthia Tam, and Anna Oh.

Author Contributions

Stephen E. Ryan developed the protocol for the present study; conducted the data analyses; drafted the initial manuscript; and read and approved the final version of the manuscript. Mary-Beth Sophianopoulos provided clinical context; proposed the applied approach for AT outcome measurement; contributed to protocol development and analytic interpretation; critically reviewed the draft manuscript; and read and approved the final version of the article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Disability in Canada: A 2006 Profile; Cat. No.: HS64-11/2010; Human Resources and Skills Development Canada, Canadian Government Publishing Services: Gatineau, QC, Canada, 2011.
  2. Browning, N. Literacy of children with physical disabilities: A literature review. Can. J. Occup. Ther. 2002, 69, 176–182. [Google Scholar] [CrossRef] [PubMed]
  3. Koppenhaver, D.; Williams, A. A conceptual review of writing research in augmentative and alternative communication. Augment. Altern. Commun. 2010, 26, 158–176. [Google Scholar] [CrossRef] [PubMed]
  4. Machalicek, W.; Sanford, A.; Lang, R.; Rispoli, M.; Molfenter, N.; Mbeseha, M.K. Literacy interventions for students with physical and developmental disabilities who use aided AAC devices: A systematic review. J. Dev. Phys. Disabil. 2010, 3, 219–240. [Google Scholar] [CrossRef]
  5. Freeman, A.R.; MacKinnon, J.R.; Miller, L.T. Assistive technology and handwriting problems: What do occupational therapists recommend? Can. J. Occup. Ther. 2004, 71, 150–160. [Google Scholar] [CrossRef] [PubMed]
  6. Carpe, A.; Harder, K.; Tam, C.; Reid, D. Perceptions of writing and communication aid use among children with a physical disability. Assist. Technol. 2010, 22, 87–98. [Google Scholar] [CrossRef] [PubMed]
  7. Peterson-Karlan, G.; Hourcade, J.J.; Parette, P.A. Review of assistive technology and writing skills for students with physical and educational disabilities. Phys. Disabil. 2008, 26, 13–32. [Google Scholar]
  8. Priest, N.; May, E. Laptop computers and children with disabilities: Factors influencing success. Aust. Occup. Ther. J. 2001, 48, 11–23. [Google Scholar] [CrossRef]
  9. Batorowicz, B.; Missiuna, C.A.; Pollock, N.A. Technology supporting written productivity in children with learning disabilities: A critical review. Can. J. Occup. Ther. 2012, 79, 211–224. [Google Scholar] [CrossRef] [PubMed]
  10. Cullen, J.; Richards, S.B.; Frank, C.L. Using software to enhance the writing skills of students with special needs. J. Spec. Educ. Technol. 2008, 23, 33–44. [Google Scholar]
  11. Mirenda, P.; Turoldo, K.; McAvoy, C. The impact of word prediction software on the written output of students with physical disabilities. J. Spec. Educ. Technol. 2006, 21, 5–12. [Google Scholar] [CrossRef]
  12. Simmons, K.D.; Carpenter, L.B. Spelling and assistive technology: Helping students with disabilities be successful writers. Phys. Disabil. 2010, 29, 5–19. [Google Scholar]
  13. Tam, C.; Archer, J.; Mays, J.; Skidmore, G. Measuring the outcomes of word cueing technology. Can. J. Occup. Ther. 2005, 72, 301–308. [Google Scholar] [CrossRef] [PubMed]
  14. Garrett, J.T.; Wolff, K.H.; Fowler, L.P.; Alberto, P.A.; Fredrick, L.D.; O’Rourke, C.M. Using speech recognition software to increase writing fluency for individuals with physical disabilities. J. Spec. Educ. Technol. 2011, 26, 25–41. [Google Scholar] [CrossRef]
  15. Lewis, R.B.; Graves, A.W.; Ashton, T.M.; Kieley, C.L. Word processing tools for students with learning disabilities: A comparison of strategies to increase text entry speed. Learn. Disabil. Res. Pract. 1998, 13, 95–108. [Google Scholar]
  16. Mezei, P.; Heller, K.W. Evaluating word prediction software for students with physical disabilities. Phys. Disabil. 2005, 23, 93–113. [Google Scholar]
  17. Mezei, P.J.; Heller, K.W. Effects of word prediction on writing fluency for students with physical disabilities. Phys. Disabil. 2012, 31, 3–26. [Google Scholar]
  18. Handley-More, D.; Deitz, J.; Billingsley, F.F.; Coggins, T.E. Facilitating written work using computer word processing and word prediction. Am. J. Occup. Ther. 2003, 57, 139–151. [Google Scholar] [CrossRef] [PubMed]
  19. Light, J.; McNaughton, D.; Weyer, M.; Karg, L. Evidence-Based literacy instruction for individuals who require augmentative and alternative communication: A case study of a student with multiple disabilities. Semin. Speech Lang. 2008, 29, 120–132. [Google Scholar] [CrossRef] [PubMed]
  20. Millar, D.C.; Light, J.C.; McNaughton, D.B. The effect of direct instruction and writer’s workshop on the early writing skills of children who use augmentative and alternative communication. Augment. Altern. Commun. 2004, 20, 164–178. [Google Scholar] [CrossRef]
  21. Borgestig, M.; Falkmer, T.; Hemmingsson, H. Improving computer usage for students with physical disabilities through a collaborative approach: A pilot study. Scand. J. Occup. Ther. 2013, 20, 463–470. [Google Scholar] [CrossRef] [PubMed]
  22. Breivik, I.; Hemmingsson, H. Experiences of handwriting and using a computerized ATD in school: Adolescents with Asperger’s syndrome. Scand. J. Occup. Ther. 2013, 20, 349–356. [Google Scholar] [CrossRef] [PubMed]
  23. Tam, C.; Wells, D. Evaluating the benefits of displaying word prediction lists on a personal digital assistant at the keyboard level. Assist. Technol. 2009, 21, 105–114. [Google Scholar] [CrossRef] [PubMed]
  24. Lancioni, G.E.; Singh, N.N.; O’Reilly, M.F.; Sigafoos, J.; Green, V.; Chiapparino, C.; Stasolla, F.; Doretta, O. A voice-detecting sensor and a scanning keyboard emulator to support word writing by two boys with extensive motor disabilities. Res. Dev. Disabil. 2008, 30, 203–209. [Google Scholar] [CrossRef] [PubMed]
  25. Burne, B.; Knafelc, V.; Melonis, M.; Heyn, P.C. The use and application of assistive technology to promote literacy in early childhood: A systematic review. Disabil. Rehabil. Assist. Technol. 2011, 6, 207–213. [Google Scholar] [CrossRef] [PubMed]
  26. Participation and Activity Limitation Survey 2006: A Profile of Assistive Technology for People with Disabilities; Cat. No.: 89-628-X no. 005; Statistics Canada, Canadian Government Publishing Services: Ottawa, ON, Canada, 2008.
  27. Light, J.; McNaughton, D. Supporting the communication, language, and literacy development of children with complex communication needs: State of the science and future research priorities. Assist. Technol. 2012, 24, 34–44. [Google Scholar] [CrossRef]
  28. Lindsay, S.; Tsybina, I. Predictors of unmet needs for communication and mobility assistive devices among youth with a disability: The role of socio-cultural factors. Disabil. Rehabil. Assist. Technol. 2011, 6, 10–21. [Google Scholar] [CrossRef] [PubMed]
  29. Dusing, S.; Skinner, A.; Mayer, M. Unmet need for therapy services, assistive devices, and related services: Data from the National Survey of Children with Special Health Care Needs. Ambul. Pediatr. 2004, 4, 448–454. [Google Scholar] [CrossRef] [PubMed]
  30. Scherer, M.J.; Jutai, J.W.; Fuhrer, M.J.; Demers, L.; Deruyter, F. A framework for modelling the selection of assistive technology devices (ATDs). Disabil. Rehabil. 2007, 2, 1–8. [Google Scholar] [CrossRef]
  31. Ryan, S.E.; Klejman, S.; Gibson, B.E. Measurement of the product attitudes of youth during the selection of assistive technology devices. Disabil. Rehabil. Assist. Technol. 2013, 8, 21–29. [Google Scholar] [CrossRef] [PubMed]
  32. Ottenbacher, K.J.; Cusick, A. Goal attainment scaling as a method of clinical service evaluation. Am. J. Occup. Ther. 1990, 44, 519–525. [Google Scholar] [CrossRef] [PubMed]
  33. Law, M.; Baptiste, S.; McColl, M.; Opzoomer, A.; Polatajko, H.; Pollock, N. The Canadian Occupational Performance Measure: An outcome measure for occupational therapy. Can. J. Occup. Ther. 1990, 57, 82–87. [Google Scholar] [CrossRef] [PubMed]
  34. Streiner, D.; Norman, G. Health Measurement Scales: A Practical Guide to Their Development and Use, 3rd ed.; Oxford University Press: New York, NY, USA, 2003. [Google Scholar]
  35. Gibbons, E.J.; Morris, C.; Fitzpatrick, R. Child and Parent Reported Outcome Measures: A Scoping Report Focusing on Feasibility for Routine Use in the NHS; Nuffield Department of Population Health, University of Oxford: Oxford, UK, 2009. [Google Scholar]
  36. McConachie, H.; Colver, A.F.; Forsyth, R.J.; Jarvis, S.N.; Parkinson, K.N. Participation of disabled children: How should it be characterised and measured? Disabil. Rehabil. 2006, 28, 1157–1164. [Google Scholar] [CrossRef] [PubMed]
  37. Voepel-Lewis, T.; Malviya, S.; Tait, A.R. Validity of parent ratings as proxy measures of pain in children with cognitive impairment. Pain Manag. Nurs. 2005, 6, 168–174. [Google Scholar] [CrossRef] [PubMed]
  38. Vance, Y.H.; Morse, R.C.; Jenney, M.E.; Eiser, C. Issues in measuring quality of life in childhood cancer: Measures, proxies, and parental mental health. J. Child. Psychol. Psychiatr. 2001, 42, 661–667. [Google Scholar] [CrossRef]
  39. Mandak, K.; O’Neill, T.; Light, J.; Fosco, G.M. Bridging the gap from values to actions: A family systems framework for family-centered AAC services. Augment. Altern. Commun. 2017, 33, 32–41. [Google Scholar] [CrossRef] [PubMed]
  40. Mortenson, W.B.; Demers, L.; Fuhrer, M.J.; Jutai, J.W.; Lenker, J.; DeRuyter, F. How assistive technology use by individuals with disabilities impacts their caregivers: A systematic review of the research evidence. Am. J. Phys. Med. Rehabil. 2012, 91, 984–998. [Google Scholar] [CrossRef] [PubMed]
  41. Saito, Y.; Turnbull, A. Augmentative and alternative communication practice in the pursuit of family quality of life: A review of the literature. Res. Pract. Persons Sev. Disabil. 2007, 32, 50–65. [Google Scholar] [CrossRef]
  42. Fuhrer, M.J.; Jutai, J.W.; Scherer, M.J.; DeRuyter, F. A framework for the conceptual modeling of assistive technology device outcomes. Disabil. Rehabil. 2003, 25, 1243–1251. [Google Scholar] [CrossRef] [PubMed]
  43. World Health Organization. ICF, International Classification of Functioning, Disability, and Health: Child and Youth Version; WHO Press: Geneva, Switzerland, 2007. [Google Scholar]
  44. Rosenbaum, P.; King, S.; Law, M.; King, G.; Evans, J. Family-Centred service: A conceptual framework and research review. Phys. Occup. Ther. Pediatr. 1998, 18, 1–20. [Google Scholar] [CrossRef]
  45. Henderson, S.; Skelton, H.; Rosenbaum, P. Assistive devices for children with functional impairments: Impact on child and caregiver function. Dev. Med. Child Neurol. 2007, 50, 89–98. [Google Scholar] [CrossRef] [PubMed]
  46. Schonberger, A. Content Validity of the Family Impact of Assistive Technology Scale for Writing Devices; Final Research Report for Master of Science in Occupational Therapy; Department of Occupational Science & Occupational Therapy, University of Toronto: Toronto, ON, Canada, 2008. [Google Scholar]
  47. Ryan, S.E.; Campbell, K.A.; Rigby, P.; Germon, B.; Chan, B.; Hubley, D. Development of the New Family Impact Assistive Technology Scale (FIATS). Int. J. Rehabil. Res. 2006, 29, 195–200. [Google Scholar] [CrossRef] [PubMed]
  48. Yeoman, R. Item Generation and Face Validity of the Family Impact of Assistive Technology Scale for Writing Devices; Final Research Report for Master of Science of Occupational Therapy; Department of Occupational Science & Occupational Therapy, University of Toronto: Toronto, ON, Canada, 2009. [Google Scholar]
  49. Ryan, S.E.; Campbell, K.A.; Rigby, P. Reliability of the Family Impact of Assistive Technology Scale for families of young children with cerebral palsy. Arch. Phys. Med. Rehabil. 2007, 88, 1436–1440. [Google Scholar] [CrossRef] [PubMed]
  50. Dillman, D.A.; Smyth, J.; Christian, L.M. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 3rd ed.; Wiley & Sons: Toronto, ON, Canada, 2009. [Google Scholar]
  51. Cappelleri, J.C.; Lundy, J.J.; Hays, R.D. Overview of classical test theory and item response theory for quantitative assessment of items in developing patient-reported outcome measures. Clin. Ther. 2014, 36, 648–662. [Google Scholar] [CrossRef] [PubMed]
  52. Coster, W.; Bedell, G.; Law, M.; Khetani, M.A.; Teplicky, R.; Liljenquist, K.; Gleason, K.; Kao, Y.C. Psychometric evaluation of the Participation and Environment Measure for Children and Youth. Dev. Med. Child Neurol. 2011, 53, 1030–1037. [Google Scholar] [CrossRef] [PubMed]
  53. Delarosa, E.; Horner, S.; Eisenberg, C.; Ball, L.; Renzoni, A.M.; Ryan, S.E. Family impact of assistive technology scale: Development of a measurement scale for parents of children with complex communication needs. Augment. Altern. Commun. 2012, 28, 171–180. [Google Scholar] [CrossRef] [PubMed]
Table 1. Preliminary dimensions, definitions, and sample items.
Table 1. Preliminary dimensions, definitions, and sample items.
DimensionDefinitionNumber of ItemsSample Item
Attitudes Towards WritingDegree to which the child has a positive attitude towards writing.8My child likes to write.
Caregiver Relief 1Degree to which parent needs relief from caregiving.9I need help to take care of my child.
Contentment 1Degree to which the child is content during the day.9My child gets frustrated easily.
Doing Activities 1Degree to which the child has control over his or her own actions.5My child can communicate with others.
Establishing RoutinesDegree to which the child and family establish daily routines.5My child looks forward to going to school each morning.
ExertionDegree of energy needed to assist the child.7It is hard work helping my child to write.
IndependenceDegree to which the child writes independently.7My child writes without help.
Parent Well-BeingDegree to which the parent’s physical and psychological health is affected.8My life is challenged due to my child’s communication difficulties.
Quality of WritingDegree to which the child produces quality written work.8My child’s written communication is clear.
Social AdaptabilityDegree to which the child interacts with others in various social situations.8My child writes to other people all the time.
Supervision 1Degree to which the child requires attention from family members.7My child needs me nearby to do many activities.
Writing ActivitiesDegree to which the child performs written tasks.7My child writes without getting tired.
1 Dimension and items shared with original FIATS.
Table 2. Survey participants, their children, and writing technologies used (n = 121).
Table 2. Survey participants, their children, and writing technologies used (n = 121).
CharacteristicNumberPercentage
Respondent
 Mother10082.6
 Father1613.2
 Other 154.2
Child Profile
 M7864.5
 F4335.5
Primary Diagnosis
 Cerebral palsy3629.8
 Developmental delay2218.2
 Acquired brain injury1714.0
 Autism spectrum disorder1310.7
 Other diagnoses 23327.3
Writing technologies used
 Switches21.7
 Special keyboard2218.2
 Special mouse1411.6
 Special personal computer2419.8
 Special software6856.2
 Standard keyboard7965.3
 Standard mouse7360.3
 Standard personal computer8771.9
 Standard software5243.3
 Other writing devices 3119.1
1 Other respondents: mother and grandmother, mother and father, not specified. 2 Other primary diagnoses (fewer than 10 children in each category): amputation, brachial plexus, Down syndrome, juvenile rheumatoid arthritis, hydrocephalus, learning disability, muscular dystrophy, spina bifida, spinal cord injury, spinal muscular atrophy, other neurodevelopmental disorders. 3 Unspecified.
Table 3. Internal consistency of revised FIATS-WI and its subscale.
Table 3. Internal consistency of revised FIATS-WI and its subscale.
DimensionNumber of Items/DimensionsnCronbach’s Alpha
Total FIATS-WI101210.86
Attitudes Towards Writing81110.88
Caregiver Relief91120.90
Contentment71150.66
Doing Activities51190.60
Exertion61140.82
Independence81170.87
Quality of Writing81120.90
Social Adaptability81130.79
Supervision71160.79
Writing Activities81110.76
Note: n: number of respondents who provided ratings for all items in a scale as required for the alpha calculation.
Table 4. Descriptive statistics and intraclass correlation coefficients (ICC) for test–retest reliability.
Table 4. Descriptive statistics and intraclass correlation coefficients (ICC) for test–retest reliability.
DimensionTime 1 (n = 33)Time 2 (n = 28)Test–Retest Reliability
MeanSDMeanSDICC95%CI
Total FIATS-WI45.328.0244.487.480.940.88–0.97
Attitudes Towards Writing4.321.364.201.440.930.86–0.97
Caregiver Relief4.021.374.001.400.950.89–0.98
Contentment4.631.024.541.010.930.85–0.97
Doing Activities5.450.965.391.120.900.78–0.95
Exertion3.941.444.051.290.870.71–0.94
Independence4.621.414.331.170.900.78–0.95
Quality of Writing4.591.154.311.170.940.84–0.98
Social Adaptability4.441.214.561.000.930.84–0.97
Supervision4.461.124.351.230.920.84–0.96
Writing Activities4.840.964.751.030.920.83–0.96
Note: ICC: intraclass correlation coefficient; CI: confidence interval.
Table 5. FIATS-WI total scale/subscale correlations with the how often and how involved the child is in home activities using the PEM-CY.
Table 5. FIATS-WI total scale/subscale correlations with the how often and how involved the child is in home activities using the PEM-CY.
DimensionsHow Often (n = 28)How Involved (n = 27)
rsp-Valuersp-Value
Total FIATS-WI0.41 *0.030.42 *0.02
Attitudes Towards Writing0.170.390.260.20
Caregiver Relief0.340.080.370.05
Contentment0.220.250.260.19
Doing Activities0.340.070.280.16
Exertion0.070.720.120.55
Independence0.060.750.160.43
Quality of Writing0.070.730.220.28
Social Adaptability0.170.400.220.28
Supervision0.42 *0.030.340.08
Writing Activities0.110.570.280.15
Note: rs: Spearman’s rank correlation coefficient; p-value: statistical significance. Coefficient marked * is significant at the 0.05 level (2-tailed).

Share and Cite

MDPI and ACS Style

Ryan, S.E.; Sophianopoulos, M.-B. Measurement of Assistive Technology Outcomes Associated with Computer-Based Writing Interventions for Children and Youth with Disabilities. Technologies 2017, 5, 19. https://doi.org/10.3390/technologies5020019

AMA Style

Ryan SE, Sophianopoulos M-B. Measurement of Assistive Technology Outcomes Associated with Computer-Based Writing Interventions for Children and Youth with Disabilities. Technologies. 2017; 5(2):19. https://doi.org/10.3390/technologies5020019

Chicago/Turabian Style

Ryan, Stephen E., and Mary-Beth Sophianopoulos. 2017. "Measurement of Assistive Technology Outcomes Associated with Computer-Based Writing Interventions for Children and Youth with Disabilities" Technologies 5, no. 2: 19. https://doi.org/10.3390/technologies5020019

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop