The Provision of Powered Mobility Devices in Italy: Linking Process with Outcomes

Abstract

The present paper reports on a series of studies conducted at the Regional Center for Assistive Technology (Centro Regionale Ausili, CRA) in Bologna, Italy. Our purpose was to identify a set of internationally validated instruments and a training circuit with a view to developing a structured and validated Powered Mobility Device (PMD) assessment and training program. To develop the program, three studies were conducted in order to identify: validated measures for assessing the user’s driving skills and training needs for using a PMD (Study 1); measures for evaluating the outcomes of the PMD (Study 2); and, the elements necessary for building a circuit for conducting PMD training (Study 3). In studies 1 and 2, the Wheelchair Skill Test 4.2 Power Wheelchair (WST) and the Wheelchair Outcome Measure (WhOM) were selected and pilot tested using QUEST 2.0. These studies represent an important step in the development and definition of a PMD assessment and training program to be implemented in routine clinical activities in a regional center for assistive technology in Italy. The measures, the circuit and the program will be further tested in future for validity and reliability in order to assess their efficacy in helping professionals to select the most adequate PMD for users, to conduct specific PMD training, and to evaluate PMD outcomes.

1. Introduction

The wheelchair is one of the most commonly used assistive technologies (AT) for enhancing the personal mobility of people with disability [1]. The term power mobility device (hereafter Powered Mobility Device (PMD)) is used here to refer to scooters, powered wheelchairs, and other devices such as self-balancing wheelchairs. The literature suggests that provision of PMDs may increase participation and quality of life for a wide range of people with limitations, including children and adolescents [2], adults [3], and the elderly [3,4]. In order for users to achieve those benefits, they must have access to experienced providers and suppliers familiar with the equipment, have recommended devices and equipment appropriate for their needs, and utilize the equipment [5]. The identification of a PMD, in particular, is a complex process which requires validated strategies and measures. Furthermore, to achieve the best match between user and PMD, it is necessary that AT professionals employ validated tools and procedures [6]. A breakdown at any point of the PMD service delivery process might adversely affect benefits from the AT intervention. Inappropriate mobility devices, for example, may result in pressure ulcers, falls, accidents, and AT abandonment or underutilization [5], which in turn may have a negative effect on quality of life. Several PMD-specific measures have been developed to guide AT professionals in the selection of the most appropriate PMD solution [7,8,9,10] and in evaluation of the outcomes of PMD intervention [11,12,13].

In Italy, however, the PMD selection process is rarely supported by validated measures. Indeed, current clinical practices are well-entrenched and clinicians’ attitudes, knowledge, and skills may be barriers to address. Furthermore, as far as the Italian context is concerned, PMD service delivery organization and mechanisms vary broadly within and between regions and may differ in relation to disability policy, socio-economic context, and history, thus resulting in a variety of service delivery systems and models. This makes the introduction and adoption of measures and practices that are not context-specific—though validated—a difficult task [14]. For these reasons, too often—and in most clinical contexts—the identification and selection of the most adequate PMD as well as the assessment of its effects depend almost exclusively on the expertise of the AT professional or technician [15].

Recently, several contributions have highlighted the need to adopt evidence-based practices with respect to all aspects of the AT service delivery process [16,17,18]. In particular, AT and rehabilitation professionals are requested to employ validated tools throughout the service delivery process [16], from identification and selection of the device to evaluation of the outcomes of the intervention [14]. To face the challenges related to the introduction of validated measures and practices in AT services [19], we propose that AT professionals adopt a strategic approach [20]. This consists of using measures as part of a broader quality assurance process in AT service delivery (ATSD). In the field of AT, a recent position paper published by the Association for the Advancement of Assistive Technology in Europe (AAATE) clearly recommends that “service delivery systems should include quality assurance procedures for self-correcting quality control of the process, of the devices provided, and the outcomes” [21]. In this view, the ultimate goal of quality assurance is to use measures in order to support actions guiding the improvement of service delivery [20].

To this end, we recently developed a framework for AT service delivery quality assurance which may be used to include validated measures in AT practice. In detail, our framework was developed by combining the healthcare quality evaluation framework developed by Donabedian [22] with the quality criteria set by the AAATE [21] (see Figure 1). According to Donabedian, healthcare evaluation may be defined by distinguishing between structure, process, and outcome [22]. Structure refers to organizational factors that define the health system under which care is provided (e.g., health facilities, staff characteristics). Process refers to interactions between users and the healthcare structure [22,23] (e.g., diagnosis, treatment, referral, interpersonal communication). Outcome refers to the consequences of interaction between individuals and the healthcare system, related to structure and process [22,23]. Furthermore, in order to assess ATSD process quality, the AAATE identified six quality criteria [19,21,24]: “Accessibility”, referring to information on ATSD-related service access (e.g., waiting times, ATSD process duration); “Competence”, referring to information on professionals’ AT-related know-how (e.g., education of the professionals involved in the ATSD process, AT training offered); “Coordination”, referring to information on AT-related service integration (e.g., contact between the professionals involved in the ATSD process); “Efficiency”, referring to information on control of the ATSD process (e.g., complexity of procedures, availability of mechanisms able to control costs and process effectiveness); “Flexibility”, referring to information on ATSD-related service ability to respond to different user needs (e.g., use of a multidisciplinary approach); and “User Influence”, referring to information on user involvement (e.g., user presence at any stage of the ATSD process). Taken together, these contributions provide AT professionals with a common frame of reference in organizing an ATSD quality assurance process.

Our framework has been used in a series of studies [20] to either develop or identify appropriate measures for assessing both AT service delivery process and outcome targeting children with multiple disabilities. The quality assurance framework we have developed assumes that a link exists between specific aspects of the AT service delivery process (e.g., use of validated strategies for AT assessment, regular follow-up services) and outcomes of AT intervention (e.g., AT device adoption, use, and increased effectiveness; users’ satisfaction with ATSD).

Recently, we sought to extend our quality assurance framework to PMD service provision. For this reason, we first needed to identify a set of measures to assess the quality of specific aspects of the PMD service delivery process and outcomes. We therefore conducted three studies to identify a set of internationally validated measures and a training circuit in order to develop a structured and validated PMD assessment and training program assuring quality during the whole PMD service delivery process. In detail, the aims of the three studies were the following: the first study (Study 1) concerned identification of the most appropriate measure to guide AT professionals during the PMD assessment and training processes; the second study (Study 2) concerned identification of measures to evaluate outcomes of PMD intervention in terms of a user’s activities and participation changes; the third study (Study 3) concerned identification of the elements necessary to develop a PMD training circuit.

Here we report on the main results of the three studies conducted and we discuss the implications in practice.

2. Materials and Methods

2.1. Context of the Studies

A detailed description of the AT service delivery process in Italy is provided in Appendix A. The studies here described were conducted at the Regional AT Center (Centro Regionale Ausili, CRA) in Bologna (Italy), which is the center of reference for technical assessment, information, advice, training, and research on AT, established in 2000 by the Departments of Health and Social Policy of the Emilia Romagna Regional Council. The CRA serves as a support service for the health professionals working in the Emilia Romagna region. Its main scope is to promote the autonomy of people with disability by giving them and their significant others (e.g., families, health professionals) advice on AT solutions in the following areas: mobility and seating, transfers and lifting, personal care, home adaptations, accessibility, and technologies supporting interaction with the environment, sports and leisure. The CRA provides multidisciplinary consultation, especially in complex cases where intervention is requested by the individual’s health service provider.

2.2. Design of the Studies

The three studies were conducted in the period October 2012–October 2015. For each study, a common three-phase approach was followed: (1) systematic literature review; (2) discussion of review results with stakeholders; and (3) validation/implementation of the decisions taken in the discussion groups. Figure 2 illustrates the entire research process.

In the following sections, we provide a synthesis of the methodology used in every phase for each of the three studies conducted.

2.2.1. Phase One: Systematic Literature Reviews

In the first phase, for each of the three studies, a systematic review of the literature was conducted using the PubMed database and employing specific inclusion/exclusion criteria for the selection of the titles, abstracts, and full texts (for details concerning the search strategy and the review results, see [26]). In brief, identification of the papers to be included in the review was undertaken in three steps [27]. The first step involved reviewing the titles of the papers retrieved by three independent researchers and scored as: not relevant (0), probably relevant (1), or relevant (2). For this step, broad inclusion criteria were used in order to include any possible paper referring to the topic under investigation. The three scores were added up to make a sum score ranging from 0 to 6. All publications at or above the cut-off score of 2 were selected for the next step. The chosen cut-off score allowed papers marked as probably relevant by two reviewers or as relevant by one reviewer to be included in the next step. In the second step, abstracts of the selected papers were read and scored by the same three reviewers according to more specific criteria than those utilized in the previous step. No restrictions were made regarding publication type. The scores given by each reviewer were added up to make a sum score ranging from 0 to 6. All papers at or above the cut-off score of 2 were included in the subsequent step. Lastly, in the third step, for each study, all the selected papers were read in full and judged by one reviewer to establish their relevance in addressing the target topic.

2.2.2. Phase Two: Discussion Groups with Stakeholders

In the second phase, the results of each systematic review were discussed by a group of rehabilitation professionals (n = 4; 1 physiotherapist, 1 occupational therapist, 1 social educator, 1 psychologist) and a PMD user. The aim of the discussion groups was to select the most adequate measures among those identified in the previous phase and to design the circuit for the PMD training. The measures selected were then adapted and translated into Italian through two forward translations and one backward independent translation following international recommended guidelines [28].

2.2.3. Phase Three: Pilot Testing of the Measures Selected and Test Usability

Once translated, the measures selected in Study 1 and Study 2 were pilot tested. In Study 1, all the referrals from the CRA were included in the study over a 12-month period. No age or pathology restrictions were used. Those with an intellectual disability were excluded. In total, 35 cases were included in the study. Two researchers independently scored the selected measure. Agreement between researchers was employed as a measure of reliability using weighted kappas. Kappa values greater than 0.80 were considered almost perfect, 0.61–0.80 substantial, 0.41–0.60 moderate, 0.21–0.40 fair, and <0.20 poor. The tool was considered reliable if all the kappas fell into the range of 0.40–0.60.

In Study 2, a convenience sample of eight users was recruited utilizing the same criteria used in Study 1. The sample was asked to complete the selected tool. Face validation was assessed by asking the users, “Do you think this tool is suitable for exploring the impact of your PMD on your participation?” Convergent validation to assess whether the construct behind the Italian version of the selected tool correlated with a well-validated tool assessing—partly—the same construct was also evaluated. To this end, respondents were asked to answer not only the selected measure but also eight questions in the QUEST 2.0 [29], applying a scale from 1.00 (“not at all satisfied”) to 5.00 (“very satisfied”). The QUEST 2.0 addresses the user’s satisfaction with the AT device in relation to different aspects (comfort, weight, durability, adjustments, simplicity of use, dimensions, effectiveness, safety). A Spearman correlation was calculated between each item in the QUEST 2.0 and the subscales of the selected measure. The level of significance was set at p = 0.05.

Lastly, usability of the selected measure in clinical practice was pursued by asking four CRA expert professionals (two occupational therapists, one social educator, and one psychologist) for their opinion on its use satisfaction (range 0–10) considering three specific properties: ease of use, time of administration, and clinical utility.

3. Results

3.1. Participants

Table 1. Participants’ characteristics.

User	Study	Gender	Age	Experienced with PMD	Health Condition	Track	Type of Controller
1	1	M	65	Experienced user	Amputee	R	Joystick
2	1	M	68	Experienced user	Polio	M	Joystick
3	1	M	33	Experienced user	SCI	R	Self-propulsion with manual wheelchair
4	1	F	16	Experienced user	CP	R	Joystick
5	1	M	27	Experienced user	MD	F	Mini-joystick
6	1	M	68	Experienced user	Polio	M	Joystick
7	1	M	12	Experienced user	CP	M	Joystick
8	1	M	50	First PMD	SCI	F	Chin control
9	1	F	54	First PMD	Myopathy	M	Joystick
10	1	M	19	Experienced user	MD	R	Joystick
11	1	M	22	Experienced user	MD	R	Mini-joystick
12	1	M	18	Experienced user	CP	R	Joystick
13	1	F	28	First PMD	TBI	M	Joystick
14	1	F	43	Experienced user	CP	R	Joystick
15	1	M	24	Experienced user	MD	R	Joystick
16	1	M	52	First PMD	ALS	M	Foot control
17	1	F	33	First PMD	MS	M	Mini-joystick
18	1	M	25	Experienced user	MD	M	Mini-joystick
19	1	M	58	First PMD	MND	R	Joystick
20	1	M	51	First PMD	ALS	R	Joystick
21	1	F	69	First PMD	Stroke	R	Joystick
22	1	M	52	Experienced user	CP	M	Joystick
23	1	M	24	Experienced user	CP	R	Joystick
24	1	M	31	Experienced user	SCI	R	Joystick (via Smartphone)
25	1	M	51	Experienced user	SCI	M	Joystick
26	1	M	58	First PMD	ALS	M	Head control
27	1	M	23	First PMD	MD	R	Joystick
28	1	M	53	First PMD	MD	M	Mini-joystick
29	1	M	52	First PMD	SCI	M	Joystick
30	1	F	84	First PMD	ALS	M	Joystick
31	1	M	54	First PMD	RA	R	Joystick
32	1	M	14	First PMD	CP	M	Joystick
33	1	M	54	First PMD	ALS	M	Joystick
34	1	F	33	First PMD	MS	M	Mini-joystick
35	1	M	51	First PMD	ALS	M	Joystick
36	2	M	33	First PMD	CP	R	Handlebars
37	2	F	44	First PMD	CP	R	Joystick
38	2	F	84	First PMD	CP	M	Joystick
39	2	M	71	First PMD	ALS	M	Joystick
40	2	M	26	Experienced user	MD	F	Mini-joystick
41	2	M	43	Experienced user	ALS	M	Head control
42	2	M	14	First PMD	CP	M	Joystick
43	2	M	68	Experienced user	SCI	F	Joystick

Abbreviations: CP, cerebral palsy; SCI, spinal cord injury; MD, muscular dystrophy (Duchenne); MND, motor neuron disease; ALS, amyotrophic lateral sclerosis; TBI, traumatic brain injury; MS, multiple sclerosis; RA, rheumatoid arthritis; F, front-wheel drive; R, rear-wheel drive; M, mid-wheel drive.

provides a case-by-case description of participants included in the pilot studies conducted both in Study 1 and Study 2. Overall, participants were adults (mean (M) age: 43 years; standard deviation (SD): 19.7 years; min-max: 12–84 years). Ten (23%) were female PMD users. Most were novice PMD users (n = 23 [53%]).

3.2. Study 1

In total, 23 full texts were identified in the literature review, from which four measures were identified and debated in the discussion group. Wheelchair Skill Test 4.2 Power WC (WST 4.2 PWC; [30]) was selected among the instruments found, translated into Italian, and adapted to the CRA service process. The decision to use the WST was taken since during the discussion group it appeared to both users and professionals that the WST was easier and faster to complete compared to other tools. The WST is a standardized evaluation method that permits a set of representative wheelchair skills. This test is intended to assess a specific person in a specific wheelchair in a standardized manner. Before starting the pilot tests, all changes for the Italian version were discussed with the original authors (now available upon request at [email protected]). Two raters (occupational therapists) independently scored the WST-Italian version (hereafter, WST-I) for each user (70 tests in total). Overall, pilot test results showed good agreement between professionals with weighted kappas ranging from 0.3 to 1 (see Appendix B for details), with only 5 out of 32 WST-I items showing low agreement (kappa < 0.4). On the basis of these results, the WST-I was included in routine clinical assessment of the CRA service.

3.3. Study 2

In the second study, a total of 20 references were identified, and these included 11 measures (the search strategy and review results are reported in [31]). The Wheelchair Outcome Measure (WhOM; [11]) was selected at the end of the discussion group among stakeholders, translated into Italian, and adapted for the CRA service. The WhOM is a two-part questionnaire. The first part consists of a semi-structured interview and the second part consists of structured questions. The WhOM is client-centerd: in the first part, the client identifies participation activities both in the home (question 1) and in the community (outside the home, question 2). The client then rates both perceived “importance” of each activity and “satisfaction” with his/her current performance for each activity on an 11-point Likert scale (0–10). In the second part, the client answers three questions concerning aspects related to comfort with the PMD, and the results are recorded on the scoring sheet.

From analysis of the respondents’ opinions, the Italian version of the WhOM (hereafter WhOM-I) resulted comprehensible to the users and suitable for PMD outcome assessment. Preliminary analyses conducted to explore convergent validity showed a strong correlation between WhOM-I question 2 and “efficacy” as assessed by the QUEST 2.0 (rho = 0.57; p <0.001), and between the total score for the second part of the WhOM-I and the “comfort” item as assessed by the QUEST 2.0 (rho = 0.89; p <0.001). The authors of the original version of the WhOM were consulted and agreed on the current Italian version [32].

Concerning perceived usability in clinical practice, the scale was considered easy to use (M: 8; SD: 0), quick to administer (M: 7.5; SD: 1.3), and useful (M: 9; SD: 0).

3.4. Study 3

In the third study, elements necessary for the development of the training circuit were identified among those cited in the literature. The WST 4.2 PWC was employed as a guide for the selection of physical components and the design of the training circuit. To create continuity between the assessment and training phases (see Figure 3), we also considered the ‘wheelchair skills program (WSP) obstacles course guidelines’. In addition, within the circuit we also included pedestrian traffic elements (e.g., zebra crossing, traffic lights), since Italian traffic regulations consider PMD users as pedestrians (e.g., they circulate on pavements).

4. Discussion

The three studies described here represent an important step in the development and definition of a PMD assessment and training program to be implemented in routine clinical activities. Recently, the need for AT and rehabilitation professionals to document any aspect of the AT service delivery process has been emphasized [19,21,33,34]. Previous research has indeed shown that the assessment approach adopted by AT professionals affects the effectiveness of the recommendations made [35]. In this vein, our studies were an attempt to demonstrate that introducing validated instruments in clinical practice is possible and well-accepted if a systematic approach is used [33].

The rationale behind the current studies is the well-documented gap regarding the relationship between the characteristics of the services related to AT provision (assessment, training, implementation) and the outcomes of the assistive solutions [17]. In Italy as well as in the rest of the EU, the role and importance of specialized AT services within the healthcare systems seems to be far from well-consolidated. A possible reason, in our opinion, may be the lack of data on effectiveness of such services and, more in general, on ATSD quality in different countries. Indeed, relatively little research has been done on the relationships between specific AT service delivery attributes and the reporting of outcomes [17]. For this reason, assuring and improving ATSD quality is increasingly becoming a priority both at European and regional levels [21].

In the specific case of the three studies described here, following the framework for quality assurance described earlier, the WST-I was introduced to improve the current selection and training practices (or process) of the Local Health Authority (ASL) in Bologna, while the WhOM-I will be used to collect data concerning the effectiveness (or outcome) of the PMD intervention. Further research will be conducted to demonstrate that a direct link exists between specific aspects of the PMD selection process as defined by the WST-I and the PMD outcome.

To conclude, on the basis of the study results presented here, the PMD assessment and training program has been developed. Figure 4 illustrates the main steps composing the program. For each step, specific measures are indicated in order to document the whole process. In this vein, according to the quality assurance framework described earlier, it will be possible to link data collected during the assessment and training process to data gathered at the end of the whole intervention (outcome assessment).

The PMD program is considered still in its infancy; it will be further tested for validity and reliability in the future in order to assess its efficacy in helping professionals to select the most adequate PMD for users with disabilities and to conduct specific PMD training. In particular, as concerns the WST-I results, the five items with kappa lower than 0.4 will be rephrased and further tested in a second round of pilot tests. Moreover, the search for other PMD-related measures will continue: the choice of using a single database in the present study may be considered a limitation. However, this was performed considering the limited resources available to conduct the studies described here.

Furthermore, by extending the studies here described to other PMD providers, it may be possible to develop regional or national guidelines for PMD provision, and to create training courses for AT professionals. Overall, it is important to note that the process used here to develop assessment and training resources for power wheelchairs might also apply to manual wheelchairs.

In conclusion, an Italian multicenter study is desirable in order to continue validation of the WST-I and WhOM-I to investigate the use of both measures in other clinical contexts.