Next Article in Journal
An Automated System for Classification of Chronic Obstructive Pulmonary Disease and Pneumonia Patients Using Lung Sound Analysis
Next Article in Special Issue
Identifying Fatigue Indicators Using Gait Variability Measures: A Longitudinal Study on Elderly Brisk Walking
Previous Article in Journal
Sensitivity Analysis of Acoustic Emission Detection Using Fiber Bragg Gratings with Different Optical Fiber Diameters
Previous Article in Special Issue
Dual-Task Gait Stability after Concussion and Subsequent Injury: An Exploratory Investigation
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

An Objective Methodology for the Selection of a Device for Continuous Mobility Assessment

Tecla Bonci
Alison Keogh
Silvia Del Din
Kirsty Scott
Claudia Mazzà
1 and
on behalf of the Mobilise-D consortium
Department of Mechanical Engineering & INSIGNEO Institute for in Silico Medicine, The University of Sheffield, Sir Frederick Mappin Building, Mappin Street, Sheffield S1 3JD, UK
School of Public Health, Physiotherapy and Sports Science, University College Dublin, 4 Dublin, Ireland
Translational and Clinical Research Institute, Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne NE4 5TG, UK
Author to whom correspondence should be addressed.
Membership of Mobilise-D consortium is provided in the Acknowledgements.
Sensors 2020, 20(22), 6509;
Submission received: 6 October 2020 / Revised: 5 November 2020 / Accepted: 10 November 2020 / Published: 14 November 2020
(This article belongs to the Special Issue Wearable Sensors for Movement Analysis)


Continuous monitoring by wearable technology is ideal for quantifying mobility outcomes in “real-world” conditions. Concurrent factors such as validity, usability, and acceptability of such technology need to be accounted for when choosing a monitoring device. This study proposes a bespoke methodology focused on defining a decision matrix to allow for effective decision making. A weighting system based on responses (n = 69) from a purpose-built questionnaire circulated within the IMI Mobilise-D consortium and its external collaborators was established, accounting for respondents’ background and level of expertise in using wearables in clinical practice. Four domains (concurrent validity, CV; human factors, HF; wearability and usability, WU; and data capture process, CP), associated evaluation criteria, and scores were established through literature research and group discussions. While the CV was perceived as the most relevant domain (37%), the others were also considered highly relevant (WU: 30%, HF: 17%, CP: 16%). Respondents (~90%) preferred a hidden fixation and identified the lower back as an ideal sensor location for mobility outcomes. Overall, this study provides a novel, holistic, objective, as well as a standardized approach accounting for complementary aspects that should be considered by professionals and researchers when selecting a solution for continuous mobility monitoring.

1. Introduction

Mobility is recognised as one of the vital signs, as reduced mobility, reflected by slower walking speed and its reduction over time, has been associated with greater mortality, morbidity, cognitive decline, dementia, and falls risk [1]. Therefore, walking speed could be used as an outcome to monitor health and function, and evaluate innovative interventions or drug treatments [1,2,3]. Measurement of mobility usually occurs in laboratory or clinical settings [3], where individuals’ mobility capacity (what they can do) is tested under standardised conditions. However, this assessment could be influenced by clinicians’ subjectivity or by patients’ extra effort during short-term examinations [4]. Mobility performance (what they actually do) is instead assessed in the real-world and may show a better discriminative validity, especially in diseases characterised by specific mobility dysfunctions and fluctuations, such as in Parkinsons Disease [5]. Therefore, continuous mobility monitoring could detect, measure, and eventually predict mobility loss linked to a change in speed. This is, in turn, may provide essential information for a personalized treatment [1,3] as well as other adverse clinical events or outcomes. Therefore, a low-cost, easy-to-use, and accurate approach using technology that can operate in “real-world” scenario is essential to complete this aim, and wearable devices are ideal candidates.
A variety of data processing algorithms to estimate digital mobility outcomes (DMOs, e.g., walking speed, cadence, etc.), from either a single or multiple devices, have been proposed and validated [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22], and the effect of different device locations has also been assessed [23,24,25,26,27,28,29,30,31,32]. Nonetheless, algorithms and associated wearable devices are still far from widespread adoption outside of research labs due to several other limiting factors, such as human factors, wearability, usability, and data capture.
Effective deployment of continuous mobility monitoring is strongly influenced by the perception and acceptability of a wearable device to the user [33] and its wearability and usability [34,35]. However, these aspects have not been widely investigated. Usability of different wearable devices has been assessed in older adults [36,37], patients with chronic obstructive pulmonary disease (COPD) [38,39], adults with chronic diseases [40], and on target populations interested in wearable solutions (e.g., novel vs. experienced users [34]). The data capture process, and the amount of interaction expected from the participant, might also be a limiting factor in adopting wearable devices. For example, to enhance the accuracy of DMOs, some approaches require the subject to perform a given movement before data acquisition (e.g., holding a static posture as in Bugané et al. [7]) or input anthropometric measures [6,21], which can affect the overall experience for both participants and assessors.
While all of the above factors should be considered when selecting a wearable device for continuous mobility assessment, typically, these have only been considered in isolation [30,40,41] or as subsets [27,38,39]. One reason for this is the lack of a structured methodology to combine and objectively evaluate such various factors for a comprehensive assessment of concurrent wearable devices. Among these methodologies, decision matrices, typical of well-established design processes [42], are the most practical and objective tools for a multi-domain evaluation approach in selecting one option from several alternative solutions. Therefore, the primary aim of this study is to design a bespoke decision matrix to assist in selecting the optimal wearable device for continuous mobility monitoring. The study will initially identify the factors to be evaluated and their relevant scoring criteria (i.e., scoring system). The relevant importance of these factors in the overall assessment will then be established considering the perspective of professional and research staff using an ad-hoc questionnaire. This information will then be used to determine the decision matrix, and different practical examples of its use will be provided.

2. Materials and Methods

A decision matrix (Figure 1) is generally constituted of three main elements: (1) the different elements to be evaluated, (2) a weighting system to establish their relevant importance, and (3) a scoring system to rank various solutions [42]. The following sections describe how these elements were established in this study. A demonstration of how this tool can be used is also provided using data available in the literature.

2.1. Domains and Relevant Criteria

The factors to evaluate were grouped into the following four domains ( d = 1 , , 4 ). Both domains and the associated criteria ( c = 1 , , N ) were identified (Figure 2) through a combination of literature search and expert opinions within the IMI Mobilise-D consortium, which consists of many of the world’s leading scientists, clinicians, and companies on mobility assessment (>150 professionals from 34 partners;
  • Concurrent validity–factors related to the validity of the measurements;
  • Human factors–factors related to the context of data capture, perception of the user towards the technology, data security and privacy, effect of monitoring outside clinical settings;
  • Wearability & usability for the wearer–e.g., size, location, fixation modality, charging frequency;
  • Data capture process–e.g., whether a calibration procedure, device programming, or anthropometric information are required for appropriate data capture.

2.1.1. Concurrent Validity criteria

To properly assess the criteria within this domain, reference parameters measured with a gold standard system (e.g., stereophotogrammetry or instrumented walking for mobility evaluation) need to be available. While several parameters can be captured during continuous mobility monitoring, this study focused on real-world walking speed (RWS), as a representative example. Level of agreement (expressed as the interclass correlation coefficient–ICC), accuracy, robustness, and reliability of RWS measurements can be assessed to quantify associated sources of error. Since the validity of RWS estimation depends on both the identification of a walking bout [5], and the initial and final contacts of the foot with the floor [29], the validity of these events needs to be considered as well.

2.1.2. Human Factors Criteria

Acceptance and adoption of wearable devices are affected by the wearer’s view on the use of such devices to manage their health condition [33,34], data security [33], and their experience of, and adherence to, the proposed data capture process [38]. Of paramount importance for the wearer is the perceived impact that being monitored can have on daily life activities, as well as trust in the measurements collected by the device; perceived usefulness strongly correlates with wearer acceptance [43].

2.1.3. Wearability and Usability Criteria

Widespread deployment of wearables requires “perceived usefulness” by the stakeholders, and benefits of use to be balanced with “perceived ease of use” [43]. Comfort, battery life, and feedback provided by the device are additional elements to be considered within this domain [35,36], as well as its size, location, and method of attachment to the body [39,44,45].

2.1.4. Data Capture Process Criteria

Some devices/algorithms perform optimally when additional calibration procedures are performed, such as holding a static posture [7,31], device programming [46], or providing anthropometric measurements as an input [6,21]. These elements directly affect participant–device interaction and should be accounted for.

2.2. Weighting System

A questionnaire was designed to establish the relevance (i.e., weighting system) of the selected domains and criteria. Approval from the University of Sheffield Research Ethics Committee (Application 027211) was obtained for this study, and participants agreed to take part in the research by completing the anonymous online form. The online questionnaire was circulated among 34 partner institutions belonging to the Mobilise-D consortium, which consists of more than 150 professionals (e.g., scientists, clinicians, and companies) working on mobility assessment using wearable devices (, and its external collaborators. Before widespread distribution, the ad-hoc questionnaire was pilot tested for both readability and data acquisition by using feedback from various professionals.
Following the process visualized in Figure 3, the gathered responses were used to assess:
  • Respondents’ background: clinical, technical, or both.
  • Respondents’ level of expertise ( L o E ) with the use of wearable devices in clinical practice based on four questions:
    • Do you know how a wearable device works and how it is used to identify gait features?
    • As a researcher, have you ever used a wearable device?
    • Have you ever used wearable devices directly on patients as opposed to healthy individuals?
    • Have you ever analysed the information/data extracted from wearable devices to characterise patients’ mobility?
  • Each positive response was scored as 0.25, and the total L o E was obtained as a sum of the partial scores. L o E of each participant was then classified as excellent, good, average, poor, or none if total L o E was 1.00, 0.75, 0.50, 0.25, and 0, respectively.
  • Respondents’ perceived level of importance of each domain and criterion, based on a 1–5 Likert scale (1 = unimportant; 5 = very important).
  • The modal value of the responses of each domain and criterion, ω d and ω d , c , respectively, calculated as the preferences indicated by each respondent. The latter were multiplied by the relevant L o E , which allowed us to account for the relevant respondents’ level of expertise.
Finally, the computed weights were normalised as [42]:
For   each   domain   d :   w d = ω d d = 1 4 ω d
For   each   criterion   c :   w d , c = ω d , c c = 1 N ω d , c
where N are the criteria included in the relevant domain ( d ).

2.3. Scoring System

Each criterion was first classified as either “benefit” or “cost” (Table 1) and scored higher/lower if implying a better/worse sensor/algorithm solution [42], using scores that were normalised concerning their range of variation within each criterion and domain:
Benefit   criteria   e d , c = s d , c m i n i s d , c m a x d , c s d , c m i n d , c s d , c
Cos t   criteria   e d , c = m a x d , c s d , c   s d , c m a x d , c s d , c m i n d , c s d , c
where s d , c is the score assigned to the criteria c of the d domain ( 0 e d , c 1 ).
Only respondents who had declared to have a technical background were asked to score concurrent validity criteria based on the following definitions:
  • Accuracy: closeness of an estimated parameter ( p ) to the “true value” measured using a gold standard ( p G S ) and is expressed in percentage as:
    e % = | p p G S | | p G S | × 100
  • Robustness to changes in the device positioning, quantified as e % .
  • Reliability between different trials, quantified as e % .
  • ICC: the agreement between p and p G S in different trials.
  • Sensitivity (%): describes the true positive ( T P ) events, i.e., the number of gait events (GEs–defined as initial and final foot-to-ground contacts and used to identify strides, steps, as well as gait cycle phases [18], expressed as unitless numbers) and Walking Bouts (WBs) correctly identified with a device/algorithm solution ( n G E ) as compared to the values from a gold standard ( n G E _ G S ):
    s e n s % = n G E n G E _ G S × 100   or   s e n s % = T P T P + F N × 100
  • Specificity (%): number of true negative ( T N ) events relative to the actual events assessed with a gold standard:
    s p e c % = T N F P + T N × 100
  • Positive predictive value (%): T P events over the total amount of identified GEs, including falsely detected GEs ( T P + F P ):
    P P V % = T P T P + F P × 100
Criteria from the other domains were scored using the system shown in Table 1. Location and fixation modality criteria scores were defined by asking participants to rank possible choices taken from the literature [20,47,48,49]. They were then asked to indicate the best three from twelve locations (lower back/hip/waist; pocket; chest; neck (body-fixed); neck (pendant); head; foot; ankle; shank; thigh; wrist; arm) and five fixation modalities (adhesive on the skin; strap above/below clothes; clip above/below clothes). The recorded ranking scores (1, 2, 3 for 3rd, 2nd, 1st, respectively) were then scaled by the respondents’ L o E .

2.4. Comparison of Concurrent Solutions

For each monitoring solution ( E i ), an overall score, based on the partial scores obtained for the different domains and criteria and on the calculated weights and scores, was finally computed:
E i = d = 1 4 e d w d = d = 1 4 ( c = 1 N e d , c w d , c ) w d
where e d is the overall score of each domain d , obtained as the combination of the scores e d , c and normalised weights w d , c , assigned to each of the N criteria.

2.5. Application of the Decision Matrix

Among the different studies in the literature evaluating either different solutions for DMOs estimations, the information and results extracted from two studies were used to feed the decision matrix and practically demonstrate how this tool can be used in future research.
  • Example 1. Three different concurrent methods [10,16,21] for gait temporal parameter estimations with a single device that was attached to the lower trunk [31].
  • Example 2. An evaluation of four (Movemonitor, Mc Roberts, The Hague, The Netherlands; Up, Jawbone, San Francisco, USA; One, Fitbit, San Francisco, USA; ActivPAL, PAL Technologies Ltd., Glasgow, UK) of the seven wearable devices placed in different locations as explored in Storm et al. [28].
Among the different domains’ proposed criteria, a subset of the available scores for the relevant studies was available and used in the decision matrix. The weighting systems were, therefore, accordingly adjusted based on the results obtained in this study. Benefit and cost scores were assigned based on Table 1 and the relevant information obtained through the ad-hoc questionnaire (i.e., fixation modality and device location) and normalised as described in Section 2.3. For each wearable device, the overall score was calculated using Equation (5).

3. Results

3.1. Participants

Sixty-nine participants submitted their responses to the questionnaire (Figure 4). Among them, 83% had either an excellent or good level of expertise ( L o E     0.75 ) in the use of wearable devices.

3.2. Weighting System

Table 2 shows the normalised weights for domains and relevant criteria, as calculated based on each respondent’s perceived level of importance (Figure 5).
Based on the obtained modal values ω d of each domain, both concurrent validity and wearability and usability domains were classified as “very important” for a seven-day mobility monitoring solution. The other two domains were labeled as “important”; and this classification was not modified when the respondents’ L o E was considered (Figure 5).

3.3. Scores

The favourite location and fixation criteria were the “lower back/hip/waist” and “strap below clothes,” respectively, as shown by the results reported in Figure 6. The most common explanations behind the choice of the lower back/hip/waist location were the respondents’ previous experience with this solution with their patients, comfort, proximity to the centre of mass location, the possibility of the device to be integrated with a belt and the potential to “track” the movement of both lower limbs with a single device. The fixation with a strap below the clothes was indicated as preferred due to this method’s robustness, the possibility of hiding the sensor, and preserving participant privacy and past positive experiences with this approach.

3.4. Use of the Decision Matrix

3.4.1. Example 1

Among the methods described in Reference [31], the three for which the robustness had been assessed (T1–Zijlstra and Hof [21]; T2–González et al. [10], T3–McCamley et al. [16], Table 3) were considered for the concurrent evaluation. Step time accuracy and robustness (highest e % value reported for each method) were considered as representative for walking bout detection accuracy and robustness (Table 3), respectively.

3.4.2. Example 2

Among the seven wearable devices explored in Storm et al. [28], four (S1–Movemonitor, S2–Up, S3–One, S4–ActivPAL, Table 4) were selected for the concurrent evaluation, performed using step detection accuracy as a concurrent validity criterion (Table 4). The mean step detection accuracy value was calculated for each monitoring solution over those reported for slow, self-selected, and fast walking speeds.

4. Discussion

This study aimed to propose a standardised methodology for selecting the optimal device for continuous mobility monitoring, with a special focus on walking speed. Although this method was implemented using professionals/researchers, a similar approach could also be used to evaluate user perspectives. This approach’s novelty allows researchers to assess the relevance of domains that were previously quantified only in isolation [33,34,35,36,37,39,40], such as the wearability and usability of a device, in combination with aspects related to its validity and other domains. This ensures a more robust choice of a specific solution.
The different aspects to be considered while exploring concurrent continuous mobility monitoring solutions were first identified, and their relevance assessed by capturing information from experts in this research area. The identified domains of relevant criteria, and calculated weighting and scoring systems, were the three elements that identify the decision matrix, representing the successfully developed method.
The scoring system, which combined “benefit” and “cost” criteria, highlighted the differences among monitoring solutions and allowed the calculation of an overall score for each of them [42]. This procedure allows a trade-off on multiple and concurrent domain/criteria.
The weighting system was obtained via an experts’ questionnaire and constituted an objective methodology to assess the selected elements’ relevance while aiming to identify an optimal monitoring solution. Critically, this method’s reliability does not rely on the knowledge and expertise of a single decision-maker, which could bias the outcomes [42]. The novelty of this developed approach is that it allows researchers to consider the respondents’ expertise, making the unbiased results especially relevant for the field. The use of examples taken from the literature demonstrated how this framework could be used when only a subset of domains/criteria are available by adjusting the relevant scoring system to specific requirements.
From a professional’s perspective, the concurrent validity domain, which is the one most widely considered in the literature when a new wearable device is proposed, was also confirmed to be the most important in this study (37%), even by respondents from a non-technical background (33%). Nonetheless, results indicate that the other domains are also important for the widespread deployment of wearable devices (wearability and usability: 30%; human factors: 17%; data capture process: 16%). Recently, a study [50] attempted to provide some guidelines for selecting and comparing different devices; however, the focus was still mostly on how technical specifications and raw data quality affect the validity domain.
Respondents to the questionnaire, who were professionals (i.e., developers, clinicians, and researchers) who deploy the technology, were asked to select the best three location and fixation solutions. This has allowed for the establishment of an exact ranking among different solutions for continuous mobility monitoring. Although previous studies have assessed the effect of different device locations [23,24,25,26,27,28,29,30,31,32], the effect of a variety of fixation methods had not yet been explored. Thus, we have developed and applied a novel quantitative approach to allow these criteria to be explicitly identified and ranked. Almost 90% of the responders chose a device placed on the lower back (of these, 62%, 24%, and 14% identified this as the first, second, and third choice, respectively) because it provides accurate measurement and can be integrated with a belt. This solution is indeed usually accepted for long-term at-home use, approximates the centre of mass location, and is the most common location adopted in studies assessing mobility [48]. For the fixation method, the solution identified as an ideal one can be hidden under the clothes (83% of the respondents). In particular, a strap (43%) and adhesive on the skin (43%) were indicated as the most robust fixation methods (i.e., less relative motion between the device and the segment where it is placed). Other explicit preferences for location and fixation modality included choosing a solution that provides reliable measures, allows comfort, as well as device aesthetics, confirming what has been previously reported in the literature [35,43].
Once developed, the proposed framework was successfully used for ranking concurrent solutions using data extracted from the literature to compare different algorithms applied to the same raw data, which led to conclusions similar to those from the original study [31] while also providing a single summary score for each proposed solution. When used to evaluate the performance of the different wearable devices reported in Storm et al. [28], the differences among the solutions can be further highlighted, not just considering their concurrent validity, but also the other three domains, which are key elements for the widespread use of this technology. Moreover, a similar methodology could also be implemented when selecting concurrent devices for different applications.
Users (i.e., wearer, either patients or participants), which are the real stakeholders who will directly use this technology, did not participate in this questionnaire, which certainly represents the main limitation of this study. Nonetheless, their opinion might have been biased by their previous experience, which is usually limited to using a single wearable device. In order to include this essential aspect, future studies should either recruit a specific population of individuals who previously experienced different wearable solutions for mobility monitoring or having them participating in an ad-hoc comparative assessment. As highlighted by Manta et al. [51], both patients and care partners should be engaged in the selection and development of digital mobility outcome solutions for identifying a solution that is effective, helpful, and improves both quality and efficiency in clinical research and care. Future studies might include the opinion from a specific population of individuals who previously experienced different wearable solutions for mobility monitoring or include them in a comparative assessment. Their relevant perception of the importance of the identified factors could then be integrated and combined with the information collected in this study. Moreover, it would be of interest to evaluate the effect that awareness of the criteria adopted behind the design of a device might have on user perception and acceptance of the device.

5. Conclusions

This study proposed a new methodology that provides a novel, holistic, objective, as well as standardised approach accounting for complementary aspects that should be considered by professionals and researchers when selecting a solution for continuous mobility monitoring. An ad-hoc decision matrix has been established for this aim, the definition of which made it explicit that a comprehensive approach should be adopted when choosing a technology for continuous mobility monitoring if aiming for widespread adoption. In particular, the four identified domains: concurrent validity, human factors, wearability and usability, and data capture process should be simultaneously considered when evaluating concurrent solutions.

Author Contributions

Conceptualization, T.B., C.M., and S.D.D.; methodology, T.B., A.K., C.M., and S.D.D.; software, T.B.; formal analysis, T.B.; data curation, T.B.; writing—original draft preparation, T.B.; writing—review and editing, T.B., A.K., K.S., C.M., and S.D.D.; visualization, T.B.; supervision, C.M.; project administration, C.M.; funding acquisition, C.M. and S.D.D. All authors have read and agreed to the published version of the manuscript.


This study was co-funded by the NIHR through the Sheffield Biomedical Research Centre (BRC, grant number IS-BRC-1215-20017), by the European Union’s Horizon 2020 research and innovation programme and EFPIA via the Innovative Medicine Initiative 2 (Mobilise-D project, grant number IMI22017-13-7-820820), and the UK Engineering and Physical Sciences Research Council (Multisim and MultiSim2 projects, grant numbers EP/K03877X/1 and EP/S032940/1, respectively). SDD is also supported by the National Institute for Health Research (NIHR) Newcastle Biomedical Research Centre (BRC) based at Newcastle Upon Tyne Hospital NHS Foundation Trust and Newcastle University. The work was also supported by the NIHR/Wellcome Trust Clinical Research Facility (CRF) infrastructure at Newcastle upon Tyne Hospitals NHS Foundation Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care, the IMI, the European Union, the EFPIA, or any Associated Partners.
Sensors 20 06509 i001


We thank Lynn Rochester (Translational and Clinical Research Institute, Newcastle University Faculty of Medical Sciences, Newcastle upon Tyne, UK), Jennifer Rowson (Department of Mechanical Engineering & Insigneo Institute for in Silico Medicine, The University of Sheffield, UK), Andrea Cereatti (Department of Biomedical Sciences, University of Sassari, Italy), Felix Kluge (Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Bayern, Germany), Sarah Koch (Barcelona Institute for Global Health (ISGlobal); Universitat Pompeu Fabra (UPF), Barcelona, Spain), Vita Lanfranchi (Department of Computer Science, University of Sheffield, UK), M Encarna Micó-Amigo (Translational and Clinical Research Institute, Newcastle University Faculty of Medical Sciences, Newcastle upon Tyne, UK), and Luca Palmerini (Department of Electrical, Electronic, and Information Engineering, University of Bologna, Italy) for their invaluable input and comments into the initial study design. Full membership of the Mobilise-D consortium is available on the webisite:

Conflicts of Interest

The authors declare no conflict of interest.


  1. Studenski, S.; Perera, S.; Patel, K.; Rosano, C.; Faulkner, K.; Inzitari, M.; Nevitt, M. Gait speed and survival in older adults. JAMA 2011, 305, 50–58. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Cesari, M.; Kritchevsky, S.B.; Penninx, B.W.; Nicklas, B.J.; Simonsick, E.M.; Newman, A.B.; Visser, M. Prognostic value of usual gait speed in well-functioning older people—results from the Health, Aging and Body Composition Study. J. Am. Geriatr. Soc. 2005, 53, 1675–1680. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Perera, S.; Patel, K.V.; Rosano, C.; Rubin, S.M.; Satterfield, S.; Harris, T.; Newman, A.B. Gait speed predicts incident disability: A pooled analysis. J. Gerontol. Ser. A Biomed. Sci. Med Sci. 2016, 71, 63–71. [Google Scholar] [CrossRef]
  4. Galperin, I.; Hillel, I.; Del Din, S.; Bekkers, E.M.; Nieuwboer, A.; Abbruzzese, G.; Avanzino, L.; Nieuwhof, F.; Bloem, B.R.; Rochester, L.; et al. Associations between daily-living physical activity and laboratory-based assessments of motor severity in patients with falls and Parkinson’s disease. Parkinsonism Relat. Disord. 2019, 62, 85–90. [Google Scholar] [CrossRef] [PubMed]
  5. Del Din, S.; Godfrey, A.; Mazzà, C.; Lord, S.; Rochester, L. Free-living monitoring of Parkinson’s disease: Lessons from the field. Mov. Disord. 2016, 31, 1293–1313. [Google Scholar] [CrossRef] [PubMed]
  6. Aminian, K.; Najafi, B.; Büla, C.; Leyvraz, P.F.; Robert, P. Spatio-temporal parameters of gait measured by an ambulatory system using miniature gyroscopes. J. Biomech. 2002, 35, 689–699. [Google Scholar] [CrossRef]
  7. Bugané, F.; Benedetti, M.G.; Casadio, G.; Attala, S.; Biagi, F.; Manca, M.; Leardini, A. Estimation of spatial-temporal gait parameters in level walking based on a single accelerometer: Validation on normal subjects by standard gait analysis. Comput. Methods Programs Biomed. 2012, 108, 129–137. [Google Scholar] [CrossRef]
  8. Catalfamo, P.; Ghoussayni, S.; Ewins, D. Gait event detection on level ground and incline walking using a rate gyroscope. Sensors 2010, 10, 5683–5702. [Google Scholar] [CrossRef] [Green Version]
  9. Ferrari, A.; Ginis, P.; Hardegger, M.; Casamassima, F.; Rocchi, L.; Chiari, L. A mobile Kalman-filter based solution for the real-time estimation of spatio-temporal gait parameters. IEEE Trans. Neural Syst. Rehabil. Eng. 2015, 24, 764–773. [Google Scholar] [CrossRef] [Green Version]
  10. González, R.C.; López, A.M.; Rodriguez-Uría, J.; Alvarez, D.; Alvarez, J.C. Real-time gait event detection for normal subjects from lower trunk accelerations. Gait Posture 2010, 31, 322–325. [Google Scholar] [CrossRef]
  11. Lee, H.K.; Hwang, S.J.; Cho, S.P.; Lee, D.R.; You, S.H.; Lee, K.J.; Choi, H.S. Novel algorithm for the hemiplegic gait evaluation using a single 3-axis accelerometer. In Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA, 3–6 September 2009; pp. 3964–3966. [Google Scholar]
  12. Lee, J.A.; Cho, S.H.; Lee, Y.J.; Yang, H.K.; Lee, J.W. Portable activity monitoring system for temporal parameters of gait cycles. J. Med Syst. 2010, 34, 959–966. [Google Scholar] [CrossRef] [PubMed]
  13. Khandelwal, S.; Wickström, N. Identification of gait events using expert knowledge and continuous wavelet transform analysis. In Proceedings of the 7th International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS 2014), Angers, France, 3–6 March 2014; pp. 197–204. [Google Scholar]
  14. Köse, A.; Cereatti, A.; Della Croce, U. Bilateral step length estimation using a single inertial measurement unit attached to the pelvis. J. Neuroeng. Rehabil. 2012, 9, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Mariani, B.; Jiménez, M.C.; Vingerhoets, F.J.; Aminian, K. On-shoe wearable sensors for gait and turning assessment of patients with Parkinson’s disease. IEEE Trans. Biomed. Eng. 2012, 60, 155–158. [Google Scholar] [CrossRef] [PubMed]
  16. McCamley, J.; Donati, M.; Grimpampi, E.; Mazza, C. An enhanced estimate of initial contact and final contact instants of time using lower trunk inertial sensor data. Gait Posture 2012, 36, 316–318. [Google Scholar] [CrossRef]
  17. Sabatini, A.M.; Martelloni, C.; Scapellato, S.; Cavallo, F. Assessment of walking features from foot inertial sensing. IEEE Trans. Biomed. Eng. 2005, 52, 486–494. [Google Scholar] [CrossRef] [Green Version]
  18. Salarian, A.; Russmann, H.; Vingerhoets, F.J.; Dehollain, C.; Blanc, Y.; Burkhard, P.R.; Aminian, K. Gait assessment in Parkinson’s disease: Toward an ambulatory system for long-term monitoring. IEEE Trans. Biomed. Eng. 2004, 51, 1434–1443. [Google Scholar] [CrossRef]
  19. Shin, S.H.; Park, C.G. Adaptive step length estimation algorithm using optimal parameters and movement status awareness. Med. Eng. Phys. 2011, 33, 1064–1071. [Google Scholar] [CrossRef]
  20. Trojaniello, D.; Cereatti, A.; Pelosin, E.; Avanzino, L.; Mirelman, A.; Hausdorff, J.M.; Della Croce, U. Estimation of step-by-step spatio-temporal parameters of normal and impaired gait using shank-mounted magneto-inertial sensors: Application to elderly, hemiparetic, parkinsonian and choreic gait. J. Neuroeng. Rehabil. 2014, 11, 152. [Google Scholar] [CrossRef] [Green Version]
  21. Zijlstra, W.; Hof, A.L. Assessment of spatio-temporal gait parameters from trunk accelerations during human walking. Gait Posture 2003, 18, 1–10. [Google Scholar] [CrossRef] [Green Version]
  22. Sahoo, S.; Saboo, M.; Pratihar, D.K.; Mukhopadhyay, S. Real-Time Detection of Actual and Early Gait Events During Level-Ground and Ramp Walking. IEEE Sens. J. 2020. [Google Scholar] [CrossRef]
  23. Bongartz, M.; Kiss, R.; Lacroix, A.; Eckert, T.; Ullrich, P.; Jansen, C.P.; Feißt, M.; Mellone, S.; Chiari, L.; Becker, C.; et al. Validity, reliability, and feasibility of the uSense activity monitor to register physical activity and gait performance in habitual settings of geriatric patients. Physiol. Meas. 2019, 40, 095005. [Google Scholar] [CrossRef] [PubMed]
  24. Jasiewicz, J.M.; Allum, J.H.; Middleton, J.W.; Barriskill, A.; Condie, P.; Purcell, B.; Li, R.C.T. Gait event detection using linear accelerometers or angular velocity transducers in able-bodied and spinal-cord injured individuals. Gait Posture 2006, 24, 502–509. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Khandelwal, S.; Wickström, N. Evaluation of the performance of accelerometer-based gait event detection algorithms in different real-world scenarios using the MAREA gait database. Gait Posture 2017, 51, 84–90. [Google Scholar] [CrossRef]
  26. Mansour, K.B.; Rezzoug, N.; Gorce, P. Analysis of several methods and inertial sensors locations to assess gait parameters in able-bodied subjects. Gait Posture 2015, 42, 409–414. [Google Scholar] [CrossRef] [PubMed]
  27. Panebianco, G.P.; Bisi, M.C.; Stagni, R.; Fantozzi, S. Analysis of the performance of 17 algorithms from a systematic review: Influence of sensor position, analysed variable and computational approach in gait timing estimation from IMU measurements. Gait Posture 2018, 66, 76–82. [Google Scholar] [CrossRef] [PubMed]
  28. Storm, F.A.; Heller, B.W.; Mazzà, C. Step detection and activity recognition accuracy of seven physical activity monitors. PLoS ONE 2015, 10, e0118723. [Google Scholar] [CrossRef]
  29. Storm, F.A.; Buckley, C.J.; Mazzà, C. Gait event detection in laboratory and real life settings: Accuracy of ankle and waist sensor based methods. Gait Posture 2016, 50, 42–46. [Google Scholar] [CrossRef] [Green Version]
  30. Storm, F.A.; Nair, K.P.S.; Clarke, A.J.; Van der Meulen, J.M.; Mazza, C. Free-living and laboratory gait characteristics in patients with multiple sclerosis. PLoS ONE 2018, 13, e0196463. [Google Scholar] [CrossRef] [Green Version]
  31. Trojaniello, D.; Cereatti, A.; Della Croce, U. Accuracy, sensitivity and robustness of five different methods for the estimation of gait temporal parameters using a single inertial sensor mounted on the lower trunk. Gait Posture 2014, 40, 487–492. [Google Scholar] [CrossRef]
  32. Nguyen, M.D.; Mun, K.R.; Jung, D.; Han, J.; Park, M.; Kim, J.; Kim, J. IMU-based Spectrogram Approach with Deep Convolutional Neural Networks for Gait Classification. In Proceedings of the 2020 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 4–6 January 2020; pp. 1–6. [Google Scholar]
  33. Wang, X.; White, L.; Chen, X.; Gao, Y.; Li, H.; Luo, Y. An empirical study of wearable technology acceptance in healthcare. Ind. Manag. Data Syst. 2015, 115, 1704–1723. [Google Scholar]
  34. Jia, Y.; Wang, W.; Wen, D.; Liang, L.; Gao, L.; Lei, J. Perceived user preferences and usability evaluation of mainstream wearable devices for health monitoring. PeerJ 2018, 6, e5350. [Google Scholar] [CrossRef] [PubMed]
  35. Puri, A.; Kim, B.; Nguyen, O.; Stolee, P.; Tung, J.; Lee, J. User acceptance of wrist-worn activity trackers among community-dwelling older adults: Mixed method study. JMIR Mhealth Uhealth 2017, 5, e173. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Keogh, A.; Dorn, J.F.; Walsh, L.; Calvo, F.; Caulfield, B. Comparing the Usability and Acceptability of Wearable Sensors Among Older Irish Adults in a Real-World Context: Observational Study. JMIR Mhealth Uhealth 2020, 8, e15704. [Google Scholar] [PubMed]
  37. McMahon, S.K.; Lewis, B.; Oakes, M.; Guan, W.; Wyman, J.F.; Rothman, A.J. Older adults’ experiences using a commercially available monitor to self-track their physical activity. JMIR Mhealth Uhealth 2016, 4, e35. [Google Scholar] [CrossRef]
  38. Rabinovich, R.A.; Louvaris, Z.; Raste, Y.; Langer, D.; Van Remoortel, H.; Giavedoni, S.; Burtin, C.; Regueiro, E.M.; Vogiatzis, I.; Hopkinson, N.S.; et al. Validity of physical activity monitors during daily life in patients with COPD. Eur. Respir. J. 2013, 42, 1205–1215. [Google Scholar] [CrossRef] [Green Version]
  39. Vooijs, M.; Alpay, L.L.; Snoeck-Stroband, J.B.; Beerthuizen, T.; Siemonsma, P.C.; Abbink, J.J.; Rövekamp, T.A. Validity and usability of low-cost accelerometers for internet-based self-monitoring of physical activity in patients with chronic obstructive pulmonary disease. Interact. J. Med Res. 2014, 3, e14. [Google Scholar] [CrossRef]
  40. Mercer, K.; Giangregorio, L.; Schneider, E.; Chilana, P.; Li, M.; Grindrod, K. Acceptance of commercially available wearable activity trackers among adults aged over 50 and with chronic illness: A mixed-methods evaluation. JMIR Mhealth Uhealth 2016, 4, e7. [Google Scholar] [CrossRef]
  41. Welk, G.J.; Blair, S.N.; Wood, K.; Jones, S.; Thompson, R.W. A comparative evaluation of three accelerometry-based physical activity monitors. Med. Sci. Sports Exerc. 2000, 32, S489–S497. [Google Scholar] [CrossRef]
  42. Carver, S.J. Integrating multi-criteria evaluation with geographical information systems. Int. J. Geogr. Inf. Syst. 1991, 5, 321–339. [Google Scholar] [CrossRef] [Green Version]
  43. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Green Version]
  44. Caramia, C.; Torricelli, D.; Schmid, M.; Muñoz-Gonzalez, A.; Gonzalez-Vargas, J.; Grandas, F.; Pons, J.L. IMU-Based Classification of Parkinson’s Disease From Gait: A Sensitivity Analysis on Sensor Location and Feature Selection. IEEE J. Biomed. Health Inform. 2018, 22, 1765–1774. [Google Scholar] [CrossRef] [PubMed]
  45. Chen, S.; Lach, J.; Lo, B.; Yang, G.Z. Toward pervasive gait analysis with wearable sensors: A systematic review. IEEE J. Biomed. Health Inform. 2016, 20, 1521–1537. [Google Scholar] [CrossRef] [PubMed]
  46. Greene, B.R.; McGrath, D.; O’Neill, R.; O’Donovan, K.J.; Burns, A.; Caulfield, B. An adaptive gyroscope-based algorithm for temporal gait analysis. Med. Biol. Eng. Comput. 2010, 48, 1251–1260. [Google Scholar] [CrossRef] [PubMed]
  47. Awais, M.; Palmerini, L.; Chiari, L. Physical activity classification using body-worn inertial sensors in a multi-sensor setup. In Proceedings of the 2016 IEEE 2nd International Forum on Research and Technologies for Society and Industry Leveraging a better tomorrow (RTSI), Bologna, Italy, 7–9 September 2016; pp. 1–4. [Google Scholar]
  48. Howcroft, J.; Kofman, J.; Lemaire, E.D. Review of fall risk assessment in geriatric populations using inertial sensors. J. Neuroeng. Rehabil. 2013, 10, 91. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Summa, A.; Vannozzi, G.; Bergamini, E.; Iosa, M.; Morelli, D.; Cappozzo, A. Multilevel upper body movement control during gait in children with cerebral palsy. PLoS ONE 2016, 11, e0151792. [Google Scholar] [CrossRef] [PubMed]
  50. Zhou, L.; Fischer, E.; Tunca, C.; Brahms, C.M.; Ersoy, C.; Granacher, U.; Arnrich, B. How We Found Our IMU: Guidelines to IMU Selection and a Comparison of Seven IMUs for Pervasive Healthcare Applications. Sensors 2020, 20, 4090. [Google Scholar] [CrossRef] [PubMed]
  51. Manta, C.; Patrick-Lake, B.; Goldsack, J.C. Digital Measures That Matter to Patients: A Framework to Guide the Selection and Development of Digital Measures of Health. Digit. Biomark. 2020, 4, 69–77. [Google Scholar] [CrossRef]
Figure 1. (Left Panel) Structure of the procedures required to identify the three elements that compose a decision matrix. L o E = level of experience in the use of wearable devices. (Right Panel) Visual representation of the use of the decision matrix for ranking different wearable devices.
Figure 1. (Left Panel) Structure of the procedures required to identify the three elements that compose a decision matrix. L o E = level of experience in the use of wearable devices. (Right Panel) Visual representation of the use of the decision matrix for ranking different wearable devices.
Sensors 20 06509 g001
Figure 2. Identified key domains and their relevant criteria affecting wearable devices selection.
Figure 2. Identified key domains and their relevant criteria affecting wearable devices selection.
Sensors 20 06509 g002
Figure 3. Process figure showing how the gathered responses about the perceived level of importance of the different domains and criteria are used to identify the normalized weights for each domain (d) and criterion (c).
Figure 3. Process figure showing how the gathered responses about the perceived level of importance of the different domains and criteria are used to identify the normalized weights for each domain (d) and criterion (c).
Sensors 20 06509 g003
Figure 4. (a) Background of the respondents (n = 69); (b) Respondents’ level of expertise on the use of wearable devices in clinical settings as assessed through the purposely developed questionnaire.
Figure 4. (a) Background of the respondents (n = 69); (b) Respondents’ level of expertise on the use of wearable devices in clinical settings as assessed through the purposely developed questionnaire.
Sensors 20 06509 g004
Figure 5. For each perceived level of importance (1–5 Likert scale; 1 = unimportant, 5 = very important), the absolute number of responses expressed by the participants for the four domains (a) concurrent validity, (b) human factors, (c) data capture process, and (d) wearability and usability) are shown with a pattern fill. The responses adjusted by the relevant L o E of each participant are shown with a solid fill.
Figure 5. For each perceived level of importance (1–5 Likert scale; 1 = unimportant, 5 = very important), the absolute number of responses expressed by the participants for the four domains (a) concurrent validity, (b) human factors, (c) data capture process, and (d) wearability and usability) are shown with a pattern fill. The responses adjusted by the relevant L o E of each participant are shown with a solid fill.
Sensors 20 06509 g005
Figure 6. Scores for the different identified device locations (a) and fixation modality (b). Values were obtained based on the best three choices expressed from each participant and their relevant L o E .
Figure 6. Scores for the different identified device locations (a) and fixation modality (b). Values were obtained based on the best three choices expressed from each participant and their relevant L o E .
Sensors 20 06509 g006
Table 1. Cost/benefit criteria and scoring system.
Table 1. Cost/benefit criteria and scoring system.
Concurrent ValidityWalking speed accuracy Scores based on the relevant technical definitions
Walking speed robustness
Walking speed reliability
Walking speed–Interclass coefficient
Walking bout detection sensitivity
Walking bout detection specificity
Walking bout detection accuracy
Walking bout detection robustness
Walking bout detection reliability
Gait event sensitivity
Gait events identification
Human FactorsUse of technology in healthcare *
Data security Yes(1)/No(0)
Adherence to data capture Yes(1)/No(0)
Burden of data capture *
Impact of monitoring Yes(1)/No(0)
Trust in the device Commercial: Yes(1)/No(0)
Wearability and usabilityComfort *
Location 1
Ease of use Interaction: Yes(1)/No(0)
Frequency of recharging Battery Life 2
Perceived usefulness * NA
Whether it provides feedback Yes(1)/No(0)
Size width x height x depth x mass
Fixation modality 1
Data Capture ProcessCalibration procedure Yes(1)/No(0)
Required static/functional movements Yes(1)/No(0)
Required device programming Yes(1)/No(0)
Questionnaires/Anthropometric measures Yes(1)/No(0)
1 Scores established via the purposely developed questionnaire. 2 Daily recharging (5/5); 2–3 days BL (4/5); 4–5 days BL (3/5); 6–7 days BL (2/5); 7+ days BL (1/5). * Scores usually established through dedicated questionnaires available in the litarature.
Table 2. Weighting system.
Table 2. Weighting system.
Weight Weight
Concurrent Validity0.368Walking speed accuracy0.133
Walking speed reliability0.130
Walking speed robustness0.107
Walking speed–Interclass coefficient0.107
Walking bout detection specificity0.097
Walking bout detection reliability0.095
Walking bout detection accuracy0.087
Walking bout detection sensitivity0.064
Walking bout detection robustness0.062
Gait event sensitivity0.059
Gait events identification (PPV)0.057
Human Factors0.175Trust in the device0.193
Burden of data capture0.193
Data security0.181
Impact of monitoring0.163
Adherence to data capture0.136
Use of technology in healthcare0.134
Wearability and usability0.296Ease of use0.185
Fixation modality0.141
Perceived usefulness0.096
Frequency of recharging0.092
Whether it provides feedback0.083
Data Capture Process0.161Calibration procedure0.326
Required static/functional movements0.286
Required device programming0.197
Questionnaires/Anthropometric measures0.192
Table 3. Evaluation matrix applied to three concurrent methods. Normalised scores are reported in bold.
Table 3. Evaluation matrix applied to three concurrent methods. Normalised scores are reported in bold.
Domains Criteria
Weight WeightT1T2T3
Concurrent Validity0.368Walking bout detection accuracy 10.328842
Walking bout detection robustness 10.234942
Gait event identification (PPV)0.21510097100
Gait events sensitivity0.2239782100
Human Factors0.175Trust in the device0.516111
Data security0.484111
Wearability & usability0.296Fixation modality0.3010.1370.1370.137
Frequency of recharging0.197111
Data Capture Process0.161Calibration procedure0.326000
Required static/functional movements0.286111
Required device programming0.197000
Questionnaires/Anthropometric measures0.192100
Overall score0.700.730.95
1 Represented as step time accuracy and robustness.
Table 4. Evaluation matrix applied to four wearable devices. Normalised scores are reported in bold.
Table 4. Evaluation matrix applied to four wearable devices. Normalised scores are reported in bold.
Domains Criteria
Weight WeightS1S2S3S4
Concurrent Validity0.368Step detection accuracy1.0001.4834.8971.5672.493
Human Factors0.175Trust in the device0.5161111
Data security0.4841111
Wearability & usability0.296Fixation modality0.3010.3190.1740.1740.271
Frequency of recharging0.1970.
Data Capture Process0.161Calibration procedure0.3260000
Required static/functional movements0.2860000
Required device programming0.1971000
Questionnaires/Anthropometric measures0.1920000
Overall score0.890.410.810.78
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Bonci, T.; Keogh, A.; Del Din, S.; Scott, K.; Mazzà, C.; on behalf of the Mobilise-D consortium. An Objective Methodology for the Selection of a Device for Continuous Mobility Assessment. Sensors 2020, 20, 6509.

AMA Style

Bonci T, Keogh A, Del Din S, Scott K, Mazzà C, on behalf of the Mobilise-D consortium. An Objective Methodology for the Selection of a Device for Continuous Mobility Assessment. Sensors. 2020; 20(22):6509.

Chicago/Turabian Style

Bonci, Tecla, Alison Keogh, Silvia Del Din, Kirsty Scott, Claudia Mazzà, and on behalf of the Mobilise-D consortium. 2020. "An Objective Methodology for the Selection of a Device for Continuous Mobility Assessment" Sensors 20, no. 22: 6509.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop