Next Article in Journal
Transient Ischemic Attack Outpatient Clinic: Past Journey and Future Adventure
Next Article in Special Issue
Detecting Comparative Features of Comprehensive Geriatric Assessment through the International Classification of Functioning, Disability, and Health Linkage: A Web-Based Survey
Previous Article in Journal
Cervical Spinal Epidural Abscess: Diagnosis, Treatment, and Outcomes: A Case Series and a Literature Review
Previous Article in Special Issue
Comparison of Grip Strength Measurements by Widely Used Three Dynamometers in Outpatients Aged 60 Years and Over
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

eHealth in Geriatric Rehabilitation: An International Survey of the Experiences and Needs of Healthcare Professionals

by
Jules J. M. Kraaijkamp
1,2,3,*,
Anke Persoon
4,
Sorina Aurelian
5,
Stefan Bachmann
6,7,
Ian D. Cameron
8,
Mohamed-Amine Choukou
9,10,
Frances Dockery
11,
Kseniia Eruslanova
12,
Adam L. Gordon
13,
Stefan Grund
14,
Hyub Kim
15,
Andrea B. Maier
16,17,18,
Laura M. Pérez Bazan
19,
José E. Pompeu
20,
Eva Topinkova
21,22,
Mark A. Vassallo
23,
Niels H. Chavannes
2,
Wilco P. Achterberg
1,2 and
Eléonore F. Van Dam van Isselt
1,2
1
University Network for the Care Sector Zuid-Holland, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands
2
Department of Public Health and Primary Care, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands
3
ZZG Zorggroep, 6561 LE Nijmegen, The Netherlands
4
Department of Primary and Community Care, Radboud University Medical Center Nijmegen, 6525 GA Nijmegen, The Netherlands
5
Department of Geriatric, University of Medicine and Pharmacy “Carol Davila”-Chronic Disease Hospital “Sf. Luca”, 041915 Bucharest, Romania
6
Department of Internistic and Musculoskeletal Rehabilitation, Kliniken Valens, 7317 Valens, Switzerland
7
Department of Geriatrics, Faculty of Medicine, Inselspital, University of Bern, 3010 Bern, Switzerland
8
John Walsh Centre for Rehabilitation Research, Northern Sydney Local Health District and Faculty of Medicine and Health, University of Sydney, St Leonards, NSW 2065, Australia
9
Department of Occupational Therapy, College of Rehabilitation Sciences, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3E 0T6, Canada
10
Centre on Aging, University of Manitoba, Winnipeg, MB R3T 2N2, Canada
11
Beaumont Hospital & Royal College of Surgeons in Ireland, D09 V2NO Dublin, Ireland
12
Russian Clinical and Research Center of Gerontology, 129226 Moscow, Russia
13
Academic Unit of Injury, Recovery and Inflammation Sciences (IRIS), School of Medicine, University of Nottingham, Nottingham NG7 2UH, UK
14
Center for Geriatric Medicine, Agaplesion Bethanien Hospital Heidelberg, Geriatric Center at the Heidelberg University, 69117 Heidelberg, Germany
15
Department of Occupational Therapy, Far East University, 76-32, Daehak-gil, Gamgok-myeon, Eumseong 27601, Republic of Korea
16
Healthy Longevity Program, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117597, Singapore
17
Centre for Healthy Longevity, @AgeSingapore, National University Health System, Singapore 117456, Singapore
18
Department of Human Movement Sciences, @AgeAmsterdam, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, 1081 BT Amsterdam, The Netherlands
19
Research on Aging, Frailty and Care Transitions in Barcelona (REFiT-BCN), Parc Sanitari Pere Virgili and Vall d’Hebron Institute (VHIR), 08023 Barcelona, Spain
20
Department of Physiotherapy, Speech Therapy and Occupational Therapy, School of Medicine, University of São Paulo, São Paulo 05360-160, Brazil
21
Department of Geriatric Medicine, First Faculty of Medicine and General Faculty Hospital, 120 00 Prague, Czech Republic
22
Faculty of Health and Social Sciences, South Bohemian University, 370 11 České Budějovice, Czech Republic
23
Geriatric Medicine Society of Malta, Karin Grech Hospital, PTA 1312 Pieta, Malta
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2023, 12(13), 4504; https://doi.org/10.3390/jcm12134504
Submission received: 9 May 2023 / Revised: 30 June 2023 / Accepted: 3 July 2023 / Published: 5 July 2023
(This article belongs to the Special Issue Updates on Rehabilitation Care for Geriatric Disease)

Abstract

:
While eHealth can help improve outcomes for older patients receiving geriatric rehabilitation, the implementation and integration of eHealth is often complex and time-consuming. To use eHealth effectively in geriatric rehabilitation, it is essential to understand the experiences and needs of healthcare professionals. In this international multicentre cross-sectional study, we used a web-based survey to explore the use, benefits, feasibility and usability of eHealth in geriatric rehabilitation settings, together with the needs of working healthcare professionals. Descriptive statistics were used to summarize quantitative findings. The survey was completed by 513 healthcare professionals from 16 countries. Over half had experience with eHealth, although very few (52 of 263 = 20%) integrated eHealth into daily practice. Important barriers to the use or implementation of eHealth included insufficient resources, lack of an organization-wide implementation strategy and lack of knowledge. Professionals felt that eHealth is more complex for patients than for themselves, and also expressed a need for reliable information concerning available eHealth interventions and their applications. While eHealth has clear benefits, important barriers hinder successful implementation and integration into healthcare. Tailored implementation strategies and reliable information on effective eHealth applications are needed to overcome these barriers.

1. Introduction

With an aging global population and an ever-expanding number of older adults with one or more long-term conditions, the demands placed on geriatric rehabilitation are increasing rapidly [1]. Geriatric rehabilitation has been defined as “a multidimensional approach of diagnostic and therapeutic interventions, the purpose of which is to optimize functional capacity, promote physical activity and preserve functional reserve and social participation in older people with disabling impairments” [2]. Due to a rapidly expanding older population and an increasing lack of staff, new strategies are required to maintain and advance the implementation and delivery of geriatric rehabilitation. Promising solutions such as eHealth may be one way to help overcome these challenges.
One definition of eHealth is “the use of digital information and communication to support and/or improve health and healthcare” [3]. eHealth interventions vary widely, from simple approaches such as video communication, to complex treatment applications involving robotics. A growing body of evidence suggests that eHealth can contribute to improved outcomes for older patients receiving geriatric rehabilitation [4,5,6,7]. The COVID-19 pandemic highlighted the need for substantial changes in the delivery of rehabilitation, with reduced capacity, reduced time spent per patient and reduced access to rehabilitation facilities [8,9]. This emphasizes the importance of eHealth interventions that enable remote monitoring and treatment of patients, enhancing the accessibility and the continuity of rehabilitation. Although the COVID-19 pandemic has accelerated the use of eHealth, the adoption of eHealth is still lagging behind and a number of obstacles hinder the successful development, implementation and integration of eHealth in geriatric rehabilitation [10].
Successful implementation of eHealth involves considerable time and effort and is often complex [11,12,13]. To facilitate integration into clinical practice, implementation of eHealth may also require changes to a healthcare professional’s workflow [14,15]. Another challenge facing healthcare professionals is the ever-increasing number of eHealth interventions and staying up to date in which eHealth interventions are effective, feasible, usable and suit their specific context [16,17].
As healthcare professionals are central to the successful application of eHealth, the key to promoting implementation and integration of eHealth in geriatric rehabilitation is a better understanding of the experiences and needs of healthcare professionals. The goal of this study was to provide an overview of the use, benefits, feasibility, usability and needs of healthcare professionals regarding eHealth in geriatric rehabilitation across different countries. This study is part of the EAGER (EheAlth in GEriatric Rehabilitation) research line. The first study consisted of a systematic review of the effectiveness, feasibility and usability of eHealth in geriatric rehabilitation [4].

2. Materials and Methods

2.1. Design

An online international multicentre cross-sectional survey study was conducted between December 2021 and April 2022. Results were reported based on the Checklist for Reporting Results of Internet E-Surveys (CHERRIES), a 30-item checklist for web surveys [18].

2.2. Study Population and Setting

We included healthcare professionals who were (1) working in a geriatric rehabilitation setting, (2) aged 18 years old and over, (3) understood English and (4) had at least three months experience with the patient population. Healthcare professionals not available during the study period were excluded. Taking into account international variation between different healthcare systems and provision of geriatric rehabilitation [19,20], we included a range of geriatric rehabilitation settings such as post-acute rehabilitation facilities, acute hospitals, ambulatory settings, geriatric day hospitals, nursing homes, skilled nurse facilities and rehabilitation hotels.

2.3. Recruitment and Consent

Eligible healthcare professionals were recruited in geriatric rehabilitation facilities across 16 countries: Australia, Brazil, Canada, Czech Republic, Germany, Ireland, Malta, The Netherlands, New Zealand, Romania, Russia, Singapore, South Korea, Spain, Switzerland and the United Kingdom. Per country, one primary contact person was designated to distribute the survey to the geriatric rehabilitation facilities within that country. All primary contacts were experts in the field of geriatric rehabilitation and/or eHealth and were native speakers. Almost all persons acting as primary contacts were members of the European Geriatric Medical Society’s ‘Special Interest Group for Geriatric Rehabilitation’ and were recruited through this network. Distribution of the survey varied per country, based on the personal preferences and experiences of the primary contact. Distribution variously consisted of email lists, posts to specific professional societies (such as the British Geriatrics Society) and posts on social networks (Twitter and LinkedIn). The survey invitation included a link to the online survey and study information including purpose, expected duration (10 min), voluntariness of participation, confidentiality of responses and contact details of the principal investigator. To increase response rates, in each participating country a reminder was sent two weeks after the initial invitation.

2.4. Data Collection

A digital survey was designed based on the experiences of experts in eHealth in geriatric rehabilitation and the results of our previous systematic review on eHealth in geriatric rehabilitation [4]. We designed the first draft in Dutch and piloted it in a national study within the Netherlands. The first draft consisted of a total of 24 questions, four of which were open-ended to obtain detailed information. To improve accuracy and reliability of data analysis, the results of these open-ended questions were indexed and converted into multiple-choice questions. The second draft was then translated into English and sent to our primary contacts in each country for feedback. Based on their suggestions for improvement, the survey was revised with the goal of ensuring an adequate balance between the existing and the revised or new questions. The main changes entailed the phrasing of the questions and questions related to specific eHealth interventions. In the final version of the survey, questions could only be answered by participants who had experience with that type of eHealth intervention. The final survey was then translated into six languages (Czech, German, Portuguese, Romanian, Russian and Spanish) by the primary contact person in the corresponding country. The online survey was hosted by Castor Electronic Data Capture (Castor EDC; Castor, Amsterdam, The Netherlands) [21], a secure, cloud-based electronic data capture platform. The survey had a maximum of 10 questions per page, all of which were mandatory. If a respondent failed to complete a particular question, they were asked to complete it before they moved on to the next section. Respondents could review and edit answers at any time during completion of the survey. No personal information was collected and no participant IP addresses were stored or downloaded.

2.5. Measures

The survey was divided into six sections: participant characteristics, use of eHealth, benefits, usability, feasibility and the needs of professionals regarding eHealth in geriatric rehabilitation. Questions in the sections regarding benefits, usability and feasibility only became visible to respondents who indicated that they had used eHealth during their treatments. Respondents who indicated that they had used specific types of eHealth interventions were asked about their experience regarding benefits and usability for each type of eHealth. The final survey consisted of 33 questions. All questions were structured and were multiple-choice or scale questions. The scale questions were formulated as follows: For each type of eHealth intervention, respondents were asked to rate the ease of use for the professional and the patient based on a 5-point Likert scale (1 = very complex to 5 = very easy). Respondents were asked how satisfied they were with the implementation of eHealth in their institution based on a 100-point scale (0 = very dissatisfied to 100 = very satisfied). Lastly, respondents were asked to rate their institution’s vision regarding the use of eHealth on a 100-point scale (0 = inadequate to 100 = good).

2.6. Statistical Analyses

Descriptive statistics and frequency distributions were used to describe the single-choice and multiple-choice questions. A Pearson product–moment correlation was run to determine the relationship between satisfaction with the implementation of eHealth and the vision of the use of eHealth in the corresponding institution. A one sample t-test was run to determine the difference between mean scores for ease of use of all types of eHealth interventions. A heatmap was created for results related to the benefits of eHealth, with results classified and color-coded from red (0%) to green (100%). Surveys less than 90% complete were excluded from the final data analysis. Data were analyzed with SPSS version 25.0.

2.7. Ethical Considerations

This study was approved by the Medical Ethics Review Committee of Leiden–Den Haag–Delft (N20.126.1) and approved by the relevant ethics committee in participating countries as per local requirements. All participants signed the informed e-consent by clicking a dedicated button available in the invitation link, with which they stated that they were aware that participation was voluntary.

3. Results

Overall, the survey was initialized 794 times, with 513 (65%) participants completing 90% or more of the survey questions. Participant characteristics are presented in Table 1. The majority were from Europe (439 of 513 = 86%), of whom most were from The Netherlands (248 of 513 = 48%) or the Czech Republic (52 of 513 = 10%). The median age of participants was 39 years (IQR 32–49), the median number of years of work experience within geriatric rehabilitation was 8 (IQR 4–15) and 64% (329 of 513) of the respondents were female. Participants mostly worked as physiotherapists (163 of 513 = 33%), medical practitioners/geriatricians (107 of 513 = 22%) or as nurses (82 of 513 = 17%).

3.1. Use of eHealth

Results for the use of eHealth are presented in Table 2. Just over half of the respondents (263 of 513 = 51%) reported using eHealth during their treatments. Of the participants with experience in eHealth during their treatments, only a small proportion (20%) used eHealth daily or almost daily. Overall, only a small percentage of the total number of participants included in this study used eHealth daily or almost daily (52 of 513 = 10%). We also found wide variation between countries in terms of experience with eHealth (ranging from 35% to 94%) and the daily use of eHealth (ranging from 2% to 56%). Of the 263 participants with experience in eHealth, a substantial number had used simple interventions such as mobile apps (153 of 263 = 58%) and video consultations with patients (140 of 263 = 53%). More complex eHealth interventions, such as robotics (42 of 263 = 16%) or virtual reality (36 of 263 = 14%) were used far less often. A little less than half of the participants who responded to questions concerning training in the use of eHealth (78 of 160 = 49%) had received some form of training.

3.2. Benefits

The benefits experienced per form of eHealth are described in Table 3. Most participants who had experience with specific types of eHealth indicated that virtual reality (20 of 26 = 77%), exergames (29 of 50 = 73%) and robotics (28 of 36 = 78%) improved the rehabilitation environment. These participants also felt that virtual reality (23 of 26 = 64%), exergames (28 of 50 = 70%) and robotics (28 of 36 = 78%) increased patients’ self-management. Almost all participants who had used video consultation for contact with patients (61 of 68 = 90%) indicated that it was beneficial for remote care.
Regarding specific patient benefits, most participants with experience in robotics (23 of 36 = 64%) indicated that it helped a faster recovery. Of the participants with experience in video consultations, 68% (46 of 68) indicated that it contributed to increasing the frequency of treatment. Similarly, of the participants with experience in exergames, 62% (25 of 40) stated that it increased patients’ confidence, while 62% (29 of 47) of participants with experience in health sensors perceived increased self-direction amongst patients during treatment. Participants also indicated that virtual reality (19 of 26 = 79%), exergames (36 of 40 = 90%) and robotics (25 of 36 = 69%) offered the patient a more entertaining form of therapy.

3.3. Feasibility

Outcomes concerning feasibility are presented in Table 4. Participants who had previously used eHealth were asked about problems they may have encountered during regular use. The most frequently reported problems were (1) insufficient available resources (89 of 136 = 65%), (2) no organization-wide method of working or implementation (69 of 136 = 51%) and (3) costs (58 of 136 = 43%). Participants reported certain risks associated with using eHealth, such as technical problems (105 of 136 = 77%), no supervision (58 of 136 = 43%) and concerns regarding the replacement of physical contact (57 of 136 = 42%). When asked to rate the implementation process of eHealth within their department, participants reported low satisfaction with the implementation of eHealth within their settings (median 40, IQR 4.0–63), while only 11% (15 of 136) indicated they were very satisfied (range 75–100).

3.4. Usability

The ease of use per type of eHealth intervention is displayed in Figure 1. According to healthcare professionals, patients found virtual reality (median 3, IQR 2–4) and robotics (median 3, IQR 3–4) the most easy-to-use eHealth interventions. For professionals themselves, video consultations (median 4, IQR 3–5) and virtual reality (median 4, IQR 3–5) were the most easy-to-use forms of eHealth. Mobile apps were felt to be the most complex type to use by patients (median 3, IQR 2–4) and as the second most complex by professionals (median 3, IQR 2–3), with professionals rating robotics as the most complex form (median 3, IQR 3–4). With the exception of robotics (p = 0.208), professionals found all types of eHealth interventions significantly easier to use compared with patients (p ≤ 0.01).

3.5. Needs

Enabling factors and barriers to the use or implementation of eHealth are presented in Figure 2 and Figure 3. Results for the analysis of professional needs are described in Table 5. The majority of participants indicated that the availability of technical resources (362 of 513 = 71%), digital support during use (278 of 513 = 54%), enthusiasm among colleagues/employers (268 of 513 = 52%) and ease of use (258 of 513 = 50%) were enabling factors that influenced the use or implementation of eHealth. By contrast, lack of knowledge (288 of 513 = 56%), inadequate tailoring to the older population in geriatric rehabilitation (276 of 513 = 54%) and financial issues (268 of 513 = 52%) were considered barriers to the use or implementation of eHealth.
According to participants, the most important information was related to the types of eHealth available (381 of 513 = 74%), the application or implementation of eHealth (355 of 513 = 69%) and the benefits of eHealth (311 of 513 = 61%). Fifty-eight percent (297 of 513) of participants indicated they would like to increase their use of eHealth. The odds of participants who had experience with eHealth considering making more use of eHealth was 3.135 (95% CI 2.19, 4.47) times compared to participants who did not have any experience with eHealth. Participants rated their institution’s vision regarding the use of eHealth as inadequate (median: 25, IQR 3–50) and only 8% (22 of 265) held a positive view of their institution’s vision (range 75–100) concerning the use of eHealth. There was a strong correlation between satisfaction with the implementation of eHealth and a clear institutional vision regarding the use of eHealth (r = 0.716, p ≤ 0.01).

4. Discussion

4.1. Principal Findings

This international survey provided an overview of the use, benefits, feasibility, usability and needs of healthcare professionals regarding eHealth in geriatric rehabilitation. The survey included 513 professionals working in geriatric rehabilitation facilities across 16 countries. This large study is the first regarding eHealth in this setting. First, while over half of all participating healthcare professionals had experience of eHealth in clinical practice, only a tiny percentage (20%) integrated eHealth into their daily practice. Second, an institution-wide strategy for the use and implementation of eHealth (that includes topics such as the availability of technical resources, digital support and training) is an important enabling factor for the successful use and implementation of eHealth. Third, according to healthcare professionals, patients find eHealth complex to use, especially patients with cognitive impairment. Finally, there is a considerable need among professionals for more information concerning available and effective eHealth interventions, together with how they can be best applied and implemented.

4.2. Comparison with Prior Work

Overall, the healthcare professionals involved in this study reported a low daily use of eHealth interventions. To the best of our knowledge (based on literature available in English), this is the first study investigating the use of eHealth in geriatric rehabilitation. Previous studies that examined the use of eHealth in other healthcare settings such as home care [22], inpatient rehabilitation centers [23,24], or primary care [25,26,27] reported moderate (43%) to low (13%) use of eHealth. However, the reported use of eHealth varied greatly, dependent on the publication date, type of eHealth and countries included. Furthermore, the frequency of use was not reported. Variation between countries can potentially be explained by factors ranging from the healthcare professional’s personal characteristics such as attitudes toward digital technology, personal experience with eHealth interventions, trust in eHealth interventions or demographics [26,27,28], to regional factors such as readiness of a healthcare system, as well as policy and cultural differences [29].
Although daily use of eHealth was low, most respondents expressed a willingness to increase their use. Our results identified several barriers to increasing the structured use of eHealth in geriatric rehabilitation, including insufficient availability of resources, the lack of an organization-wide implementation strategy and a lack of knowledge. Similar barriers, including a limited knowledge of eHealth, lack of resources and the lack of integration into the daily workflow, have been reported in earlier studies [13,30,31]. Conversely, an institution-wide strategy that includes topics such as the availability of technical resources and digital support for the use of eHealth are important enabling factors that increase the structured use of eHealth. Earlier reviews that examined barriers and facilitators influencing the implementation and integration of eHealth arrived at similar conclusions [13,22,30,32,33], with regularly identified facilitators including ease of use, leadership engagement and adaptability of eHealth [13,30,31]. Although these earlier studies were not conducted in a geriatric rehabilitation setting, the commonality of results suggests that barriers and facilitators influencing implementation and integration of eHealth are likely generalizable across different healthcare settings. However, these studies also noted that barriers are dynamic and likely to change over time [30]. Finally, while the COVID-19 pandemic accelerated the use of eHealth, healthcare systems still face challenges when attempting to adopt eHealth, primarily due to difficulty in adjusting workflows and a funding system geared to delivering face-to-face care [10].
According to the professionals participating in our study, their patients find eHealth complex to use, although this varied considerably depending on the eHealth intervention and professionals might underestimate patients’ capabilities. Age-related impairments such as the cognitive, physical and visual limitations that are common in older adults can greatly influence one’s ability to effectively use eHealth interventions [34,35,36,37]. Furthermore, in a recent review, we identified studies with exclusion rates of up to 80%, with cognitive impairment as the most commonly reported reason for exclusion [4]. This is in line with our present findings, since adequate cognitive functioning as well as adequate vision, hearing or speech are all frequently reported requirements if patients are to make effective use of eHealth. Of the available eHealth interventions, patients find mobile apps the most complex type according to healthcare professionals. Although mobile apps are usually widely available and easily downloadable from an app store, few apps have been developed using a co-creation process or fewer still are sufficiently tailored to the age-related impairments of an older adult receiving geriatric rehabilitation [33,38]. Finally, while healthcare professionals viewed eHealth as complex for patients, the complexity for healthcare professionals should not be underestimated. Ease of use is the most frequently cited factor underlying successful use of eHealth by healthcare professionals, making it a key prerequisite for the implementation and integration of eHealth [13,22,30].
Our results also indicated that the majority of respondents are willing to make greater use of eHealth. However, it should be noted that acceptance of eHealth by healthcare professionals may differ in daily practice, since previous studies have found a limited acceptance of eHealth [17,23,39]. Acceptance is often based on prior experience, added value and social support for eHealth from colleagues [22,40,41]. Barriers can be overcome with continuing education for healthcare professionals, a modernized education of healthcare students that includes eHealth awareness, as well as co-creation and behavior change techniques that should be part of any implementation strategy [23,28].
In the survey, respondents indicated a need for reliable information on types of available eHealth interventions, how they might be applied and the benefits they may have. These findings support existing literature which stresses the urgent need to provide healthcare providers with information on both effective and ineffective eHealth applications, as well as those that might suit their local context [32,42]. Due to a rapidly changing landscape of eHealth applications, in which eHealth interventions are constantly added, updated or deleted, it is difficult for professionals to remain up to date, to determine which eHealth interventions are easy to use for older adults and to understand the assessed criteria. Our findings on benefits and usability indicate which types of eHealth interventions are easier to use or are suitable, for example, for improving the rehabilitation environment or increasing patients’ self-management. Nevertheless, we do not provide a comprehensive overview. Assessment frameworks of eHealth interventions, such as the CEN-ISO/TS 82304-2 can keep pace with the development of eHealth interventions and may help healthcare professionals obtain the information necessary for informed decision making [43,44].

4.3. Strengths and Limitations

An important strength of this study was the process of survey creation, which was comprehensive and had both valuable and executable aspects, improving the accuracy and reliability of the data analysis. Another strength of the survey was the inclusion of 513 respondents from 16 countries. This provided a good overview of the use and experiences of healthcare professionals regarding eHealth in geriatric rehabilitation. Nonetheless, some limitations of the study should be mentioned. While the study provided a broad view across a range of countries, the number of participants per country varied considerably and the majority of participants were from countries within Europe, in particular from The Netherlands and Czech Republic. This inevitably led to less reliable data for those countries with fewer respondents. Furthermore, due to the iterative development of the survey, some questions were only visible to participants outside The Netherlands, making comparisons between countries difficult. Therefore, while our paper presents the trends observed in data collected from 16 countries, our conclusions do not necessarily apply to all the countries cited in this paper. Lastly, the focus of this study was on the perspective of healthcare professionals. Future studies with a larger focus on the perspective of older adults receiving geriatric rehabilitation are needed to explore this key stakeholder’s voice.

5. Conclusions

Our primary conclusions are (1) eHealth is not yet sufficiently integrated in geriatric rehabilitation, (2) an institution-wide strategy that addresses context-specific barriers and facilitators is critical for the successful use and implementation of eHealth, (3) eHealth interventions that are simple, tailored and preferably developed through a co-creation process are essential, especially for older adults who suffer from cognitive impairment and (4) there is an urgent need to support healthcare providers by offering training and information on how to identify, assess and use eHealth, as well as how to evaluate implementation. Future studies on this topic should focus more on greater geographic diversity, including the views and attitudes of older adults receiving geriatric rehabilitation in various contexts, as well as take account of individual characteristics such as attitudes towards eHealth, gender, ethnicity, education and social network. These studies are preferably conducted using qualitative methods, such as in-depth interviews or focus groups. Furthermore, as assessment frameworks such as the CEN-ISO/TS 82304-2 are more widely adopted, it is advisable that these frameworks are tailored to geriatric rehabilitation via a greater emphasis on usability and specific age-related limitations.

Author Contributions

J.J.M.K., A.P. and E.F.V.D.v.I. conceptualized the study. J.J.M.K. prepared the original draft, conducted the analysis and the visualization of data. All authors reviewed the draft and provided feedback. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was approved by the Medical Ethics Review Committee of Leiden–Den Haag–Delft. This study did not fall under Medical Research Involving Human Subjects legislation.

Informed Consent Statement

Informed consent was obtained from all participants included in this study.

Data Availability Statement

The data presented in this study are available upon reasonable request from the corresponding author. Requests will be judged based on the originality of the research question and the feasibility of the analysis plan.

Acknowledgments

The authors thank the professionals who gave their time to complete the surveys for this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Becker, C.; Achterberg, W. Quo vadis geriatric rehabilitation? Age Ageing 2022, 51, afac040. [Google Scholar] [CrossRef] [PubMed]
  2. Grund, S.; Gordon, A.L.; van Balen, R.; Bachmann, S.; Cherubini, A.; Landi, F.; Stuck, A.E.; Becker, C.; Achterberg, W.P.; Bauer, J.M.; et al. European consensus on core principles and future priorities for geriatric rehabilitation: Consensus statement. Eur. Geriatr. Med. 2020, 11, 233–238. [Google Scholar] [CrossRef] [Green Version]
  3. van Lettow, B.; Wouters, M.; Sinnige, J. E-Health, Wat Is Dat? Nictiz: Den Haag, The Netherlands, 2019. [Google Scholar]
  4. Kraaijkamp, J.J.M.; van Dam van Isselt, E.F.; Persoon, A.; Versluis, A.; Chavannes, N.H.; Achterberg, W.P. eHealth in Geriatric Rehabilitation: Systematic Review of Effectiveness, Feasibility, and Usability. J. Med. Internet Res. 2021, 23, e24015. [Google Scholar] [CrossRef]
  5. Pol, M.C.; ter Riet, G.; van Hartingsveldt, M.; Kröse, B.; Buurman, B.M. Effectiveness of sensor monitoring in a rehabilitation programme for older patients after hip fracture: A three-arm stepped wedge randomised trial. Age Ageing 2019, 48, 650–657. [Google Scholar] [CrossRef]
  6. Bernocchi, P.; Giordano, A.; Pintavalle, G.; Galli, T.; Spoglia, E.B.; Baratti, D.; Scalvini, S. Feasibility and clinical efficacy of a multidisciplinary home-telehealth program to prevent falls in older adults: A randomized controlled trial. J. Am. Med. Dir. Assoc. 2019, 20, 340–346. [Google Scholar] [CrossRef] [PubMed]
  7. Solis-Navarro, L.; Gismero, A.; Fernández-Jané, C.; Torres-Castro, R. Effectiveness of home-based exercise delivered by digital health in older adults: A systematic review and meta-analysis. Age Ageing 2022, 51, afac243. [Google Scholar] [CrossRef]
  8. Grund, S.; Gordon, A.L.; Bauer, J.M.; Achterberg, W.P.; Schols, J. COVID-19 Pandemic and Consecutive Changes in Geriatric Rehabilitation Structures and Processes—A Deeper Attempt to Explain the COVID Rehabilitation Paradox (Lessons to Learn to Ensure High Quality of Care in GR Services). J. Nutr. Health Aging 2022, 26, 64–66. [Google Scholar] [CrossRef]
  9. Grund, S.; Gordon, A.L.; Bauer, J.M.; Achterberg, W.P.; Schols, J. The COVID rehabilitation paradox: Why we need to protect and develop geriatric rehabilitation services in the face of the pandemic. Age Ageing 2021, 50, 605–607. [Google Scholar] [CrossRef] [PubMed]
  10. Golinelli, D.; Boetto, E.; Carullo, G.; Nuzzolese, A.G.; Landini, M.P.; Fantini, M.P. Adoption of Digital Technologies in Health Care During the COVID-19 Pandemic: Systematic Review of Early Scientific Literature. J. Med. Internet Res. 2020, 22, e22280. [Google Scholar] [CrossRef]
  11. Ossebaard, H.C.; De Bruijn, A.; van Gemert-Pijnen, J.E.; Geertsma, R. Risks Related to the Use of eHealth Technologies: An Exploratory Study; National Institute for Public Health and the Environment: Bilthoven, The Netherlands, 2013.
  12. Glasgow, R.E.; Phillips, S.M.; Sanchez, M.A. Implementation science approaches for integrating eHealth research into practice and policy. Int. J. Med. Inform. 2014, 83, e1–e11. [Google Scholar] [CrossRef]
  13. Ross, J.; Stevenson, F.; Lau, R.; Murray, E. Factors that influence the implementation of e-health: A systematic review of systematic reviews (an update). Implement. Sci. 2016, 11, 146. [Google Scholar] [CrossRef] [Green Version]
  14. Swinkels, I.C.S.; Huygens, M.W.J.; Schoenmakers, T.M.; Oude Nijeweme-D’Hollosy, W.; van Velsen, L.; Vermeulen, J.; Schoone-Harmsen, M.; Jansen, Y.J.; van Schayck, O.C.; Friele, R.; et al. Lessons Learned From a Living Lab on the Broad Adoption of eHealth in Primary Health Care. J. Med. Internet Res. 2018, 20, e83. [Google Scholar] [CrossRef]
  15. van Gemert-Pijnen, J.E.; Nijland, N.; van Limburg, M.; Ossebaard, H.C.; Kelders, S.M.; Eysenbach, G.; Seydel, E.R. A holistic framework to improve the uptake and impact of eHealth technologies. J. Med. Internet Res. 2011, 13, e111. [Google Scholar] [CrossRef] [PubMed]
  16. Rauwerdink, A.; Chavannes, N.H.; Schijven, M.P. Needed: Evidence Based EHealth! Clin. Ehealth 2019, 2, 1–2. [Google Scholar] [CrossRef]
  17. Safi, S.; Thiessen, T. Acceptance and Resistance of New Digital Technologies in Medicine: Qualitative Study. JMIR Res. Protoc. 2018, 7, e11072. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Eysenbach, G. Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J. Med. Internet Res. 2004, 6, e34. [Google Scholar] [CrossRef]
  19. van Balen, R.; Gordon, A.L.; Schols, J.M.G.A.; Drewes, Y.M.; Achterberg, W.P. What is geriatric rehabilitation and how should it be organized? A Delphi study aimed at reaching European consensus. Eur. Geriatr. Med. 2019, 10, 977–987. [Google Scholar] [CrossRef] [Green Version]
  20. Grund, S.; van Wijngaarden, J.P.; Gordon, A.L.; Schols, J.M.G.A.; Bauer, J.M. EuGMS survey on structures of geriatric rehabilitation across Europe. Eur. Geriatr. Med. 2020, 11, 217–232. [Google Scholar] [CrossRef] [Green Version]
  21. Castor EDC. Available online: https://www.castoredc.com/ (accessed on 8 March 2023).
  22. Mathijssen, E.; de Lange, W.; Bleijenberg, N.; van Houwelingen, T.; Jaarsma, T.; Trappenburg, J.; Westland, H. Factors That Influence the Use of eHealth in Home Care: Scoping Review and Cross-sectional Survey. J. Med. Internet Res. 2023, 25, e41768. [Google Scholar] [CrossRef]
  23. Hennemann, S.; Beutel, M.E.; Zwerenz, R. Ready for eHealth? Health professionals’ acceptance and adoption of eHealth interventions in inpatient routine care. J. Health Commun. 2017, 22, 274–284. [Google Scholar] [CrossRef]
  24. Oesch, P.; Kool, J.; Fernandez-Luque, L.; Brox, E.; Evertsen, G.; Civit, A.; Hilfiker, R.; Bachmann, S. Exergames versus self-regulated exercises with instruction leaflets to improve adherence during geriatric rehabilitation: A randomized controlled trial. BMC Geriatr. 2017, 17, 77. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Stroetmann, K.A.; Artmann, J.; Stroetmann, V.N.; Protti, D.; Dumortier, J.; Giest, S.; Walossek, U.; Whitehouse, D. European Countries on Their Journey towards National eHealth Infrastructures; Office for Official Publications of the European Communities: Luxembourg, 2011. [Google Scholar]
  26. European Commission; Directorate-General for Communications Networks, Content and Technology; Valverde-Albacete, J.; Folkvord, F.; Lupiáñez-Villanueva, F.; Hocking, L.; Devaux, A.; Altenhofer, M.; Harshfield, A. Benchmarking Deployment of eHealth among General Practitioners (2018): Final Report; Publications Office: Luxembourg, 2019. [Google Scholar]
  27. Ortega Egea, J.M.; González, M.V.R.; Menéndez, M.R. eHealth usage patterns of European general practitioners: A five-year (2002–2007) comparative study. Int. J. Med. Inform. 2010, 79, 539–553. [Google Scholar] [CrossRef] [PubMed]
  28. de Grood, C.; Raissi, A.; Kwon, Y.; Santana, M.J. Adoption of e-health technology by physicians: A scoping review. J. Multidiscip. Healthc. 2016, 9, 335–344. [Google Scholar] [CrossRef] [Green Version]
  29. Currie, W.L.; Seddon, J.J.M. A cross-national analysis of eHealth in the European Union: Some policy and research directions. Inf. Manag. 2014, 51, 783–797. [Google Scholar] [CrossRef]
  30. Schreiweis, B.; Pobiruchin, M.; Strotbaum, V.; Suleder, J.; Wiesner, M.; Bergh, B. Barriers and Facilitators to the Implementation of eHealth Services: Systematic Literature Analysis. J. Med. Internet Res. 2019, 21, e14197. [Google Scholar] [CrossRef] [Green Version]
  31. Tossaint-Schoenmakers, R.; Versluis, A. The Challenge of Integrating eHealth Into Health Care: Systematic Literature Review of the Donabedian Model of Structure, Process, and Outcome. J. Med. Internet Res. 2021, 23, e27180. [Google Scholar] [CrossRef]
  32. Versluis, A.; van Luenen, S.; Meijer, E.; Honkoop, P.J.; Pinnock, H.; Mohr, D.C.; Neves, A.L.; Chavannes, N.H.; van der Kleij, R.M.J.J. SERIES: eHealth in primary care. Part 4: Addressing the challenges of implementation. Eur. J. Gen. Pract. 2020, 26, 140–145. [Google Scholar] [CrossRef]
  33. Morey, S.A.; Stuck, R.E.; Chong, A.W.; Barg-Walkow, L.H.; Mitzner, T.L.; Rogers, W.A. Mobile health apps: Improving usability for older adult users. Ergon. Des. 2019, 27, 4–13. [Google Scholar] [CrossRef]
  34. Broekhuis, M.; van Velsen, L.; ter Stal, S.; Weldink, J.; Tabak, M. Why My Grandfather Finds Difficulty in using Ehealth: Differences in Usability Evaluations between Older Age Groups. In Proceedings of the ICT4AWE, Crete, Greece, 2–4 May 2019; pp. 48–57. [Google Scholar]
  35. Sousa, V.E.C.; Dunn Lopez, K. Towards Usable E-Health. A Systematic Review of Usability Questionnaires. Appl. Clin. Inform. 2017, 8, 470–490. [Google Scholar] [CrossRef] [Green Version]
  36. Wildenbos, G.A.; Peute, L.; Jaspers, M. Aging barriers influencing mobile health usability for older adults: A literature based framework (MOLD-US). Int. J. Med. Inf. 2018, 114, 66–75. [Google Scholar] [CrossRef] [PubMed]
  37. Alley, S.J.; Samra, P.; Rebar, A.L.; Schoeppe, S.; Parkinson, L.; Power, D.; van Uffelen, J.G.Z.; Schneiders, A.; Vandelanotte, C. A focus group study of older adults’ perceptions and preferences towards web-based physical activity interventions. Inform. Health Soc. Care 2020, 45, 273–281. [Google Scholar] [CrossRef]
  38. Demiris, G.; Finkelstein, S.M.; Speedie, S.M. Considerations for the design of a Web-based clinical monitoring and educational system for elderly patients. J. Am. Med. Inform. Assoc. 2001, 8, 468–472. [Google Scholar] [CrossRef] [Green Version]
  39. Gun, S.Y.; Titov, N.; Andrews, G. Acceptability of Internet Treatment of Anxiety and Depression. Australas. Psychiatry 2011, 19, 259–264. [Google Scholar] [CrossRef] [PubMed]
  40. Li, J.; Talaei-Khoei, A.; Seale, H.; Ray, P.; MacIntyre, C.R. Health Care Provider Adoption of eHealth: Systematic Literature Review. Interact. J. Med. Res. 2013, 2, e7. [Google Scholar] [CrossRef]
  41. Liu, L.; Miguel Cruz, A.; Rios Rincon, A.; Buttar, V.; Ranson, Q.; Goertzen, D. What factors determine therapists’ acceptance of new technologies for rehabilitation—A study using the Unified Theory of Acceptance and Use of Technology (UTAUT). Disabil. Rehabil. 2015, 37, 447–455. [Google Scholar] [CrossRef] [PubMed]
  42. Morton, E.; Torous, J.; Murray, G.; Michalak, E.E. Using apps for bipolar disorder—An online survey of healthcare provider perspectives and practices. J. Psychiatr. Res. 2021, 137, 22–28. [Google Scholar] [CrossRef] [PubMed]
  43. Bonten, T.N.; Rauwerdink, A.; Wyatt, J.C.; Kasteleyn, M.J.; Witkamp, L.; Riper, H.; van Gemert-Pijnen, L.J.; Cresswell, K.; Sheikh, A.; Schijven, M.P.; et al. Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study. J. Med. Internet Res. 2020, 22, e17774. [Google Scholar] [CrossRef] [PubMed]
  44. Hoogendoorn, P.; Versluis, A.; van Kampen, S.; McCay, C.; Leahy, M.; Bijlsma, M.; Bonacina, S.; Bonten, T.; Bonthuis, M.-J.; Butterlin, A.; et al. What Makes a Quality Health App—Developing a Global Research-Based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2: Delphi Study. JMIR Form. Res. 2023, 7, e43905. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Usability: ease of use per form of eHealth, as perceived by professionals. Distribution of the ease-of-use scales per form of eHealth, ranging from very complex to very easy.
Figure 1. Usability: ease of use per form of eHealth, as perceived by professionals. Distribution of the ease-of-use scales per form of eHealth, ranging from very complex to very easy.
Jcm 12 04504 g001
Figure 2. Enabling factors that influence the use or implementation of eHealth (n = 513).
Figure 2. Enabling factors that influence the use or implementation of eHealth (n = 513).
Jcm 12 04504 g002
Figure 3. Barriers that influence the use or implementation of eHealth (n = 513).
Figure 3. Barriers that influence the use or implementation of eHealth (n = 513).
Jcm 12 04504 g003
Table 1. Sociodemographic and professional characteristics of participants (n = 513).
Table 1. Sociodemographic and professional characteristics of participants (n = 513).
n (%)
Sex
Female329 (64)
Male178 (35)
Prefer not to say5 (1)
Age
18–29100 (20)
30–39158 (31)
40–49130 (25)
50–5983 (16)
>6042 (8)
Profession
Physiotherapist163 (33)
Medical practitioner/geriatrician107 (22)
Nurse82 (17)
Occupational therapist61 (13)
Speech therapist29 (6)
Other 74 (15)
Working years
0–5171 (33)
6 to 15218 (43)
16 to 2592 (18)
>25 32 (6)
Continent
Europe (including the United Kingdom and Ireland)439 (86)
Asia50 (10)
North and South America32 (6)
Oceania10 (2)
Type of rehabilitation facility
Post-acute rehabilitation facility 342 (67)
Acute hospital45 (9)
Ambulatory (home based)39 (8)
Geriatric day hospital38 (7)
Other49 (10)
Experience with eHealth during treatments?
Yes263 (51)
No250 (49)
Profession, other: Nurse practitioner physician assistant, medical practitioner in training, psychologist, dietician, manager/team leader, researcher, social worker. Type of rehabilitation facility, other: Nursing home, skilled nursing facility, rehab hotel.
Table 2. Frequency of the use of eHealth (n = 263).
Table 2. Frequency of the use of eHealth (n = 263).
n (%)
Applied types of eHealth interventions
Mobile apps153 (58)
Video consultation with patients140 (53)
Health sensors101 (38)
Exergames101 (38)
Robotics 42 (16)
Virtual reality36 (14)
Frequency of use
Incidental92 (35)
Weekly65 (25)
Few times a month54 (21)
Daily or almost daily52 (20)
eHealth part of a rehabilitation program
Yes, right now143 (51)
Yes, in the past36 (13)
No91 (33)
I don’t know8 (3)
Training received in the use of eHealth (n =160)
No, I’ve only read the included manual52 (33)
No29 (18)
Yes, I received training on how to use eHealth40 (25)
Yes, I received training on the implementation of eHealth22 (14)
Yes, I received training on how to tailor eHealth to age-related barriers (e.g., cognitive or physical disabilities)16 (10)
I don’t know1 (1)
Table 3. Heatmap of benefits per form of eHealth.
Table 3. Heatmap of benefits per form of eHealth.
Mobile Apps
n = 75
Health Sensors
n = 47
Virtual Reality
n = 26
Exergames
n = 40
Video Consultation
n = 68
Robotics
n = 36
Types of benefits experienced
Ease of use24 (32%)18 (38%)6 (23%)10 (25%)26 (38%)9 (25%)
Better quality treatment36 (47%)23 (49%)6 (23%)10 (25%)30 (44%)7 (9%)
Improvement of the rehabilitation environment23 (30%)17 (36%)20 (77%)29 (73%)18 (26%)28 (78%)
Increasing self-management of the patient21 (28%)14 (30%)18 (69%)28 (70%)8 (12%)23 (64%)
Possibility of remote care 43 (57%)19 (50%)6 (23%)7 (18%)61 (90%)2 (18%)
Efficient deployment of staff37 (49%)23 (49%)9 (35%)17 (43%)30 (44%)12 (33%)
Types of benefits for the patient
Faster recovery14 (19%)13 (28%)13 (54%)18 (45%)10 (15%)23 (64%)
Increase in treatment frequency39 (52%)15 (32%)12 (50%)18 (45%)46 (68%)13 (36%)
More confidence38 (51%)26 (55%)12 (50%)25 (62%)25 (37%)11 (31%)
More self-direction40 (53%)29 (62%)10 (42%)17 (42%)32 (47%)11 (31%)
More fun form of therapy30 (40%)15 (32%)19 (79%)36 (90%)15 (22%)25 (69%)
Types of disadvantages encountered
It does not meet the needs of the patient well28 (39%)14 (30%)3 (16%)12 (31%)26 (40%)10 (29%)
Applications crash or do not work properly22 (31%)11 (24%)4 (21%)10 (26%)25 (39%)6 (18%)
Difficult to use or apply30 (42%)18 (39%)3 (16%)16 (31%)19 (29%)17 (50%)
None16 (22%)14 (30%)10 (53%)13 (33%)16 (25%)12 (35%)
Color code:0% 100%
Table 4. Results related to feasibility of eHealth (n = 136).
Table 4. Results related to feasibility of eHealth (n = 136).
n (%)
Problems encountered in structural use of eHealth
Insufficient available resources89 (65)
No organization-wide method of working/implementation69 (51)
Costs58 (43)
Shortage of professional knowledge55 (40)
Lack of time47 (34)
Space shortage37 (27)
Inappropriate target group26 (19)
Lack of motivation20 (15)
Other (adherence, accessibility, lack of effort)10 (7)
No problems5 (4)
Patient‘s skills to use eHealth
Sufficient cognitive functioning112 (82)
No problems with vision, hearing or speech77 (57)
Supervision from caregiver/family72 (53)
Motivation67 (49)
Independence57 (42)
Digital literacy56 (41)
Sufficient motor functioning24 (18)
Risks of eHealth
Technical problems105 (77)
No supervision58 (43)
Concerns regarding replacement of physical contact57 (42)
Distress/confusion in patients53 (39)
Difficult to implement52 (38)
Reduction in quality of care39 (29)
Privacy sensitive31 (23)
Discomfort (i.e., lower back pain, lower limb pain)22 (16)
Other (no risks, solitude, digital literacy, poor performance exercises)10 (3)
Table 5. Needs of professionals regarding the use of eHealth (n = 513).
Table 5. Needs of professionals regarding the use of eHealth (n = 513).
n (%)
Information needs concerning eHealth
Which types of eHealth exist381 (74)
Applying or implementing eHealth355 (69)
The benefits of eHealth311 (61)
The operation of eHealth applications260 (51)
I don’t have any information needs23 (4)
I don’t know13 (3)
Other (training)6 (1)
How to receive information about eHealth (n = 265)
Digital course140 (53)
Course on location137 (52)
Webinar129 (49)
Written (article, information letter, manual)100 (38)
Fact sheet76 (29)
No preference18 (7)
Would you like to make more use of eHealth?
Yes297 (58)
Maybe160 (31)
I don’t know41 (8)
No15 (3)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kraaijkamp, J.J.M.; Persoon, A.; Aurelian, S.; Bachmann, S.; Cameron, I.D.; Choukou, M.-A.; Dockery, F.; Eruslanova, K.; Gordon, A.L.; Grund, S.; et al. eHealth in Geriatric Rehabilitation: An International Survey of the Experiences and Needs of Healthcare Professionals. J. Clin. Med. 2023, 12, 4504. https://doi.org/10.3390/jcm12134504

AMA Style

Kraaijkamp JJM, Persoon A, Aurelian S, Bachmann S, Cameron ID, Choukou M-A, Dockery F, Eruslanova K, Gordon AL, Grund S, et al. eHealth in Geriatric Rehabilitation: An International Survey of the Experiences and Needs of Healthcare Professionals. Journal of Clinical Medicine. 2023; 12(13):4504. https://doi.org/10.3390/jcm12134504

Chicago/Turabian Style

Kraaijkamp, Jules J. M., Anke Persoon, Sorina Aurelian, Stefan Bachmann, Ian D. Cameron, Mohamed-Amine Choukou, Frances Dockery, Kseniia Eruslanova, Adam L. Gordon, Stefan Grund, and et al. 2023. "eHealth in Geriatric Rehabilitation: An International Survey of the Experiences and Needs of Healthcare Professionals" Journal of Clinical Medicine 12, no. 13: 4504. https://doi.org/10.3390/jcm12134504

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop