Development of a Set of Assessment Tools for Health Professionals to Design a Tailored Rehabilitation Exercise and Sports Program for People with Stroke in South Korea: A Delphi Study

We developed a set of assessment tools for health professionals to evaluate the physical functions, mental functions, and social abilities of people with stroke (PWS) from 6 months to 3 years after stroke onset, to design a tailored “Rehabilitation Exercise and Sports” (RES) program, which the South Korean government was required to provide by the Act on Guarantee of Right to Health and Access to Health Services for people with disabilities. Since previous studies mainly dealt with the chronic stage of PWS, it would not be appropriate to apply assessment tools used in previous studies, as they are not compatible with the time window (6 months to 3 years) used to define the target population of the RES program. We reviewed the literature to identify evaluation factors and assessment tools applied in previous studies, and developed a Delphi questionnaire with closed-ended questions based on the literature review’s results and supplementary open-ended questions. A 20-expert panel conducted four rounds of the Delphi survey, including two rounds to determine evaluation factors and two rounds to determine assessment tools. The Delphi survey revealed that 22 evaluation factors and 24 corresponding assessment tools reached consensus among the experts. However, no assessment tools reached consensus for three evaluation factors: muscle endurance, flexibility, and dynamic balance. A comprehensive set of assessment tools would be useful for health professionals to understand the health status of PWS from 6 months to 3 years after stroke onset, and help the design of tailored RES programs.


Introduction
South Korea is expected to face an ultra-aged society, in which more than 20% of the total population will be 65 years or older by 2025.Along with this high aging rate, the population of patients with stroke (PWS) is also growing, causing medical costs to be approximately six times higher than those of the general population [1].Stroke also represents the third most prevalent type of disability [2,3].Furthermore, more than 25% of people with a history of stroke are at an increased risk of recurrence [4].Stroke recurrence is influenced by a number of metabolic risk factors, such as hypertension, impaired glucose control, dyslipidemia, obesity, and low cardiorespiratory fitness [5,6].
Exercise is a safe, inexpensive, and effective method for reducing metabolic risk factors in PWS with minimal side effects [7,8].Moreover, exercise improves not only physical function, such as strength, endurance, balance, and walking performance [8][9][10][11][12][13], but also psychological functions in PWS [14].However, the participation rate of non-disabled people in exercise is 60.8%, whereas the rate for people with disabilities (PWD) is only 24.2% [15].Furthermore, well-equipped infrastructure and services for rehabilitation or exercise in the community is lacking, eventually forcing patients to be readmitted to another hospital or to be left alone at home after discharge [16].
In South Korea, to address the issue of health inequality between people with and without disabilities, the Act on Guarantee of Right to Health and Access to Health Services for PWD (PWD Health Rights Act) has been in effect since 2017 [17].In particular, Article 15 stipulates that the government should strive to improve the physical and mental functions, and social abilities of PWD or those expected to become PWD within a certain period of time after an injury or disease by providing them with "Rehabilitation Excise and Sports" (RES) programs [17]."A certain period of time" is usually interpreted as approximately between 6 months and 3 years after disease or injury onset.Therefore, this regulation aims to help patients discharged from acute hospitals settle into society early and encourage them to enjoy sports upon their return.
As suggested by its name, RES is an intermediate step between "therapeutic exercise" provided in hospitals and "sports for all" performed in the community.It should be possible to simultaneously achieve the goals of "recovery of function" and "enhancement of physical fitness" for PWD.Furthermore, the improvement of mental function and social ability is also included in the purpose of the RES program, as stipulated in the PWD Health Rights Act.PWS are expected to have wide variations in physical function levels, depending on the stroke type and severity.There are also differences in mental function and social ability, depending on individual personalities and the socioecological environment to which each person belongs.Therefore, to achieve the goals of the RES program for PWS, it is important to design interventions tailored to each individual or groups of individuals with similar types, taking into account the physical, mental, and social abilities of each subject based on the biopsychosocial approach.A prerequisite for adopting this approach is to accurately understand the health status of PWS during this period and provide health professionals with the necessary information to design tailored interventions.
Therefore, this study aims to develop a set of assessment tools necessary for confirming the physical and mental functions, and social abilities of PWS from 6 months to 3 years after stroke onset, constituting the target population of the RES program.In particular, we used the Delphi method to collect expert opinions on the development of this set of assessment tools.

Study Design
This study adopted the Delphi method to develop a set of assessment tools to help health professionals evaluate the health status of PWS and design a tailored RES program.The Delphi method is usually anonymous, iterative, and survey-based.This method is designed to transform opinions into a group consensus, with each person having an equal voice in the decision-making process [45].This method has been used to enhance effective decision-making in health and social care [45].

Development of a Set of Assessment Tools
The hierarchical structure for evaluation factors and a set of assessment tools corresponding to each evaluation factor were developed through the following four steps (Figure 1).

Study Design
This study adopted the Delphi method to develop a set of assessment tools to help health professionals evaluate the health status of PWS and design a tailored RES program.The Delphi method is usually anonymous, iterative, and survey-based.This method is designed to transform opinions into a group consensus, with each person having an equal voice in the decision-making process [45].This method has been used to enhance effective decision-making in health and social care [45].

Development of a Set of Assessment Tools
The hierarchical structure for evaluation factors and a set of assessment tools corresponding to each evaluation factor were developed through the following four steps (Figure 1).and "Stroke" in the article title, on Scopus, CINAHL, and Web of Science.A total of 98 articles were identified.Second, among the identified articles, only 26 studies  that applied exercise or sports interventions to PWS for a period of time, on an individual-or group-basis in community settings were selected.The researchers tried to identify the functions targeted for improvement by the authors by applying exercise or sports (e.g., muscle strength and flexibility), and also tried to identify the tools applied to assess these functions before and/or after intervention.In the current study, our research team defined the former as an "evaluation factor" and the latter as an "assessment tool".In this step, the researchers identified 14 evaluation factors and 58 assessment tools among the 26 selected studies.Third, among the identified assessment tools, the researchers determined a total of 47 tools whose reliability or validity had been verified in previous studies.The researchers developed a draft version of Delphi items based on the determined 14 evaluation factors and 47 evaluation tools.and "Stroke" in the article title, on Scopus, CINAHL, and Web of Science.A total of 98 articles were identified.Second, among the identified articles, only 26 studies  that applied exercise or sports interventions to PWS for a period of time, on an individualor group-basis in community settings were selected.The researchers tried to identify the functions targeted for improvement by the authors by applying exercise or sports (e.g., muscle strength and flexibility), and also tried to identify the tools applied to assess these functions before and/or after intervention.In the current study, our research team defined the former as an "evaluation factor" and the latter as an "assessment tool".In this step, the researchers identified 14 evaluation factors and 58 assessment tools among the 26 selected studies.Third, among the identified assessment tools, the researchers determined a total of 47 tools whose reliability or validity had been verified in previous studies.The researchers developed a draft version of Delphi items based on the determined 14 evaluation factors and 47 evaluation tools.

Step 2. Building a Draft Version of the Hierarchical Structure for Evaluation Factors and Determining Corresponding Assessment Tools for Each Evaluation Factor
Before the initiation of the Delphi process, an expert steering committee was convened to build a draft version of a hierarchical structure for evaluation factors and determine corresponding assessment tools for each evaluation factor.This step was intended to systematically classify the evaluation factors and assessment tools by grouping factors and tools with similar purposes.Steering committee members were selected from those who met one of the following criteria: (1) >10 years of exercise intervention experience for community-dwelling PWS, or (2) >10 years of teaching experience as a professor in related fields such as rehabilitation medicine, sports medicine, special physical education, kinesiology, and physical therapy.First, a steering committee of 10 experts listed the evaluation factors and assessment tools as broadly and inclusively as possible to ensure the generation of a comprehensive list based on possible evidence from the reviewed literature.Second, experts suggested other evaluation factors or assessment tools in addition to those reviewed, based on their expert knowledge and experience.Third, experts built a draft version of a hierarchical structure by grouping factors with similar purposes and corresponding assessment tools for each evaluation factor using the design-thinking process.The design-thinking process provided the optimal solution derived from the opinions of steering committee members through repeated deductive and conductive verification [46,47].

Step 3. Determination of Evaluation Factors
The researchers developed a questionnaire in which each evaluation factor consisted of a Delphi item, and these items were presented based on the draft version of the hierarchical structure discussed in Step 2. The questionnaire was based on a 5-point Likert scale ranging from 1 to 5 (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree).A free-text comment box for each item was included in the survey for the expert panel to suggest the addition, removal, or modification of items.A web-based platform was employed to gather input from the expert panel to maintain anonymity and avoid individual bias in consensus formation during the Delphi survey.In the current study, a Delphi survey was requested by an expert panel in two rounds [20] between September 2022 and October 2022.After completing Delphi Round 1, all text comments on the questionnaire were evaluated, modifications were made to reduce ambiguity if necessary, and new elements were added if deemed appropriate.In Delphi Round 2, all ratings from the first round were analyzed and expressed as the percentage of respondents with an item score ranging from 1 to 5 for each element.The experts who responded in the first Delphi round subsequently received the summarized results and were requested to reconsider their ratings.The final version of the assessment factors was determined based on consensus from the second round.

Step 4. Determination of Assessment Tools
The researchers developed a questionnaire in which each assessment tool consisted of a Delphi item.The Delphi items (i.e., assessment tools) were classified and presented corresponding to each evaluation factor.In each Delphi item, "importance" and "feasibility" of the assessment tool were rated based on a 5-point Likert scale, ranging from 1 to 5 (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree).The Delphi survey for assessment tools was conducted in two rounds, between November 2022 and December 2021.The detailed Delphi survey process was similar to the one used to determine evaluation factors described in Step 3.

Expert Panel Selection
Purposive and snowballing sampling strategies were employed to recruit experts among those who met one of the following criteria [48]: (1) >10 years of exercise intervention experience for community-dwelling PWS, or (2) >10 years of teaching experience as a professor in related fields, such as rehabilitation medicine, sports medicine, special physical education, kinesiology, and physical therapy.This study was premised on the stage after a doctor comprehensively evaluated the patient's disease and screened whether he or she could perform exercise considering risk factors, focusing on determining a set of assessment tools necessary to design an intervention tailored to each individual or groups of individuals with similar types.Thus, doctors were not included as a panel in the current study.A total of 20 multidisciplinary experts specializing in rehabilitation medicine (n = 3), sports medicine (n = 2), kinesiology (n = 8), special physical education (n = 4), and physical therapy (n = 3) were included.Among them, 17 experts (69%) had doctoral degrees, and the other experts had master's degrees (19%).The mean work experience in the related field was 23.45 ± 5.9 years.The response rates for the determination of assessment factors and assessment tools in rounds 1 and 2 were 100% (n = 20) and 95% (n = 19), respectively (Table 1).

Data Analysis
Analysis of responses to closed-ended questions of the Delphi survey for evaluation factors (Rounds 1 and 2) and tools (Rounds 1 and 2) was performed after the completion of each round using SPSS 21. Results of descriptive statistics were presented as mean and standard deviation.Content validity was verified using the content validity of individual items (I-CVI) and the positive coefficient of variation (CV), a relative measure of statistical dispersion.I-CVI was calculated as the number of experts giving a rating of 4 and 5 divided by the total number of experts [22,40].We used a mean ≥ 4.0, I-CVI ≥ 0.75 (75%), and CV < 0.50 [18,22] as the stop criteria.In this approach, the opinions of most experts are agreed upon if the mean, I-CVI, and CV values meet the stop criteria [18,22].During the analysis of the results of Delphi survey for the determination of assessment tools, opinions of most experts were considered "agreed upon" only when the stop criteria were satisfied in both "importance" and "feasibility" aspects.
The research team and steering committee members classified the evaluation factors based on several criteria (Figure 2).First, it was largely categorized into the following three factors, defined as the purpose of the RES program in the PWD Health Rights Act  (1) Added by the steering committee during the design-thinking process. (2)Added to reflect the results of the first Delphi survey for the determination of evaluation factors.

Delphi Survey for the Determination of Evaluation Factors
A Delphi survey was conducted with 20 Delphi items determined through literature review and design thinking.
Round 1 of the Delphi survey revealed that the expert panel agreed that all 20 evaluation factors were required for PWS to perform the RES program, satisfying the stop cri-  (1) Added by the steering committee during the design-thinking process. (2)Added to reflect the results of the first Delphi survey for the determination of evaluation factors.
Second, physical function was subcategorized into (1) health-related factors (n = 5), (2) skill-related factors (n = 2), and (3) ADL-related factors (n = 4) (Figure 2).According to previously reported guidelines [49], physical fitness is classified into health-related and skill-related factors.Health-related factors consist of components of physical fitness that are related to good health, commonly defined as body composition, cardiovascular endurance, flexibility, muscular endurance, and strength.Skill-related factors consist of physical fitness components that have a relationship with enhanced performance in sports and motor skills, commonly defined as agility, balance, coordination, power, speed, and reaction time.However, among these factors, agility, power, speed, and reaction time were considered too difficult to apply to subjects of the RES program by the steering committee members.However, since factors related to physical fineness are not sufficient to accurately evaluate the health status of PWS, the steering committee members additionally suggested ADLrelated factors to evaluate the disability-specific function of PWS.As a result, evaluation factors related to physical function were classified as detailed in Figure 2.

Delphi Survey for the Determination of Evaluation Factors
A Delphi survey was conducted with 20 Delphi items determined through literature review and design thinking.
Round 1 of the Delphi survey revealed that the expert panel agreed that all 20 evaluation factors were required for PWS to perform the RES program, satisfying the stop criteria (mean ≥ 4.0, I-CVI ≥ 0.75, and CV < 0.50).In Round 1, several experts suggested in a free text comment box that "muscle tone" needed to be added to ADL-related factors, and that balance needed to be divided into "static balance" and "dynamic balance", considering the diversity of functional levels according to the severity of disability.Therefore, in Round 2, a total of 22 items were generated to reflect these opinions, and the mean, I-CVI, and CV values of all Delphi items satisfied the stop criteria, indicating that the expert panel reached consensus on all 22 evaluation factors (Table 2).

Delphi Survey for the Determination of Assessment Tools
A Delphi survey with 53 items determined through literature review (47 items) and the design-thinking process (6 items) was conducted in Round 1.After completing Round 1, there was an opinion to add an assessment tool for muscle tone through a free text comment box; accordingly, in Round 2, a Delphi was conducted on a total of 54 Delphi items.Round 2 of the Delphi survey revealed that the expert panel agreed that 24 assessment tools among the 54 Delphi items were important and feasible for the evaluation of the health status of PWS during their performance of RES programs in community settings, meeting the stop criteria (mean ≥ 4.0, I-CVI ≥ 0.75, and CV < 0.50) (Table 3).
For assessment tools to evaluate social ability, disability acceptance (importance: mean  Among the evaluation factors, there was no assessment tool that experts agreed to be appropriate for the evaluation of muscle endurance, flexibility, and dynamic balance.In other words, while experts recognized that these functions should be evaluated, the assessment tools presented in this Delphi survey were not appropriate for the target population of the RES program.
In Rounds 1 and 2, several experts suggested in a free-text comment box that assessment tools needed to be different, depending on the level of severity of stroke or the ability to walk; step or walking tests were deemed not appropriate due to the risk of falling, some tools would be difficult to apply in community settings because they take a long time to evaluate, and self-report tests would be difficult to apply to those with cognitive impairment or hand function problems.

Discussion
In this study, we developed a set of assessment tools for health professionals to evaluate the physical, mental, and social abilities of PWS from 6 months to 3 years after stroke onset.As such, the health status of PWS during this period could be accurately understood and health professionals could be provided with the information necessary to design a tailored RES program.By developing a hierarchical structure for the evaluation factors, we examined whether the factors evaluating overall health status were included without omission.Through the Delphi survey, expert consensus was reached on a total of 22 evaluation factors and 24 assessment tools.However, experts judged that none of the evaluation tools presented in the Delphi items were appropriate for assessing muscle endurance, flexibility, and dynamic balance.
To develop the hierarchical structure, researchers and experts derived 22 evaluation factors based on several criteria.First, it was largely categorized into three areas: physical function, mental function, and social ability, as defined by the purpose of the RES program in the PWD Health Rights Act of South Korea [17].The purpose of the RES program is in line with the definition of health presented by the International Classification of Functioning, Disability, and Health (ICF) [50].The ICF consists of four areas: body and structure, activity and participation, environment, and individual factors, each of which includes several domains that correspond to the evaluation factors of this study.Since the ICF consists of a total of 362 second-level categories and the maximum number of factors for health examination, it is difficult to apply these criteria to all situations.Therefore, an ICF Coreset composed of only the categories appropriate for specific diseases and onset periods has been developed for several diseases.For neurological conditions, Coresets for acute, post-acute, and sub-acute phases have been developed and published [51].However, the Coreset for the post-acute or sub-acute phase is composed of a total of 116 second-level categories, making it difficult to apply to chronic patients in community settings.Therefore, the evaluation factors derived from the results of this study could serve as a reference for an ICF domain to be applied to community-based exercise in the chronic stage.
Second, physical functions were subcategorized into health-related, skill-related, and ADL-related factors based on previously reported guidelines [49].In particular, the inclusion of ADL-related function was based on the fact that the RES program aims to improve recovery of function as well as physical fitness in PWS within 6 months to 3 years of discharge, differentiating it from sports programs for all PWS (with an emphasis on physical fitness).Therefore, this systematic structure can play a role in guiding the design of interventions to meet the purpose of the RES defined by the PWD Health Rights Act.
Through the Delphi survey, expert opinions were reached on 22 evaluation factors and 24 assessment tools.However, it is necessary to pay attention to the expert opinion that none of the presented assessment tools were deemed appropriate as evaluation factors for muscle endurance, flexibility, and dynamic balance, which are the main assessment tools that require dynamic motion.This finding suggests that assessment tools such as (1) Step test, Step-box test, 30 s chair stand test for measuring muscle endurance, (2) Sit-andreach test, Trunk-forward-flexion test, Shoulder-flexibility test for measuring flexibility, and (3) Timed up and go, Berg balance scale, Tinetti test, Short-physical-performance battery, 4-square step test for measuring dynamic balance are not appropriate for PWS from 6 months to 3 years after stroke onset, although some of these tools were confirmed to be reliable or valid for PWS in acute or chronic stages in previous studies [52][53][54].Therefore, it is necessary to develop new assessment tools suitable for PWS at this age.In addition, it is necessary to refer to the opinions presented by experts in the open-ended questionnaire to develop new evaluation tools to evaluate muscle endurance, flexibility, and dynamic balance for PWS during this period, including (1) adjusting assessment tools based on the level of severity of the stroke or the ability to walk, and (2) recognizing that step or walking tests were not appropriate due to the risk of falling.
There are several limitations in the current study.Firstly, the expert panel for the determination of evaluation factors and assessment tools consisted only of professors and instructors; therefore, the opinions of PWS were lacking.In follow-up studies, assessment tools should be supplemented to reflect the opinions of PWS.Secondly, the reliability and validity of each assessment tool for PWS from 6 months to 3 years after stroke onset have not been verified.Follow-up studies should analyze the reliability and validity of the evaluation tools selected through this study.

Conclusions
A set of assessment tools for health professionals to design a tailored "Rehabilitation Exercise and Sports" program for PWS from 6 months to 3 years after stroke onset was developed using the Delphi survey.The survey revealed that 22 evaluation factors and 24 corresponding assessment tools reached consensus among experts, although no assessment tools reached consensus for three evaluation factors: muscle endurance, flexibility, and dynamic balance, suggesting the necessity of developing new assessment tools for these evaluation factors.Health professionals would thus be able to gain a more accurate understanding of the health status of PWS during this time, as well as be provided with the information needed to design a tailored RES program.

Figure 1 .
Figure 1.Development of a set of assessment tools through the Delphi method.2.2.1.Step 1. Literature Review First, the researchers searched for articles published in the last 10 years, from 2011 to the present, which included the combination of the words "Exercise or Sports or Muscle strength or Muscle endurance or Cardiovascular endurance or Flexibility or Body composition or Balance or Coordination or Range of motion or Upper/Lower extremity function or Activity of daily living or Cognition or Depression or Fear of falling or Quality of life"and "Stroke" in the article title, on Scopus, CINAHL, and Web of Science.A total of 98 articles were identified.Second, among the identified articles, only 26 studies that applied exercise or sports interventions to PWS for a period of time, on an individual-or group-basis in community settings were selected.The researchers tried to identify the functions targeted for improvement by the authors by applying exercise or sports (e.g., muscle strength and flexibility), and also tried to identify the tools applied to assess these functions before and/or after intervention.In the current study, our research team defined the former as an "evaluation factor" and the latter as an "assessment tool".In this step, the researchers identified 14 evaluation factors and 58 assessment tools among the 26 selected studies.Third, among the identified assessment tools, the researchers determined a total of 47 tools whose reliability or validity had been verified in previous studies.The researchers developed a draft version of Delphi items based on the determined 14 evaluation factors and 47 evaluation tools.

Figure 1 .
Figure 1.Development of a set of assessment tools through the Delphi method.2.2.1.Step 1. Literature Review First, the researchers searched for articles published in the last 10 years, from 2011 to the present, which included the combination of the words "Exercise or Sports or Muscle strength or Muscle endurance or Cardiovascular endurance or Flexibility or Body composition or Balance or Coordination or Range of motion or Upper/Lower extremity function or Activity of daily living or Cognition or Depression or Fear of falling or Quality of life"and "Stroke" in the article title, on Scopus, CINAHL, and Web of Science.A total of 98 articles were identified.Second, among the identified articles, only 26 studies that applied exercise or sports interventions to PWS for a period of time, on an individualor group-basis in community settings were selected.The researchers tried to identify the functions targeted for improvement by the authors by applying exercise or sports (e.g., muscle strength and flexibility), and also tried to identify the tools applied to assess these functions before and/or after intervention.In the current study, our research team defined the former as an "evaluation factor" and the latter as an "assessment tool".In this step, the researchers identified 14 evaluation factors and 58 assessment tools among the 26 selected studies.Third, among the identified assessment tools, the researchers determined a total of 47 tools whose reliability or validity had been verified in previous studies.The researchers developed a draft version of Delphi items based on the determined 14 evaluation factors and 47 evaluation tools.

15 Figure 2 .
Figure 2. Development of a hierarchical structure for evaluation factors.ADL, activities of daily living; IADL, instrumental activities of daily living.(1)Added by the steering committee during the design-thinking process.(2)Added to reflect the results of the first Delphi survey for the determination of evaluation factors.

Figure 2 .
Figure 2. Development of a hierarchical structure for evaluation factors.ADL, activities of daily living; IADL, instrumental activities of daily living.(1)Added by the steering committee during the design-thinking process.(2)Added to reflect the results of the first Delphi survey for the determination of evaluation factors.

Table 1 .
The composition of the expert panel who participated in the Delphi survey.

Table 2 .
Survey results for the determination of evaluation factors.
ADL, activities of daily living; CV, positive coefficient of variation; IADL, instrumental activities of daily living; I-CVI, content validity of individual items; SD, standard deviation.

Table 3 .
Survey results for the determination of evaluation factors.