Design and Validation of an Instrument for Technical Performance Indicators of the Kick ( Chagi ) Technique in Taekwondo

: Taekwondo is a martial art and combat sport that originated in medieval or similar cultures, but today it is primarily a combat activity conditioned by safety rules. The kick technique is called “ chagi ” (Korean language). “Eyeballing” is a problem that is being addressed with traditional training methods used in Taekwondo. (1) Background: To solve this problem, the main aim of this study was to develop an Observation System for Technical Performance Indicators-Chagi (OSTPI-C). The validation and reliability processes were carried out by 19 expert judges, who were required to meet at least four out of the ﬁve selected inclusion criteria. (2) Methods: The content validity was calculated using Aiken’s V coefﬁcient value on qualitative (relevance, univocity, and importance) and quantitative levels (rating scale from one to ﬁve), through the agreement and consensus of the panel experts. The reliability of the instrument was analyzed using Cronbach’s alpha coefﬁcient. (3) Results: The results demonstrated the high content validity indexes (0.90) and high reliability (alpha > 0.70) of the observational instrument. (4) Conclusions: The OSTIP-C observational instrument could be used in the Olympic and Paralympic Taekwondo sport environment as a valid and reliable tool to evaluate the technical execution of kick technical processes.


Introduction
Taekwondo is defined as self-defense, a weaponless martial art of Korean origin, and a modern combat sport [1][2][3]. This martial art and combat sport originated in medieval or similar cultures, and today it is primarily a combat activity conditioned by safety rules.
Athletes can perform in the upper zone when they are in the presence of advanced and experienced athletes [15]. Thus, [16] concluded that elite athletes successfully change and adjust their foot path earlier than athletes in the sub-elite category. Another researcher [17] identified four phases: start (A), toe off (B), maximum knee flexion (C), and impact (D), and three moments: push, lift, and strike.
They concluded that higher kicks required increased vertical right and left arm (elbow and wrist) separation in the release phase. The kicks were broken into four phases. The preparation phase was defined as occurring from the toe down of the support leg until the toe off of the kicking leg. The chamber phase started at the end of the preparation phase and continued until the beginning of knee extension in the kicking leg. The extension phase followed and continued until contact with the target pad. The final phase was the recoil phase; it commenced immediately after contact with the kicking pad had ended and terminated with heel down of the kicking leg [18,19]. All top athletes adapt their technique according to their morphological, physiological, and mechanical characteristics [20][21][22]. In competition, hand and foot techniques provide contact between opponents, and points are scored each time they come into contact with their opponent in the permitted area. The target and permitted zones are the trunk (protected by a shield) and head (protected by a helmet). For the trunk target zone, foot and hand techniques are allowed, whereas for the helmet target zone only foot techniques are permitted [23].
The results indicated that the winners and losers scored one and three points similarly, but they used different patterns prior to, and after scoring. To score, the winners used direct attacks to the chest [24].
Taekwondo athletes possess low levels of body fat and have a somatotype that is characterized by a blend of moderate musculoskeletal tissue and relative body linearity, with a somewhat variable VO 2max . As a consequence, moderate to high levels of cardiorespiratory fitness are necessary to support the metabolic demands of fighting [25]. Taekwondo practice may increase selective attention in adolescents and could be considered an appropriate no pharmacological therapeutic method to combat/counteract the attention impairment of individuals with attention deficit hyperactivity disorder [26].
In Taekwondo, as for any combat sport, the sporting outcome depends largely on the perfection of technical skills. This study aimed to build and validate an observational instrument relating to the process of technical execution of the kick. The tool was designed to establish the effectiveness and identify the critical moment of the Taekwondo kick technical process. The overall goal was broken down into three specific pillars: (1) to design and improve an observational instrument for the identification of relevant aspects in the Taekwondo kick process; (2) to validate a tool that would help to determine the critical and successful moments in the technical process of the taekwondo kick, and (3) to assess the validity and reliability of this observational instrument.
The observational instrument was given the name of Observation System for Technical Performance Indicators-Chagi (OSTPI-C) and took the form of a questionnaire for presentation to prospective experts.

Materials and Methods
The observation instrument in this study focused on chagi (taekwondo kick skill process) through the motor performance of the athletes. Thus, it was essential to use ad hoc instruments with sufficient flexibility to adapt to the flow of conduct and context during observation [27,28].

Design and Observation Unit
This work is classified as an instrumental study because it promotes the development of new procedures, apparatus, instruments, or tests, as well as their psychometric properties [29]. The purpose of the observational tool was to obtain empirical knowledge [30] regarding the Taekwondo kick technique, as well as critical and/or successful moments. The observation unit tool for the validation of the OSTPI-C was created following an in-depth review of previous studies of the process of training expert subjects using the biographical method and interviews for data gathering [31].

Dataset
To gather the quality control data, the investigators sent an email questionnaire using a Google form, Appendix A, to 52 prospective experts. Of the 52 questionnaires, 28 were returned duly completed. Finally, 19 expert judges (the experts' panel) carried out quality control of the sampled data. All 19 experts were chosen in accordance with the inclusion criteria. The research report needed to include a detailed description of the participants, in particular, their number and representativeness compared to the wider population [32].

Participants
Nineteen Taekwondo expert coaches participated (n = 19) in this study. Deliberate and intentional sampling procedures were conducted to recruit the participants [32]. The number of judges required in a trial depends on levels of expertise and diversity of knowledge [33].
The subjects selected to be part of the expert panel (Committee of Experts' Criteria-CEC) had to meet 4 of the 5 inclusion criteria that were established. These were: CEC1: to hold a Ph.D. degree/be or have been a university professor (sport sciences); CEC2: to have participated as a coach in a Europe/World Taekwondo Championships; CEC3: to hold an International Taekwondo Coach Certificate; CEC4: to hold a National Taekwondo Coach Certificate; CEC5: to have 10 or more years of experience in Taekwondo Olympic sport.
The following Table 1 presents the criteria met by each member of the expert panel. One member met all five criteria established by the investigators. Legend: E = expert; CEC = committee of experts' criteria; = meets inclusion criteria.

Study Variables
To validate an observational tool, the use of expert judgment arises from the need to estimate the validity of the content of a test. For this, it is necessary to collect information in a systematic way, defining the objective and criteria of the expert jury [34].
There were three sections of contextual data and six categories of variables that related to the examination of the validity of the instrument and its external reliability (generalizability).
An evaluation sheet was used for data collection, including all information required for the study, as this was the optimal means for establishing the data for evaluation [35]. The evaluation sheet used a quantitative format that included a five-point Likert-type scale with a response choice of: 1, not adequate; 2, poor; 3, moderately adequate; 4, good; or 5, totally adequate, for all surveys based on relevance (adaptation to the objectives to be evaluated), univocity (clarity in the wording of the questions), and importance (relevance, interest or significance of the questions).
The panel of experts was also allowed to offer their own ideas in the form of qualitative suggestions or perceptions. These qualitative data provided additional insights and perspectives relevant to the study [30]. Cronbach's alpha value for scale reliability was used to determine the internal consistency of the instrument. Reliability refers to the reproducibility of an instrument [36].

Instrument
In order to advance and refine the OSTIP-C observational instrument for the evaluation of moments and phases in the process of kick technical execution, it was essential to examine the object of study further in relation to performance in the sport [30].
The first part of the instrument was used to collect information about the biographic and demographic details (age, gender, education) and background (academic degree, experience, and coach at national and international level) of each expert panel member. These variables were used as performance indicators in the analysis, [37]. Each variable received a numerical categorization to facilitate subsequent statistical analysis.
The OSTIP-C was designed to identify the diverse variables that affect critical moments and successive phases in the process of technical execution of the kick. It enables understanding of the different phases, provision of a detailed assessment, and identification of pertinent signs during the athlete's movements in the performance of the kick.
The observational instrument was divided into three sections of contextual data and six moments (consisting of the variables V1, V2, V3, V4, V5, and V6). Each variable refers to the performance indicators of the kick technique in the sport of Taekwondo. At each observation moment, all variables were analyzed for relevance, univocity, and importance. Instrument reliability was also analyzed. To refine each variable as well as the categorical object analysis, the procedure proposed by [38] was followed.
The structure of the questionnaire (Table 2) allowed the experts to request a qualitative evaluation of any item to identify possible alternatives where they deemed necessary [31,39].

Element
Purpose of Questions Section 1 Biographical and sporting aspects. Section 1 concerned biographical and sporting aspects. In Section 2, we defined the following three phases and six moments of contextual data, which were classified, as follows: Phase 1: take-off the foot and knee lift; Phase 2: start of leg extension and contact moment, and Phase 3: start of leg flexion and thigh extension. This section consisted of more specific questions about the phases under analysis ( Table 3) that influenced the intended analysis and observation process. In Section 3, the expert could add any observations that they deemed appropriate for improvement of the instrument.

Materials
An online survey tool (Google form) was sent by e-mail and returned by the expert evaluators. To calculate the values of Aiken's V, the free Visual Basic 6.0 language program developed by [40] was used. This enables the calculation of confidence intervals via an overall method score [41] at 90%, 95%, and 99% confidence intervals. Microsoft Excel was used to analyze the data collected from the Google form. Finally, the SPSS 21.0 Premium + Amos statistical package was used for the evaluation of the reliability of the OSTIP-C observational instrument.

Procedure
To start the process, an exploratory literature review was conducted regarding both the main aim of this study, i.e., the technical performance indicators of kick skills (chagi), in Taekwondo, and procedures for the construction, development, and validation of observational instruments.
The cover letter for the study was sent by e-mail to potential experts together with the Google form questionnaire. The letter was structured as follows. Firstly, an introduction with the investigator's name and his academic background. Then, an explanation setting out that we were conducting a study on the construction and validation of an ad hoc observational instrument for Taekwondo, namely, "Observation System for Technical Performance Indicators-Chagi (OSTPI-C)", its purpose, contextual data from the collaborating experts, and the time that would be needed to complete the questionnaire (about 10 min). Secondly, recipients were asked to answer a series of questions to gather relevant information about the potential experts. Finally, a statement that all information provided would be treated with complete confidentiality, with protection of any personal data.
The expert panel member evaluations were obtained using the Google form application to create a survey. The respondents carried out their evaluation and returned it to the issuer. To finalize the survey process, the data were statistically analyzed.

Statistical Analysis
To validate the OSTIP-C observational instrument with the input of the specialists/experts, the Aiken V value was used for content validity [42]. This coefficient is one of the principal means to quantify and validate the content or relevancy of each item relative to the content domain for N judgments (number of expert judges). Aiken's V coefficient value ranges from 0 to 1, with 1 (highest value) being perfect agreement among experts in relation to the validity of the content evaluated [43]. To obtain the content validity coefficient, the following algebraic equation, as modified by [41], was used: In the equation presented above, X represents the mean of the scores provided by the experts in the sample, l is the lowest score obtained, and k represents the range of possible values on the 5-point Likert-type rating scale that was used. Sentence deleted.
For the calculation of this coefficient and to test whether the magnitude of the coefficient was greater than the minimum established level, the free program Visual Basic 6.0 [40] was used. This program generates a range of values (maximum score-minimum score) and Aiken's V value as well as the confidence interval values at 90%, 95%, and 99% which are the usual estimated confidence levels. In this investigation, a 95% confidence level was used to correspond with the baseline level of acceptance for researchers in the social sciences [40]. The exact critical reference value for the acceptance of Aiken's V coefficient value given the number of judges and the range of responses for each item was 0.69; p = 0.041. The number of items was 0.75; p = 0.006 [42].
Aiken's V [43] and Cronbach's alpha coefficients were used to evaluate the reliability of the OSTIP-C observational instrument. The internal consistency assessment ranges from 0 to 1 and serves to determine whether the instrument has obtained response patterns that are conceptually consistent within the items comprising the scale [43]. A Cronbach's alpha value of around 0.70 is generally acceptable, values exceeding 0.80 are desirable, and values around 1.0 represent the strongest evidence. However, the relative strength of agreement associated with kappa statistics indicates that the reliability of the observational instrument is almost perfect if the value falls between 0.81-1.00 [44][45][46][47]. The relative strength of agreement associated with kappa statistics created six divisions to corresponding ranges of kappa: <0.00 (poor); 0.00-0.20 (slight); 0.21-0.40 (fair); 0.41-0.60 (moderate); 0.61-0.80 (substantial); and 0.81-1.00 (almost perfect) [48].

Results
The results are presented in the same order in which the study was designed and in which the OSTPI-C validation was carried out. The following Table 4 presents the relevance of the variables according to the Aiken V coefficient values, with confidence intervals for each of the six variables. The following Table 5 sets out the results for the univocity of variables according to Aiken's V coefficient values, with confidence intervals for each of the six variables. The following Table 6 presents the results for the importance of variables according to the Aiken's V coefficient values, with confidence intervals for each of the six variables. The Table 7 below presents the results of the qualitative evaluations of the variables performed by each expert, based on their own opinion. Table 7. Qualitative evaluations provided by the expert panel.

Experts
Expert Panel Own Opinion 1 V1: The right standing start position, and extension of the foot moment are the key to high performance in the kick process. V2: A high and good position of the knee moment is essential for the techniques. V3: At the beginning of the extension of the contact leg moment it is important to hit the target. V4: The contact moment depends on all other matters/phases. V5: The start leg flexion moment is crucial and defines the correct technique. V6: The tight extension moment is important, but it is necessary to do it with power. 8 V2: It is not only better to see but the quality of the knee will be preserved longer.

15
V1: This kind of investigation is good to improve technique progression. V2: I think it is important to support leg-foot rotation for knee joint protection. V3: A good learning experience in the extension of the knee will be important at the time of being able to carry out a technique with a positive goal. V4: It is very important to have knowledge about contact actions, as this knowledge will be of vital importance for athletes to achieve greater performance in their attacks. V6: It is important that the muscle always works in its maximum extension when this is with controlled workloads that do not damage its structures.

16
V1: Very important to know the foot action. V2: It is very important to know the knee lift to a good target. V4: This is the goal of the kick. V6: Important because of the balance of the athlete's body.

18
V1: It is very relevant to know the foot extension in this phase.
The next Table 8 presents the results of the reliability analysis for the OSTIP-C observational instrument which covers the different phases, relevant actions, and evaluation of the kick movement. The reliability statistics for the OSTIP-C tool, by grouped content, indicated very high levels using Cronbach's alpha, with N of items 18, and a value of 0.901. No error percentage was found, and the results ensured convergent probability based on the criteria presented by [48] whereby the value found in the OSTIP-C tool regarding the nomenclature associated with kappa statistics (0.81-1.00) indicated that the relative strength of agreement was "Almost Perfect".
The OSTIP-C observation instrument (Table 9) is presented below with all its analysis phases, variables, descriptions, and frame instants of observation (Figure 1). This tool was designed to characterize the observation moments for analysis of the technical performance cycle (behavioral technical events of the athletes). Table 9. OSTIP-C Observation Instrument.

Discussion
The main aim of this study was to design and validate an observational tool, namely, OSTIP-C, to identify and establish the effectiveness of critical and successful moments in the technical Taekwondo kick process. The methodology and procedures suggested in various literature sources were taken into account [34,38,49,50]. The previous construction, design, and validation of similar instruments for use in the sports context, were also reviewed [39,[51][52][53].
The first step was to build the observational instrument with the help of a small expert group of individual collaborators. The second step involved a validation process using a larger group of expert panel members. To create this larger group, the investigators sent an email questionnaire using a Google form to 52 prospective judges. Of these, 28 were returned duly completed. At the end of this process, after applying inclusion criteria, the expert panel consisted of 19 expert judges. In this study, each expert panelist provided a quantitative and qualitative evaluation of the items and suggested potential improvements to the OSTIP-C.
There are some issues regarding the composition of the expert panel and the methods used to assess and report the experts' ratings which merit discussion [54]. Because the value of scientific data depends on precision and measurement, the investigators decided to use expert knowledge to build a sophisticated observational instrument for the

Discussion
The main aim of this study was to design and validate an observational tool, namely, OSTIP-C, to identify and establish the effectiveness of critical and successful moments in the technical Taekwondo kick process. The methodology and procedures suggested in various literature sources were taken into account [34,38,49,50]. The previous construction, design, and validation of similar instruments for use in the sports context, were also reviewed [39,[51][52][53].
The first step was to build the observational instrument with the help of a small expert group of individual collaborators. The second step involved a validation process using a larger group of expert panel members. To create this larger group, the investigators sent an email questionnaire using a Google form to 52 prospective judges. Of these, 28 were returned duly completed. At the end of this process, after applying inclusion criteria, the expert panel consisted of 19 expert judges. In this study, each expert panelist provided a quantitative and qualitative evaluation of the items and suggested potential improvements to the OSTIP-C.
There are some issues regarding the composition of the expert panel and the methods used to assess and report the experts' ratings which merit discussion [54]. Because the value of scientific data depends on precision and measurement, the investigators decided to use expert knowledge to build a sophisticated observational instrument for the purpose of measurement [42]. The item content-relevance of this study was established using an expert panel [50]. Validation by experts is thus a form of content validity, and this was demonstrated by asking experts to review the content of the instrument [55]. For the validation process to be satisfactory it was necessary to use a minimum number of expert judges to ensure consistency of the responses to each of the items in the observational instrument. The number of expert judges that participated in the study was appropriate and valid, providing relevance to the study.
The high level of expertise of the panel expert members reviewing OSTIP-C was guaranteed because each judge met four of the five inclusion criteria adopted for this study. All the expert members who participated in this study had been coaches in a European/World Taekwondo championship, held the National and International Taekwondo coach certificate, and had 10 or more years of experience in the Taekwondo Olympic sport. Six of them held Ph.D. degrees and were or had been university professors in sport sciences. The expert panel members were capable of providing knowledge regarding the topic of the study, such as providing reflexive assessment and insights that could enable the researchers to further reflect on the topic [34]. The quality of the judges enhanced the quality of the tool. Taekwondo is a minority physical sports activity, which has few resources. Additionally, it is difficult to find observers who can be classified as experts in the field, despite being a successful Olympic sport.
Previous researchers [56] conducted a study to design and validate a tool to identify the characteristics of the strokes that are used in the sport of Padel with the help of eleven expert judges (n = 11) who met four out of five inclusion criteria. Another study [57] used a sample of participants composed of thirteen experts (n = 13), nine of whom were experts in football and six of whom were experts in basketball and handball. A group of thirteen experts (n = 13) participated in a study conducted by [44] to design and validate the basketball learning and performance assessment instrument (BALPAI). Another study worked with fourteen expert judges (n = 14) to evaluate the performance of basketball referees [58]. An instrument validation process used to analyze sportive formation in volleyball was carried out using the expert judgment method involving fifteen judges (n = 15) [49]. One of the main reasons for employing such a large number of expert judges (n = 19) in our study, was to highlight the important influence that differences in the characteristics of judges can have on the evaluation process [54]. The expert panel members that participated in our study were 19 expert Taekwondo coaches. Nineteen was an acceptable number and exceeds the classic minimum of ten identified in the literature. The number of expert judges presented in similar studies varies, ranging from 3 to 36; however, these all contributed to an acceptable content validity for the corresponding observational instrument that was designed, built, and subsequently validated [47,50,59,60]. The number of expert judges imparts a quantitative value to the tool.
Researchers must give careful consideration to (a) the characteristics of potential expert judges; (b) the number of judges comprising the expert panel; (c) the procedures used by judges to rate item content-relevance (or item importance or item representativeness); (d) the quantitative or statistical procedures used to assess judges' ratings, and (e) the selection criteria used to determine whether items are selected, modified, or deleted from the final item set to be included in the instrument [54].
This study employed a five-point Likert-type evaluation scale to assess item contentrelevance during the construction of the instrument. The stem for each item was "on Taekwondo kick technical process" and responses were indicated on a five-point Likert scale. In accordance with [61], an item was retained if there was 100% agreement among the judges (to its classification and if there was a mean of 4.0 or greater) on the rating of the quality of the item. All items met these criteria. After rating each item, the expert judges were encouraged to provide comments regarding the wording or content of the items [54]. The evaluators' own opinions, reflections, and personal views regarding the variables were considered indispensable for the continued refinement of the observation instrument [55,62]. In this study, we benefitted from some additional qualitative reinforcement contributions from the experts.
The quantitative evaluation of the items that comprised the observational instrument for the performance indicators of the kick technique in Taekwondo consisted of the variables V1, V2, V3, V4, V5, and V6. All the variables were analyzed for relevance, univocity, and importance at each observation moment, as well as reliability. To refine each variable as well as the categorical object analysis a procedure proposed by [38] was followed. The qualitative contributions of the expert judges served to improve the tool, making it more comprehensive and eliminating doubts that might arise in its application.
The Cronbach's alpha value index obtained for the OSTIP-C observational tool was 0.90. The reliability value is adequate for the assumption within a group, with acceptable levels of 0.70 usually set as a minimum standard although it is typically advisable to have values that exceed 0.80 [47]. None of the observational instrument items had Aiken's V coefficient values lower than the critical levels for acceptance, Aiken's 0.69; p= 0.041. The number of expert judges was 0.75; p = 0.006 [42]. The Aiken's V coefficient values, through confidence intervals for each of the six variables, presented the relevance of variables found in the ranges of confidence intervals. The results obtained were acceptable and meet the requirements for these types of social science studies.
The minimum level of Aiken's V coefficient can be considered acceptable from a value of 0.50 [42,43]. This result could be useful for the validation of observation instruments. Other authors propound Aiken's V coefficient value of 0.70 as more suitable for social science studies [40,63]. The result obtained for the OSTIP-C exceeded the minimum levels proposed by these experts. The result for this study is in line with the standard set out by [39], in that the content validity indices were obtained with a confidence level of 95%.
The results obtained for all the items meant that it was not necessary to modify or revise or eliminate items that did not achieve a sufficiently high Aiken's V coefficient value that was established as an exclusion criterion [31,59,64]. The prior design of the tool by the group of expert judges proved to be adequate, showing a deep knowledge of the sport, which resulted in the items ratified by the experts. In addition, a few qualitative contributions were analyzed. Some of them were merely the reflections of the expert judges but others allowed some of the opening ranges to be defined more precisely.
The reliability of each variable in the content validation process (level of adequacy and clarity) was assessed and the values exceeded 0.70. Observational tools can be considered to reach an appropriate validation standard when this value is obtained [30]. In this study, the Cronbach's alpha value obtained for the observational instrument (OSTIP-C) was 0.90. Therefore, the OSTIP-C can be considered to be a valid observational tool.
Many studies examine performance indicators in relation to scoring indices, such as goals, baskets, winners, shots, points, corners, etc., or in relation to the quality of game performance, such as moves, tackles, possession, passes, etc. [65]. Few studies analyze aspects related to the development, improvement, and performance of the individual technique of each athlete during the training process. The action technique has a basic execution model in its cinematic structure which is adapted in each combat situation [8] and the cinematic structure is divided into three phases: preparation, initial, and final phases.
Further research is required regarding the difference between novice and expert athletes. Therefore, in future reviews, articles that evaluate the observed physical and mechanical performance of the Taekwondo kick technique may be included.

Conclusions
The OSTIP-C observational instrument could be used in the Olympic and Paralympic Taekwondo Sport environment as a valid and reliable tool for evaluation of the technical execution of the Taekwondo kick process.
In the creation of this observational instrument: (a) each judge met four of the five inclusion criteria; (b) the number of expert panel members contributed to the high reliability of the found results, and the 'n' sample was highest value found (n = 19) in comparison to other studies, and (c) the results obtained a high confidence intervals percentage (relevance, univocity, and importance = 95%) and a high value for reliability (Cronbach's alpha = 0.90). The assessment of the observational instrument as well as the expertise contributed via comments improved the core content on aspects related to the technical process of the Taekwondo kick. It is therefore considered that the instrument is adequate, and its validity and reliability are acceptable for the measurement of Taekwondo athletes' kick technical process in a sporting context.
In a training camp environment, the information that can be obtained using the OSTIP-C with video analysis, allows coaches, athletes, and researchers to consistently evaluate critical phases and success moments in the Taekwondo kick technical process.
One limitation of this study may be the reduced number of phases and success moments analyzed during the technical process of the Taekwondo kick.

Institutional Review Board Statement:
The study was conducted according to the Declaration of Helsinki and approved by the Ethics Committee of the University of Extremadura (67/2017).

Informed Consent Statement:
Informed consent was obtained from all subjects involved in the study. The researchers sent the study instrument to fifty-two prospective judges, inviting them to respond voluntarily as experts. Twenty-eight were returned duly completed. Nineteen expert judges met the inclusion criteria, and thus participated in the study.

Acknowledgments:
The authors thank all participants who agreed to take part in this study during the data collection procedure, and special thanks are sent to the panel judges.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
The Google form questionnaire sent to prospective experts: