Reliability of Kinovea® Software and Agreement with a Three-Dimensional Motion System for Gait Analysis in Healthy Subjects

Gait analysis is necessary to diagnose movement disorders. In order to reduce the costs of three-dimensional motion capture systems, new low-cost methods of motion analysis have been developed. The purpose of this study was to evaluate the inter- and intra-rater reliability of Kinovea® and the agreement with a three-dimensional motion system for detecting the joint angles of the hip, knee and ankle during the initial contact phase of walking. Fifty healthy subjects participated in this study. All participants were examined twice with a one-week interval between the two appointments. The motion data were recorded using the VICON Motion System® and digital video cameras. The intra-rater reliability showed a good correlation for the hip, the knee and the ankle joints (Intraclass Correlation Coefficient, ICC > 0.85) for both observers. The ICC for the inter-rater reliability was >0.90 for the hip, the knee and the ankle joints. The Bland–Altman plots showed that the magnitude of disagreement was approximately ±5° for intra-rater reliability, ±2.5° for inter-rater reliability and around ±2.5° to ±5° for Kinovea® versus Vicon®. The ICC was good for the hip, knee and ankle angles registered with Kinovea® during the initial contact of walking for both observers (intra-rater reliability) and higher for the agreement between observers (inter-rater reliability). However, the Bland–Altman plots showed disagreement between observers, measurements and systems (Kinovea® vs. three-dimensional motion system) that should be considered in the interpretation of clinical evaluations.


Introduction
Gait analysis is necessary to diagnose musculoskeletal and neurological disorders, as well as evaluation of the efficacy of different interventions performed in patients [1].
For the gait evaluation, three-dimensional motion analysis systems are considered the gold standard. They provide objective and quantitative data regarding kinematic and spatiotemporal parameters [2]. However, these systems present several disadvantages, such as the high cost of equipment, the need for trained personnel, considerable processing times and the large spaces requirement for installation.
The VICON Motion System ® (Oxford Metrics, Oxford, UK) was used to analyze the agreement with Kinovea ® . This system consists of eight 100 Hz infrared cameras, three AMTI ® force-plates, two BASLER A601FC-2 video cameras and a data station where information is recorded and processed.

Procedures
The research took place at the Motion Analysis, Biomechanics, Ergonomics, and Motor Control Laboratory (LAMBECOM), located in the Physiotherapy, Occupational Therapy, Rehabilitation, and Physical Medicine Department (Faculty of Health Sciences, Rey Juan Carlos University).
All participants were evaluated twice with a separation of one week between both appointments. To carry out the movement acquisition, passive and reflective markers were placed in specific anatomical areas of the lower limbs (anterior superior iliac spine, posterior superior iliac spine, middle third of thigh, external femoral condyle, middle third of tibia, external malleolus, calcaneus and head of second metatarsal), according to the biomechanical models of Davis et al. [14] and Kadaba et al. [15]. An additional marker was placed on the greater trochanter [8,13] (Figure 1). After the instrumentation was completed, the subjects were instructed to walk along the 11-m walkway (back and forth). They were asked to walk at a self-selected comfortable gait speed.
In order to delimit the recording area, two marks were placed on the footbridge that the subjects had to walk, at two meters between them. Two researchers were synchronized to start and stop the acquisition of motion with the digital cameras and the VICON system ® . The recording started when the participants entered the recording area and stopped when they left it. Recordings of five repetitions per subject were made in each of the sessions.

Analysis of Data
The motion capture with the digital cameras and the VICON system ® was repeated in the first session and in the second session. One researcher distributed the videos acquired for each session between the observers and then compiled the data.
For the kinematics analysis the left lower limb was assessed. The angles of the hip, knee and ankle joints at the initial contact phase were analyzed in the sagittal plane of the studied lower limb. Kinovea ® version 0.8.15 was used to analyze the videos.
Two observers selected the initial contact of the lower limb by observing the acquired videos, which was expected to occur at an intermediate distance between the two marks established on the footbridge. The two observers agreed to analyze the same event in the exact frame. The "angle" tool in Kinovea ® was used to acquire the kinematics of the hip, knee and ankle in this stage of gait. The angles calculation procedure ( Figure 2) followed for each joint is presented: After the instrumentation was completed, the subjects were instructed to walk along the 11-m walkway (back and forth). They were asked to walk at a self-selected comfortable gait speed.
In order to delimit the recording area, two marks were placed on the footbridge that the subjects had to walk, at two meters between them. Two researchers were synchronized to start and stop the acquisition of motion with the digital cameras and the VICON system ® . The recording started when the participants entered the recording area and stopped when they left it. Recordings of five repetitions per subject were made in each of the sessions.

Analysis of Data
The motion capture with the digital cameras and the VICON system ® was repeated in the first session and in the second session. One researcher distributed the videos acquired for each session between the observers and then compiled the data.
For the kinematics analysis the left lower limb was assessed. The angles of the hip, knee and ankle joints at the initial contact phase were analyzed in the sagittal plane of the studied lower limb. Kinovea ® version 0.8.15 was used to analyze the videos.
Two observers selected the initial contact of the lower limb by observing the acquired videos, which was expected to occur at an intermediate distance between the two marks established on the footbridge. The two observers agreed to analyze the same event in the exact frame. The "angle" tool in Kinovea ® was used to acquire the kinematics of the hip, knee and ankle in this stage of gait. The angles calculation procedure ( Figure 2) followed for each joint is presented: • Hip. A line is drawn through the anterior superior iliac spine and the posterior superior iliac spine. Perpendicular to this, another line is drawn that passes through the greater trochanter. The angle formed by the latter and the line joining the greater trochanter to the external femoral condyle will form the joint range of the hip. • Knee. A line is drawn between the reference points of greater trochanter and femoral condyle, and another between femoral condyle and external malleolus. The angle formed between the two lines will be used for calculating the knee joint range. In this work, 180 degrees will be considered as the neutral position of the knee. Joint range is calculated by the following equation: Knee Joint Range = 180-(angle obtained with Kinovea ® ), positive values correspond to knee flexion and negative values to extension.
• Ankle. A line is drawn that joins the markers of the head of the second metatarsal and the calcaneus. The angle formed between this and the line passing through the femoral condyle and the external malleolus is used to calculate the ankle joint range. In this work, 90 degrees will be considered as the neutral position of the ankle. Joint range is calculated by the following equation: Ankle Joint Range = 90-(angle obtained with Kinovea ® ), positive values correspond to dorsiflexion, and negative values to plantar flexion.
The initial contact with VICON was identified using a 20 N threshold on the vertical force component measured by the force plates [18]. The output angles for all joints were calculated from the YXZ cardan angles derived by comparing the relative orientations of the two segments. The course and direction of the segment axes are shown in the Vicon Plug-in Gait Product Guide [16].
Procedure and data analysis are summarized in Figure 3. • Hip. a line is drawn through the anterior superior iliac spine and the posterior superior iliac spine. Perpendicular to this, another line is drawn that passes through the greater trochanter. The angle formed by the latter and the line joining the greater trochanter to the external femoral condyle will form the joint range of the hip.

•
Knee. a line is drawn between the reference points of greater trochanter and femoral condyle, and another between femoral condyle and external malleolus. The angle formed between the two lines will be used for calculating the knee joint range. In this work, 180 degrees will be considered as the neutral position of the knee. For the processing of trials obtained with VICON Motion System ® (Oxford Metrics, Oxford, UK), Vicon Nexus ® 1.8.5 software was used [16,17].
The initial contact with VICON was identified using a 20 N threshold on the vertical force component measured by the force plates [18]. The output angles for all joints were calculated from the YXZ cardan angles derived by comparing the relative orientations of the two segments. The course and direction of the segment axes are shown in the Vicon Plug-in Gait Product Guide [16].
Procedure and data analysis are summarized in Figure 3.

Sample Size Calculation
Sample size was calculated based on Walter et al. [19]. Considering a minimally acceptable Intraclass Correlation Coefficient (ICC) (p0) of 0.6, an expected ICC (p1) of 0.8, and 10% of attrition, 43 subjects are needed. Finally, the sample size consisted of 50 subjects.

Statistical Analysis
In order to evaluate the reliability between the two different testing sessions and between the observers, the intra-class correlation coefficient (ICC) was used [20]. The ICC was estimated, and their 95% confident intervals were calculated, using the SPSS statistical package version 22 SPSS Inc., Chicago, IL, USA), based on absolute agreement and a mixed-effect model (ICC 3,1).
Bland-Altman analysis with 95% limits of agreement was performed to assess intra and interrater reliability and the agreement between Kinovea ® and VICON ® . The bias and the limits of agreement are shown in the plots for the parameter registered. The mean score is plotted on the xaxis, and the difference between observers, sessions or systems (mean of the differences) is plotted on the y-axis (mean of the difference ±1.96 SD, Standard Deviation). The width of the limits of agreement and the distance of the mean of the differences with respect to zero can be used to interpret the errors between measurements. Bland-Altman plots allow comparisons between two different measurement systems, observers or sessions when evaluating the same dataset to analyze the match

Sample Size Calculation
Sample size was calculated based on Walter et al. [19]. Considering a minimally acceptable Intraclass Correlation Coefficient (ICC) (p0) of 0.6, an expected ICC (p1) of 0.8, and 10% of attrition, 43 subjects are needed. Finally, the sample size consisted of 50 subjects.

Statistical Analysis
In order to evaluate the reliability between the two different testing sessions and between the observers, the intra-class correlation coefficient (ICC) was used [20]. The ICC was estimated, and their 95% confident intervals were calculated, using the SPSS statistical package version 22 SPSS Inc., Chicago, IL, USA), based on absolute agreement and a mixed-effect model (ICC 3,1).
Bland-Altman analysis with 95% limits of agreement was performed to assess intra and inter-rater reliability and the agreement between Kinovea ® and VICON ® . The bias and the limits of agreement are shown in the plots for the parameter registered. The mean score is plotted on the x-axis, and the difference between observers, sessions or systems (mean of the differences) is plotted on the y-axis (mean of the difference ±1.96 SD, Standard Deviation). The width of the limits of agreement and the distance of the mean of the differences with respect to zero can be used to interpret the errors between measurements. Bland-Altman plots allow comparisons between two different measurement systems, observers or sessions when evaluating the same dataset to analyze the match level [22]. Dependent sample t-tests were also used to compare the mean differences between the two systems. The statistically significant p-value was set at 0.05.

Results
The study group consisted of 50 subjects (26 women/24 men; age 21.62 ± 2.62 years; body mass 65.74 ± 12.94 kg; height 167.49 ± 25.57 cm) without alterations in gait. There were no missing data.
The intra-rater reliability showed a good correlation for the hip, the knee and the ankle joints (ICC > 0.85) for both observers ( Table 1). The mean of the differences between sessions for hip, knee and ankle angles were 0.21 and 0.17, 0.06 and 0.3, and 0.98 and 0.97 degrees, observer 1 and 2, respectively. In Bland-Altman plots, the limit of agreement for hip, knee and ankle angles was 5.44 to −5.87 and 6.52 to −6.17, 4.61 to −4.47 and 4.82 to −4.21, and 5.10 to −3.20 and 4.97 to −3.02 degrees, observer 1 and 2, respectively (Figures 4 and 5). Table 1. Intra-rater reliability of the Kinovea parameters.

Angles (Degrees)
Intra-Rater Reliability The ICC for the inter-rater reliability was >0.90 for the hip, the knee and the ankle joints in both observers ( Table 2). The mean of the differences was 0.8, 0.09 and 0.48 degrees, respectively. In Bland-Altman plots, the limit of agreement for the hip, knee and ankle angles was 1.40 to −3.05, 2.09 to −1.90, and 3.16 to −2.18 degrees, respectively ( Figure 6).    The ICC for the inter-rater reliability was >0.90 for the hip, the knee and the ankle joints in both observers ( Table 2). The mean of the differences was 0.8, 0.09 and 0.48 degrees, respectively. In Bland-Altman plots, the limit of agreement for the hip, knee and ankle angles was 1.40 to −3.05, 2.09 to −1.90, and 3.16 to −2.18 degrees, respectively ( Figure 6). for the hip ankles (a), knee angles (b) and ankle angles (c). Bias (black line) and limits of agreement (red lines) are shown for each parameter. The mean score is plotted on the x-axis, and the difference between sessions (mean of the differences) is plotted on the y-axis (mean difference ± 1.96 SD).  Gait parameters, measured by Kinovea ® and VICON ® , are shown in Table 3. There were significant differences in the average comparison angles between two systems (p < 0.05). Mean differences between systems for hip, knee and ankle angles were 0.83, 2.02 and −1.19 degrees, respectively. In Bland-Altman plots, the limit of agreement for hip, knee and ankle angles was 5.26 to −3.58, 5.01 to -0.98, and 3.70 to −6.09 degrees, respectively (Figure 7).  Gait parameters, measured by Kinovea ® and VICON ® , are shown in Table 3. There were significant differences in the average comparison angles between two systems (p < 0.05). Mean differences between systems for hip, knee and ankle angles were 0.83, 2.02 and −1.19 degrees, respectively. In Bland-Altman plots, the limit of agreement for hip, knee and ankle angles was 5.26 to −3.58, 5.01 to -0.98, and 3.70 to −6.09 degrees, respectively (Figure 7). Kinematic are expressed in mean and standard deviation. MD is the mean of the differences. CI, Confidence Interval. a p-value < 0.05 is statistically significant. The mean score is plotted on the x-axis, and the difference between systems (mean of the differences) is plotted on the y-axis (mean difference ± 1.96 SD).

Discussion
The purpose of the present study was to evaluate the intra-and inter-rater reliability of Kinovea ® and the agreement between Kinovea ® and VICON ® to obtain the joint angles during the initial contact phase of walking.
The use of systems that allow the analysis with videos, such as Kinovea ® , could provide objective and quantitative data for advanced evaluations. Furthermore, these systems could be used not only as a diagnostic tool, but also as instruments for evaluating the results after an intervention. Its easy handling, low cost and high accessibility make it an alternative for the analysis of walking when there are no more sophisticated systems such as three-dimensional analysis equipment [1].
The main limitations of the reliability studies for Kinovea ® found in literature are the lack of a standardized video analysis protocol and marker placement [8,[11][12][13]. Most of them used the greater trochanter as the preferred marker position [8,13]. The use of markers on the bone reliefs, as they are in the protocol presented in this work, is highly recommended as it contributes reliability to the calculation of joint ranges. This work presents discrepancies on joint angle calculations compared to studies found in literature. , obtained the hip articular range considering the position of the femur with respect to the vertical [8]. According to this approach, the resulting angle would correspond to the position of the thigh ignoring the pelvis position [23]. Our approach suggests that hip angle must be calculated in relation to the pelvis.
The study of the reliability of Kinovea ® software is to determine that it evaluates what is intended to measure and to be able to help clinicians and researchers to interpret the data obtained shown for each parameter. The mean score is plotted on the x-axis, and the difference between systems (mean of the differences) is plotted on the y-axis (mean difference ± 1.96 SD).

Discussion
The purpose of the present study was to evaluate the intra-and inter-rater reliability of Kinovea ® and the agreement between Kinovea ® and VICON ® to obtain the joint angles during the initial contact phase of walking.
The use of systems that allow the analysis with videos, such as Kinovea ® , could provide objective and quantitative data for advanced evaluations. Furthermore, these systems could be used not only as a diagnostic tool, but also as instruments for evaluating the results after an intervention. Its easy handling, low cost and high accessibility make it an alternative for the analysis of walking when there are no more sophisticated systems such as three-dimensional analysis equipment [1].
The main limitations of the reliability studies for Kinovea ® found in literature are the lack of a standardized video analysis protocol and marker placement [8,[11][12][13]. Most of them used the greater trochanter as the preferred marker position [8,13]. The use of markers on the bone reliefs, as they are in the protocol presented in this work, is highly recommended as it contributes reliability to the calculation of joint ranges. This work presents discrepancies on joint angle calculations compared to studies found in literature. , obtained the hip articular range considering the position of the femur with respect to the vertical [8]. According to this approach, the resulting angle would correspond to the position of the thigh ignoring the pelvis position [23]. Our approach suggests that hip angle must be calculated in relation to the pelvis.
The study of the reliability of Kinovea ® software is to determine that it evaluates what is intended to measure and to be able to help clinicians and researchers to interpret the data obtained by subjects with specific pathology [24]. In this sense, the ICC was good for the hip, knee and ankle angles for both observers (intra-rater reliability) and higher for the agreement between the observers (inter-rater reliability). However, the ICC has been criticized as it is a dimensionless value, therefore not easily interpreted. In this sense, Bland-Altman plots may be more useful than the ICC as they can be readily and easily interpreted in a meaningful way in both the research and clinical environment [25]. Specifically, the width of the limits of agreement are useful to understand the level of agreement or disagreement between observers, measurements or systems [26].
The Bland-Altman plots showed that for the most part the magnitude of disagreement was approximately ±5 • for intra-rater reliability, ±2.5 • for inter-rater reliability and around ±2.5 • to ±5 • for Kinovea ® versus Vicon ® . In relation to the measurement errors, McGindey et al. concluded that error of 2 • or less for a three-dimensional motion system is considered acceptable in a clinical situation, as such errors are probably too small to require explicit consideration during data interpretation. Errors of between 2 • and 5 • are also reasonable but may require consideration in data interpretation. In addition, the authors suggested that errors in excess of 5 • should raise concern and may be large enough to mislead clinical interpretation [27]. Therefore, the disagreement observed in the Bland-Altman plots in this study may be reasonable for a clinical evaluation. In addition, the amplitude of the limits of agreement observed for Kinovea ® are similar to those obtained for a three-dimensional movement analysis system in a test-retest reliability study. Meldrum et al. found an amplitude of ± 8 degrees to detect the position of the ankle during the initial contact phase of the gait and similar results were obtained for the hip and knee kinematic parameters (±4 • for ranges of motion and around ± 6 • to 8 • for peak kinematics in the sagittal plane) [25].
However, the results of this work should be interpreted with caution. The agreement obtained for kinovea ® is not enough to detect small changes between sessions and observers. Differences in joint position less than five degrees after an intervention may be due to system or observer error. There are numerous sources of variability within the testing procedure that could explain the differences between intra-rater and inter-rater reliability: marker placement error, processing errors (tester error such as in gait cycle event identification) and marker position errors [28].
Regarding the agreement between Kinovea ® and Vicon ® , we found significant differences in the hip, knee and ankle angles of the systems. In addition, the Bland-Altman plots showed a disagreement between systems of ±5 • for the hip and ankle angles and ±2.5 • for knee angles. Lower agreement in ankle angle may be due to joint range calculation, which even taking the Vicon Plug in Gait ® model [17] as a reference, still differs slightly because it defines the angle of the ankle by relating the axis of the tibia and vector of rotation of the foot (projection of the foot within the transversal plane of the laboratory). Furthermore, the camera position, which was elevated more than one meter from the ground, caused alterations in the view angle of the sagittal plane. Our results are coherent with Littrell et al. (2018) [29], who, in a technical note, analyzed the agreement of Kinovea ® in relation to a three-dimensional motion capture system in five subjects without pathology. They showed larger errors for the pelvis and the foot during the stance period of the gait cycle (foot = ±7.4 • ; pelvis = ±11.8 • ). These results should be considered for the clinicians when they use the Kinovea ® for a clinical evaluation. In addition, future studies should analyze agreement in other phases of walking and in other kinematic parameters such as joint ranges, in which there seems to be less variability [25].

Study Limitations
The presented study has several limitations that must be pointed out. For instance, the single gait phase analysis in a single plane does not allow extrapolation of the results on reliability to the rest of gait phases and the frontal plane. However, the results would justify the start of new studies with more adequate designs.

Conclusions
The intraclass correlation coefficient was good for the hip, knee and ankle angles registered with Kinovea ® during the initial contact of walking for both observers (intra-rater reliability) and higher for the agreement between observers (inter-rater reliability). However, the Bland-Altman plots showed disagreement between observers, measurements and systems (Kinovea ® vs. three-dimensional motion system) that should be considered in the interpretation of the clinical evaluations.