Test-Retest Reliability of the MotionMetrix Software for the Analysis of Walking and Running Gait Parameters

The use of markerless motion capture systems is becoming more popular for walking and running analysis given their user-friendliness and their time efficiency but in some cases their validity is uncertain. Here, the test-retest reliability of the MotionMetrix software combined with the use of Kinect sensors is tested with 24 healthy volunteers for walking (at 5 km·h−1) and running (at 10 and 15 km·h−1) gait analysis in two different trials. All the parameters given by the MotionMetrix software for both walking and running gait analysis are tested in terms of reliability. No significant differences (p > 0.05) were found for walking gait parameters between both trials except for the phases of loading response and double support, and the spatiotemporal parameters of step length and step frequency. Additionally, all the parameters exhibit acceptable reliability (CV < 10%) but step width (CV > 10%). When analyzing running gait, although the parameters here tested exhibited different reliability values at 10 km·h−1, the system provided reliable measurements for most of the kinematic and kinetic parameters (CV < 10%) when running at 15 km·h−1. Overall, the results obtained show that, although some variables must be interpreted with caution, the Kinect + MotionMetrix system may be useful for walking and running gait analysis. Nevertheless, the validity still needs to be determined against a gold standard system to fully trust this technology and software combination.


Introduction
In both research and diagnosis, the use of marker-based motion capture technologies has been expanded dramatically. However, inherent limits in data collecting may restrict its use in contexts such as patient homes, sports fields, or public spaces where the use of a large number of cameras is impractical. Here, a markerless motion capture system has been offered as one possible solution [1,2].
Markerless systems do not require any markers or sensors to be attached to the body, reducing clinical feasibility and testing time significantly. The lack of markers, on the other hand, may have an impact on measuring accuracy. Thus, investigations examining the validity of such systems under different conditions are crucial. In this context, doctors, sports practitioners, and researchers have been paying close attention to a markerless motion capture system [1,[3][4][5][6][7].
The validity of the Kinect™ sensor, created first for interacting with video games on the Microsoft Xbox™ platform by using body movements, for the analysis of gait parameters has been previously evaluated [1,[3][4][5][6][7]. Various pieces of software, including various filters and calibrations, have been studied in these works. Whereas Schmitz et al. [1] analyzed the validity of the Kinect™ system with the KinectFusion software for kinematic Sensors 2022, 22, 3201 2 of 10 data assessment, Dolatabadi et al. [3] identified the concurrent validity of the Kinect™ for Windows to measure gait spatiotemporal variables. Then, the validity of the Kinect™ for gait kinematics analysis in comparison with a "gold standard" motion capture system was assessed, operating both systems under Cartesian calibration [4]. Likewise, concurrent validity of the Kinect system for spatiotemporal gait parameters was also assessed [6]. However, none of the mentioned studies considered the MotionMetrix™ software, which might affect measuring accuracy.
As far as the authors' concern, only one study took into account the Kinect + Mo-tionMetrix combination [8]. Here, the absolute reliability and concurrent validity of the Kinect + MotionMetrix combination was evaluated for spatiotemporal parameters when running at a comfortable velocity by comparing data between the combination system and two widely used systems (i.e., high-speed video analysis and OptoGait). It was found that contact time (CT) was overestimated by the system, whereas flight time (FT) was underrated. However, it resulted to be a valid tool for step frequency (SF) and step length (SL) measures [8]. Although concurrent validity has been assessed, the reliability of the Kinect + MotionMetrix system for either walking or running parameters has not been evaluated.
To identify whether findings are attributable to changes in gait pattern or merely systematic measurement errors, a gait analysis system's reliability is critical. Therefore, the aim of this study is to analyze the test-retest reliability of both walking and running gait on a treadmill running at 5, 10 and 15 km·h −1 by comparing inter-session data obtained from the Kinect + MotionMetrix system.

Materials and Methods
This study follows the STROBE recommendations for reporting observational studies [9].

Subjects
A group of 16 men and 8 women recreationally active (age = 22.7 ± 2.6 years; body mass = 69.1 ± 11.7 kg; height = 1.72 ± 0.10 m; weekly training = 6.9 ± 2.4 h/week) [10] and familiar with treadmill running voluntarily took part in the study. All of them were free from injuries and reported no physical limitations or health problems. An informed consent was signed by each participant after being informed of the objectives and procedures. It was made clear that they were free to leave at any point. The study followed the Declaration of Helsinki (2013) and was approved by the Ethics Board of the local university (No. 2546/CEIH/2022).

Procedures
Each participant attended the laboratory only once. The participants were instructed to refrain from strenuous activity for, at least, 48 h before data collection [11]. During the test, they wore their usual running clothes and shoes. A treadmill (WOODWAY Pro XL, Woodway, Inc., Waukesha, WI, USA) walking and running protocol was completed. An accommodation period on the treadmill of, at least, 8 min was completed at a selfselected velocity [12]. Thereafter, a protocol where 1-min bouts at 5, 10 and 15 km·h −1 was completed. After a 5-min break to avoid fluctuations in the running pattern caused by fatigue, all the participants completed the protocol again. Data were collected during the last 30 s of each bout to guarantee participant adaptation to the running speed

Materials and Testing
Participants body mass (kg) and height (m) were obtained using a bioimpedance scale (Inbody 230, Inbody, Seul, Korea) and a stadiometer (SECA 222, SECA, Corp., Hamburg, Germany), respectively. Table 1 shows the definition of all the parameters provided by the MotionMetrix software (MotionMetrix AB). It offers different kinetic and kinematic variables depending on gait velocity used during analysis. The combination Kinect + MotionMetrix was employed to measure such parameters and all the additional variables that the software provides for walking and running gait. To control potential influencing factors for temporal parameters, only the right leg of the participants was analyzed (i.e., asymmetry) [11]. Through the use of a depth sensor, the Microsoft KinectTM sensor (version 1.0, Microsoft, Redmond, WA, USA) can monitor 3-D motions. It can locate 20 body joints in 3D space at 30 Hz. Here, two Microsoft KinectTM sensors were placed on either side of a treadmill in a certain configuration (170 cm from the treadmill's center in forward direction and 190 cm in the perpendicular direction, according to manufacturer recommendations) and utilized in conjunction with MotionMetrixTM software (Figure 1). The Microsoft KinectTM sensors can reach 60 Hz when both sensors can track the same point at the same time (according to the manufacturer). For data collection, manufacturer recommendations were considered (i.e., software calibration, tight clothes, no shiny black fabric or reflexes, no moving shoelaces, no moving hair, no sunlight, and no treadmill parts blocking the entire view of the participant).

Gait Variables Definition
Stance phase (% gait cycle) Period when the foot is in contact with the floor Swing phase (% gait cycle) Period when the foot is not in contact with the floor Load response (ms) Period of initial double limb support Pre-swing (ms) Last phase of stance Doble support (ms) Stance with both feet in contact with the floor Step time (ms) Interval between initial contacts of the contralateral foot Step length (cm) Distance between initial contacts of the contralateral foot Step frequency (spm) Step rate per minute Hip frontal angle (deg) Hip angle at the coronal plane at the initial single support stage Knee frontal angle (deg) Knee angle at the coronal plane at the initial single support stage Step width (cm) Distance between the heels of the two feet during double stance

Running Variables Definition
Stride time (ms) Time between initial contacts of the same foot Stride length (cm) Distance between initial contacts of the same foot Step frequency (spm) Step rate per minute Step time (ms) Interval between initial contacts of the contralateral foot Step length (cm) Distance between initial contacts of the contralateral foot Contact time (ms) Time between initial contact to toe-off Flight time (ms) Time between toe off and initial contact of the contralateral foot Foot strike angle (deg) Angle between foot and ground at initial contact Ankle landing (deg) Angle between foot and shank at initial contact Center of mass vertical displacement (cm) Center of mass vertical displacement between steps Spine angle (deg)  Lateral force (% of max vertical force) Maximum lateral force during the stance phase Maximal loading rate (BW/sec) Speed at which maximum vertical force is achieved Maximal propulsion rate (BW/sec) Speed at which maximum propulsion force is achieved External work (Joules/kg/m) Work done to accelerate the center of mass with respect the environment Internal work (Joules/kg/m) Work done to accelerate the body segments with respect the center of mass Leg-spring stiffness (BW/m) Vertical leg length in response to the maximum vertical force of a step Leg length difference at stance phase (cm) Vertical leg length change during the stance phase Elastic exchange (%) Fraction of total work stored and released as elastic energy Knee mediolateral force (BW) Maximum

Test-Retest Reliability and ICC Interpretation during Walking Gait
The test-retest reliability data for the kinematic parameters reported by the Kinect + MotionMetrix system during walking at 5 km·h −1 are shown in Table 2. When analyzing walking gait, significant differences were exhibited only for load response (LR) and double support (DS) phases (p = 0.016 and 0.014, respectively) showing a small magnitude of differences (ES = 0.42 and 0.37, respectively) when evaluating both measurements. Moreover, SL and SF showed significant differences (p = 0.002 and 0.008, respectively) with a trivial magnitude of differences (ES ≤ 0.12). For the rest of variables, no significant differences were identified.
Walking gait measures showed acceptable reliability for all variables (i.e., CV < 4%) except step width (SW) (CV = 12.12%). The ICC values obtained with markerless system for ST, SL and SF exhibited an almost perfect correlation (ICC = 0.95, 0.94, and 0.96, respectively). Pre-swing phase (PSw), LR, DS and SW showed substantial correlation (ICC < 0.80). Moreover, whereas ICCs for both StP and SwP were interpreted as moderate (<0.60), ICCs for hip frontal angle (HFA) and knee frontal angle (KFA) were considered as poor (ICC < 0). Tables 3 and 4 show the test-retest reliability data for the kinetic and kinematic variables reported by the Kinect + MotionMetrix system during running at both 10 and 15 km·h −1 , respectively.

Test-Retest Reliability and ICC Interpretation during Running Gait
When analyzing running gait at 10 km·h −1 , significant differences were found between both measures for stride time (StrT), stride length (StrL), SF, SL, maximal thigh flexion (ThighFlex), maximal knee flexion during swing (KFSw), knee rotation (KRot) and SW (p < 0.05). The magnitude of the differences for these variables was interpreted as either trivial (ES < 0.20 for SF, ThighFlex, KFSw, KRot, and SW) or small (ES < 0.60 for StrT, StrL, and SL). However, no significant differences for the rest of the variables provided by the system when running at 10 km·h −1 (see Table 3). Table 4 displays running analysis at 15 km·h −1 using Kinect + MotionMetrix system. Here, no significant differences between trials were found when analyzing kinematic parameters (p > 0.05) except when assessing vertical displacement (Vdisp) and KRot (p < 0.03), which also show a small (ES = 0.22) and trivial (ES = 0.17) magnitude of the differences, respectively. When examining kinetic variables, significant differences (p < 0.05) were found for vertical force (VertF), maximal loading rate (LRmax), maximal propulsion rate (PRmax), external work (ExW), leg-spring stiffness (LSS), knee vertical force (KFv), knee frontal moment (KMf), hip vertical force (HFv), and hip frontal moment (HMf). The magnitude of the differences for all the parameters mentioned above were identified as small (ES < 0.60) except PRmax, KFv, and HFv that were identified as trivial (ES < 0.20). The rest of the variables show no significant differences when analyzing running gait at 15 km·h −1 (see Table 4).
Then, when running velocity was set at 15 km·h −1 , although all the kinematic variables provided by the system show acceptable reliability (CV < 10%), AL, SpA, and SW exhibited CV > 10% for their measures. When kinetic variables were considered, all the parameters show acceptable reliability (CV < 10%) except lateral force (LatF) and elastic exchange EEx showing CV = 13.97% and 21.4%, respectively. The ICC values obtained revealed almost perfect correlation for most of the kinetic and kinematic parameters (ICC > 0.83) except the kinematic parameters of ShA, SpA, and CT (ICC = 0.69, 0.76, 0.80, respectively) and the kinetic parameters of VertF, LRmax, Prmax, KFv, and HFv (ICC = 0.79, 0.71, 0.65, 0.80, and 0.79, respectively). Furthermore, the kinetic variable of elastic exchange (Eex) shows moderate correlation (ICC = 0.58). Table 3. Descriptive data (means, ±SD) and inter-session reliability of the kinematic parameters obtained from Kinect + MotionMetrix software running at 10 km·h −1 .

Discussion
This study aimed to determine the test-retest (inter-trial) reliability of the Kinect + MotionMetrix system for the analysis of both walking and running gait parameters (i.e., kinetic and kinematic variables) on a treadmill. Here, twenty-four participants were tested to assess the inter-trial reliability of such markerless system. Our results show that, although there were significant differences between both measurements for both LR and DS phases, and the spatiotemporal parameters of SL and SF, the system seems to provide reliable measurements when analyzing walking gait at 5 km·h −1 . Then, when considering reliability when running at 10 km·h −1 , no significant differences were found for most of the variables except when assessing StrT, StrL, SF, SL, ThighFlex, (KFSw), Krot, and SW. The system apparently provides reliable measures for all the variables apart from FSA, AL, SpA, ThighFlex, KFL, and SW. During running at 15 km·h −1 , no significant differences were found when evaluating kinematic parameters for both trials besides Vdisp and Krot obtaining, additionally, reliable measures from the system for all the kinematic parameters excepting AL, SpA, and SW. If kinematic parameters (i.e., running velocity) are considered, only Vdisp and Krot showed significant differences between both trials, providing the system reliable measurements for all these parameters except when assessing AL, SpA, and SW. For kinetic variables, although the system seems to be reliable when analyzing such parameters (excepting LatF and Eex), significant differences between trials were found for the measures of VertF, Lrmax, Prmax, ExW, LSS, KFv, KMf, HFv, and HMf. The results expose not only the overall intersession reliability of the system when assessing kinematics in walking and running gait, but also its inaccuracy when considering some kinetic parameters.
Research on validity and reliability of markerless motion capture systems for biomechanical analysis during either walking or running on a treadmill is limited. Although the validity of the Kinect™ sensor for walking gait analysis has been assessed [1,[3][4][5], the findings reported are controversial. A previous study [3] stated that the Kinect™ sensor used for Windows is a valid tool for measuring walking gait spatiotemporal parameters. Others [5,6] have reported important differences when comparing spatiotemporal parameters measured by the Kinect™ sensor and such parameters by a three-dimensional motion capture system. Particularly, Clark et al. [6] determined that walking gait parameters obtained employing the Kinect™ were lower (i.e., −16% ST, −19% StrT, −1.7% SL) than those acquired utilizing the three-dimensional system. Similarly, Xu et al. [5] claimed that the Kinect™ system reported valid ST and StrT values, but shorter stance time (i.e., −9%) regarding the three-dimensional system when walking. Seemingly, the accuracy of the Kinect™ system in measuring spatiotemporal characteristics is mainly reliant on factors such as the software and filter settings used, the gold standard or reference system examined, or the procedure followed as well as target variables. It is worth noting that the treadmill protocol used in the present study was intended to reduce any potential gait and running variability caused by either treadmill inexperience or fatigue [18,19]. It has been reported that a minimum time of 6 to 8 min is required for healthy young adults and novice treadmill runners to accommodate their locomotion on the treadmill. Thus, it remains unknown whether the Kinect™ system would perform in greater variability conditions. As treadmill running has been shown to have certain biomechanical variations from running on the ground [12], readers must be cautious when interpreting the results here reported. Some of the investigations that looked into the validity of the Kinect™ system were done on the ground [3,6], whereas just three studies were completed on a treadmill [4,5,7]. This is key as validity or reliability data obtained while walking should not be transferred to running situations since the magnitude of the parameters changes and other phases emerge (i.e., FT does not exist during walking, while there is no double-support time during running). Pfister and colleagues [7] investigated sagittal plane gait kinematics at different walking and running velocities (i.e., 4.8 to 8.8 km·h −1 ), which are lower than the velocity in the current study (i.e., 5, 10 and 15 km·h −1 ), without mentioning kinetic and kinematic parameters, and concluded that the measurement accuracy of the Kinect™ system was