1. Introduction
Motion capture systems are frequently used to assess athletic performance or to support return to play decisions, as they provide an objective insight into the kinematics of athletes, which can be compared with, either previously captured data or other benchmarks (e.g., a normative group). A commonly used technology to capture movements is marker-based optical motion capture (MMC) systems—Systems that combine cameras and active or passive markers. MMC systems record the positions of reflective markers that are attached to an object and are considered to be state of art in motion capture, due to their high accuracy when tracking rigid objects [
1]. However, MMC systems are bound to a capturing volume, often requiring considerable manual post-processing, and are sensitive to disturbances of the recording cameras and infra-red interference from other sources. As a consequence, MMC systems are often found only in research laboratories. A limitation of capturing movements, within a laboratory setting, is that the assessed individual is fully aware of the assessment and there is a lack of external distraction (e.g., opponents) due to the restricted capturing volume. Another technology that can be used to capture movements is inertial-based motion capture (IMC). IMC systems are not bound to a capturing volume, do not require manual post-processing and allow out-of-laboratory testing with rapid results output [
2]. An IMC system uses multiple inertial sensors that combine multiple devices within a unit (one or more accelerometers, gyroscopes and magnetometers), and estimate position and orientation by inferring change of position relative to a past time-point by double-integrating the data. This process, however, is susceptible to drift – the accumulation of small measurement errors within the sensor over time. To remove/reduce the effects of the sensor drift, Kalman filters or gradient descent optimization algorithms are commonly used [
3]. However, because of the drift in the inertial sensors, the accuracy of IMC systems is somewhat compromised. Previous studies, which validated IMC against MMC systems, reported good agreement in pelvis sagittal (root mean square difference [RMSD] = 8.89; r
2 > 0.74) and frontal (RMSD = 4.44; r
2 > 0.74) plane angles during daily living activities (gait, sit-to-stand transfers and block step-up) [
4], trivial to small differences in sagittal pelvis and lower extremity kinematics during kicking [
2] and very high validity (r
2 > 0.80) in the sagittal plane of hip, knee, and ankle joint angles during walking, jumping and squatting, but only demonstrating acceptable validity (0.40 < r
2 < 0.08) in the frontal and transverse planes for squatting and jumping [
5]. However, while these studies imply that IMC systems can be used to record movements, the duration of the activities examined is relatively short (<2 s), and higher measurement errors would be expected to be observed in longer data captures [
6]. The ability to capture movement accurately, over a more-extended period, enables the analysis of movement sequences, which potentially increases the external validity as preparation time and internal conscious focus on the executed movement tasks reduce. Practitioners currently utilize field-based tests of acceleration, deceleration and change of direction for this purpose, aiming to simulate the demands of a real-time sporting scenario by incorporating a series of tasks that have to be performed in the shortest time possible. The practical use of MMC systems to record data from these tests is limited because of the following three shortcomings: A large capture volume is needed, obstruction of marker visibility and consequently high post-processing demands. In contrast, an IMC system does not suffer these three short-comings and offers a solution in enabling efficient feedback of objective movement metrics to athletes and coaches. However, the validity of an IMC system, during a field test or game-like situation, is not currently known.
The aim of this study is to examine the accuracy and precision validity of an IMC system during a field test.
2. Materials and Methods
2.1. Subjects
Six male recreational athletes participated in this study. All athletes consented to the study. Ethical approval was granted from the Sports Surgery Clinic ethics committee.
2.2. Field Test
The chosen field test (VU®; Sports Surgery Clinic, Dublin) was developed by clinicians for use within their clinical practice, and used as a field test as it simulates athletic qualities associated with a variety of field sports, due to phases of acceleration, deceleration, change of direction, curved and backwards running, as well as a 15-meter sprint. These features are often assessed independently (e.g. The T test, The Change-of-Direction and Acceleration Test, The Illinois agility run, the pro agility test, the 3-cone drill, the L run or the 505), but have not previously been combined in a single test. The layout of the test is illustrated in
Figure 1 and the flow of the test is described in the figure caption.
2.3. Data Capture
Participants were tested twice and asked to complete 10 VU® tests (alternating between left- and right-sided) within each session. Data capture was performed at an indoor motion capture facility (Audio-motion Studios Limited, Oxford, UK) that was able to accommodate optical motion capture within the volume needed. This motion capture studio was equipped with 70 cameras that recorded a capturing frequency at 200 Hz and were mounted on scaffolding.
At the start of each session, each participant was asked to perform a self-selected warm-up and was subsequently familiarized with the field test by performing two walking and two jogging effort trials. After familiarization, eight inertial measure units (IMC sensor; 100 Hz; MTw2; Xsens Technologies B.V, Enschede Netherlands) were attached to the participant using Velcro straps, as per Xsens recommendations. To record the kinematics of the trunk and pelvis, IMC sensors were placed on the upper part of the sternum and directly over the sacrum, using double-sided adhesive and zinc oxide tape to keep the IMC sensor in position during data capture. To record the kinematics of the left and right limb, two IMC sensors were was placed between the layers of the Velcro strap, to secure its position and minimize any movement, the first was placed centrally on the upper limb halfway, between the greater trochanter and lateral epicondyle of the femur (thigh sensor) and medial on the proximal surface of the tibia (shank sensor). To record kinematics of the left and right foot, an IMC sensor was placed on the dorsum of each foot using double-sided adhesive tape, as well as zinc oxide tape. After the IMC sensors were placed, 32 retroreflective markers (14 mm diameter) were attached to the participants using double-sided adhesive tape as well as zinc oxide tape to keep retroreflective markers in position during the data capture. Markers were attached so that both the Plug-In-Gait (PiG) model and the optimal shape symmetrical centre of rotation estimation approach (OSSCA) could be used.
After the IMC sensors and markers were placed a “reference” trial was performed to calibrate the motion model - range of motion, as per Vicon for the MMC and n-pose walk [
7] for IMC. During the range of motion reference trial, the participant stood in a stationary position, and then for each leg, sequentially flexed/extended the knee, flexed/extended the hip, abducted/adducted the hip and circum-ducted the hip, as per Vicon recommendations. During the walking reference trial, the participant stood in a stationary position, walked 2 m forwards in a straight line, turned around, walked back to the starting position and turned around to face the original heading direction. Subsequently, participants were asked to complete a maximal effort trial on either, the left or right side (randomized), rest for 2 min and perform a maximal effort trial on the opposite side. This was done until a total of 10 trials were captured. If a marker fell off the investigators re-attached the marker and repeated the reference range of motion trial. If an IMC sensor fell off, the investigators re-attached the sensor and repeated the reference walking trial. The data capture was started independently in the optical and IMC system.
2.4. Data Preparation
To prepare the data for analysis, the following steps where performed: Data post-processing, joint angle calculation, system synchronization and data extraction (
Figure 2). Data post-processing and joint angle calculation were performed first for both the IMC and the MMC, as the centre of mass (CoM) measures were used for synchronisation, which was in turn needed for the event detection.
The IMC data was processed using MVN Analyze 2019.2 (Xsens Technologies B.V, Enschede Netherlands) and the data was then exported as mvnx files. Processing included filtering, skin movement artefact reduction and functional joint estimation via the MVN Fusion Engine. To calculate joint kinematics, the processed quaternion data of the trunk, pelvis, thigh, shank and foot were exported from the mvnx file and used to calculate the rotation matrix (distal to proximal) between trunk and pelvis (thorax angle), pelvis and thigh (hip angle), thigh and shank (knee angle), and shank and foot (ankle angle), which was subsequently transformed into Euler angles using a Y-X-Z sequence. The pelvis angle/pelvis quaternion was expressed on the external reference trial, computed using the t-pose and then transformed into Euler angles using a Z-Y-X sequence. The CoM in x, y and z as well as the foot contact definitions (as generated by the MVN Fusion Engine) were also exported for further processing. Joint kinematics, CoM measures and foot contact definitions were then exported into a SQL file.
After processing the IMC data, the optical motion data were processed. Processing included the reconstruction of marker positions, marker labelling, filling of gaps in marker trajectories, filtering of marker trajectories (Woltring; 15 Hz cut-off [
8]) and the fitting of the segments to marker trajectories. Gap filling was performed using a segment fill approach if possible and a pattern fill or spline fill approach if not. Segment bones were then fitted to the marker trajectories, using the PiG pipeline as per Nexus 2.8.1 (Vicon Metrics Group Ltd, Oxford, UK) and the OSSCA pipeline [
9] as per the Nexus Advanced Gait Workflow (Nexus 2.8.1, Vicon Metrics Group Ltd, Oxford, UK). OSSCA is a workflow that combines the optimal common shape technique (OCST; [
10]), symmetrical axis of rotation approach (SARA; [
11]) and symmetrical centre of rotation estimation (SCoRE; [
12]), and seeks to minimize the sensitivity of joint kinematics to marker placement. The hip joint centre and knee and ankle axes were then calculated using the symmetrical centre of rotation estimation (SCoRE; [
12]), and the symmetrical axis of rotation approach (SARA; [
11]), respectively. Soft tissue artefact was minimized using the optimal common shape technique (OCST: [
10]), where an optimum rigid marker configuration for each segment is formed to reduce the effects of skin elasticity. To calculate joint kinematics for the MMC, the origin of the segment data from the trunk and pelvis (thorax angle), pelvis and thigh (hip angle), thigh and shank (knee angle), as well as shank and foot (ankle angle) was translated into the global origin and then used to compute a rotation matrix (distal to proximal) that was subsequently transformed into Euler angles using a Y-X-Z sequence, for the outcomes of PiG, and OSSCA, respectively. The pelvis angle was calculated by transforming the rotation matrix between the global coordinate system and a translated pelvis segment (origin in global origin) using a Z-Y-X sequence. The x, y and z global coordinates of the CoM position was also exported. Joint kinematics and CoM measures were then exported into a SQL file.
To synchronize the IMC and MMC, the IMC data was up-sampled to 200 Hz. The vertical position of the CoM was then extracted from the SQL file for both systems and used to compute CoM vertical velocity. The timing offset between the systems was then defined by the lag index that corresponded to the maximal correlation coefficient observed during a cross-validation and subtracted from the device that triggered data capture first. To ensure both data sets contained the same number of samples, both data sets were cropped to the time point, 0, and the minimum duration across the systems. The last step in processing was the rotation of the horizontal IMC CoM measures in the CoM description of the MMC (x = left-right direction; y = forward-backward) using a rigid transformation.
The final step in the data preparation was event detection (
Figure 3). The field test can be separated into the following phases: Hurdle hop (HuHo), the first change of direction (V1) phase, backward running (Backwards) phase, p the second change of direction (V2) phase, the first curved run (U1) phase, the second curved run (U2) phase and the sprint (Sprint). For a trial that started with a left limb lateral HuHo, the start of the HuHo was identified as the second impact of two subsequent contacts of the left limb (where vertical CoM velocity was below 2 m per second). The end of the HuHo was defined as the toe off within that contact. The start of the V1 phase was defined as the frame following the end of the HuHo and ended at the local next local minima of the CoM x. The backward running phase started at the frame, following the end of the V1 phase and ended at the next local maxima of CoM x. The V2 phase was defined to start at the frame, following the end of the Backward phase and ended at the next local maxima of CoM y. The U1 phase was defined to start at the frame following the end of the V2 phase and ended at the next local maxima of CoM y. The U2 phase was defined to start at the frame, following the end of the U1 phase, and ended at the next local minima of CoM y. The Sprint phase was defined to start at the frame following the end of the U2 phase and ended when the CoM y passed the 20-meter mark.
2.5. Statistical Analysis
The reconstruction of three-dimensional (3D) marker positions using optical motion capture is reliant on simple triangulation and is accurate to <1 mm [
1]. However, multiple approaches (models) calculate the tri-planar joint angles from these marker position trajectories [
12,
13,
14,
15,
16]. These models differ in degrees of freedom and constraints, applied at each joint, as well as compensation for skin movement artefacts and calculation of joint centres and axes of rotation. This can result in considerable differences in outcome measures between different models, and it is therefore, not possible to compare IMC-derived kinematics against a single universal MMC-based ‘gold standard’. For the purpose of this study, we defined the OSSCA models as most appropriate and as a reference standard. To interpret the magnitude of differences (bias) between OSSCA derived kinematics and IMC outcomes, a benchmark was computed by comparing kinematics derived from OSSCA and the commonly used PiG model. Both accuracy (systematic [device] and random error) and precision (random error) metrics were calculated, in order to differentiate systematic and random errors in the IMC system. If differences between IMC and MMC were no larger than differences between OSSCA and PiG, we concluded that the accuracy/precision of the IMC system for the relevant variable was acceptable, e.g., within the error resulting from differences in MMC model assumptions.
Both accuracy and precision were examined by using integrated pointwise indices, within each phase of joint kinematics and horizontal CoM measures [
17], and also for the reference trial. The accuracy of the IMC system was expressed based on thhe level of agreement between the OSSCA and IMC outputs, by calculating the bias (differences) and 95% limits of the bias [
18]. The precision of the IMC system was expressed by the root mean square prediction difference (RMSD) of a regression analysis [
19], between the OSSCA and IMC outcomes, the % RMSD ([RMSD/OSSCA signal] * 100) and the resulting squared correlation coefficient (r
2; Pearson). The r
2 was used to examine shared variation and classified into very high (1.00 > r
2 > = 0.81), high (0.81 > r
2 > = 0.49), moderate (0.49 > r
2 > = 0.25), low (0.25 > r
2 > = 0.09), negligible (r
2 > 0.09) [
20]. No test for statistical differences was performed as accuracy and precision measures were compared to the benchmark set by the PiG vs. OSSCA comparison. Limits of agreement plots were visually screened for proportional bias. Bias, 95% limits of the bias, RMSD, % of RMSD and r
2 were stored in a phase (n = 7) × observation (n = 120 [10 VU × 6 subjects × 2 assessments]) matrix and are reported as the mean and standard deviation of each phase and the overall average of the mean and standard deviation.
Data processing and analysis was performed in Python 3.7.
4. Discussion
This study examined the accuracy and precision of lower-limb kinematics, quantified using an inertia-based movement capture (IMC) system, during a sports-specific field movement test, by comparing the results to those from a standard OSSCA modelling approach using marker-based optical motion capture (MMC). The identified systematic and random errors were contextualised by defining a benchmark for differences. This benchmark was generated by comparing outcomes from OSSCA to another commonly used MMC-based modelling approach, Plug-In Gait. If differences between IMC and MMC were no larger than differences between OSSCA and Plug-In Gait, we concluded that the accuracy/precision of the IMC system for the relevant variable was acceptable, i.e., within the accepted error resulting from differences in model assumptions. Joint kinematics in the sagittal plane demonstrated very high correlations, except for the trunk segment (high), with all mean RMSDs smaller than 4.44 degrees (7% of range), while mean bias ranged from 4.11 degrees (knee) to 16.60 degrees (pelvis), with all joint angles over-estimated by the IMC approach. Only the knee bias was within the benchmark (it should be noted that trunk and pelvis had no benchmark for any comparison). The hip joint was for more than half of the observations within in the benchmark. RMSD and the correlation of the knee and hip were within the benchmark (
Figure 4 and
Figure 7). For the frontal plane, correlations ranged from moderate (ankle and knee) to high (hip, pelvis and trunk). The observed mean RMSDs were below 3 degrees (13% of range), except for the ankle (RMSD = 9.31). The mean bias ranged from −1.41 degrees (ankle) to 9.40 degrees (knee). Compared to the benchmark, the knee bias, RMSD and correlations were within the benchmark (
Figure 5 and
Figure 8). For the transverse plane, correlations were very high for the pelvis and trunk, high for the hip and moderate for the ankle and knee. The ankle, knee and hip bias and RMSD were within the benchmark, while ankle and knee correlations were below the benchmark (
Figure 6 and
Figure 9). The observed RMSDs were between 2.86 and 6.09 degrees (14% of range), with a mean bias between −1.63 (trunk) to 9.91 (hip) degrees. For CoM measures, correlations were very high with RMSD of 0.05 and 0.07 m (2 and 4%), with a mean bias of 0.01 and −0.07 m, for the left-right and forward-backward direction.
Studies that have previously compared outcomes of MMC and IMC systems have reported similar findings. Bolink et al. (2015) validated pelvis kinematics from an IMC motion capture system, by comparing to MMC-derived outcomes and reported similar magnitudes of RMSD (2.70 to 8.89 and 2.68 to 4.44) and r
2 (0.72 to 0.88) for pelvis sagittal and frontal plane angles during gait, sit-to-stand transfers and block step up transfers [
4]. Lachaine et al. (2017) used an improved marker sensor alignment technique and attached optical markers on top of the IMC sensors, and reported much lower bias (<2.5), but similar RMSD (<3.0) in lower limb kinematics within an ergonomics experiment than observed in this study. This configuration, however, is not feasible for assessment of the highly-dynamic movements used in athlete field testing because of the physical restrictions on the participant resulting from sensor placement [
6]. Another study, using the same IMC as used in this study, reported biases higher than those observed in this study (possibly due to model offset) and the validity was very high in the sagittal plane for hip, knee, and ankle joint angles in all three tasks. It was acceptable in frontal and transverse planes during squat and jump activity across joints due to high correlations [
5]. In contrast to other studies discussed in this paragraph, Koska et al., [
21] reported considerable systematic and random disagreement in ankle kinematics during running, which is in line with the findings observed in this study.
All kinematic variables, calculated from both MMC and IMC systems, inevitably contain some degree of error relative to the ‘true’ kinematics of the body. These errors result from a combination of marker placement inaccuracy, soft tissue artefacts, sensor output inaccuracy and the assumptions of the model used to translate sensor outputs into joint-level kinematics. Some lead to systematic offsets from the true joint angles (bias), whilst some result in random between-trial variation in these offsets. While, MMC techniques are considered to be the ‘gold standard’ for movement analysis, the outcome variables are highly sensitive to the locations of the markers placed on the body. Contemporary modelling approaches attempt to minimise this effect by defining joints and planes, based on movement, rather than simply marker positions (similar to [
10,
11,
12]). However, they are nevertheless, still reliant on the accurate and consistent placement of calibration markers, whilst conventional widely-used modelling approaches rely wholly on marker positioning to define joint centres and tri-planar segment angles. Errors or inconsistencies (e.g., due to differences in thickness of subcutaneous fat) in the placement of markers relative to target anatomical landmarks is thus the single greatest contributor to measurement variability in contemporary clinical gait analysis [
22]. It affects the kinematic measures by up to 15 degrees in the sagittal plane, 15 degrees in the transversal plane and 17 degrees in the transverse plane [
22,
23,
24,
25,
26]. In interpreting the accuracy and precision found for the IMC system in this study, we compared the bias and RMSD to those previously reported for MMC-based systems, resulting from marker placement error within, and between, assessors (
Table 2). The biases observed for the ankle, knee and hip, with the exception of hip sagittal plane angle, are within the published error ranges, resulting from MMC marker placement. However, when considering the benchmark generated in this study all three planes of the ankle (as in [
21]), pelvis tilt and hip flexion all carry an error that cannot be explained by the effect of marker placement systematic error.
The increased ankle error may be explained by the fact that the foot segment’s orientation is modelled from a single IMC sensor, which is attached to the shoe tongue, whilst the MMC approach uses 3 markers (or more; 7 markers in [
22]) to define the segment. As such, the IMC model may be more sensitive to movement artefacts caused by deformation of the shoe material. Errors could potentially be reduced by developing IMC solutions that rely on more than one sensor to define the orientation of the foot segment. However, this might not resolve the disagreement between MMC and IMC, as the MMC system is also sensitive to marker movement: Deformation of shoe/feet on the metatarsal can result in erroneously moved markers, and hence, errors in calculated frontal plane angles. Similarly to the ankle joint, the description of the pelvis could potentially be improved using a model using 3 pelvis sensors. The IMC system could determine pelvis tilt/drop, by measuring their offset in the vertical/horizontal axis, which should reduce disagreements in pelvis kinematics and their propagation into the hip angle (
Figure 4). While the mean values of bias and RMSD for hip abduction angle are within the benchmark range, we note that the upper quartiles are consistently outside the benchmark. These errors may be due to differences in the anatomical model and other processing steps, different effects of movement artefact/soft tissue movement filters [
8,
27], the fact that sensors were attached to different locations, or differences in the identification of joint centres and axes between the MVN Fusion Engine and OSSCA model. Further, the findings highlight the importance in maintaining consistency in the use of one model regardless of the motion capture system (both the IMC and PiG demonstrated considerable bias and poor correlations). This needs to be considered when comparing motion analysis studies.
The true systematic error of the IMC system itself is probably best described by the correlation and RMSD of the pelvis angles, as these are the only kinematic measures that are computed by a single sensor. Correlations between the IMC and optical motion capture were very high for pelvis tilt and rotation (r2 > = 0.84) and high for pelvic drop (r2 = 0.79), while RMSDs were below 3 degree. As such, the maximal systematic error should not be above 6 degrees (joint angles = proximal segment – distal segment). Errors above this threshold are likely to be random errors.
In term of the length of data capture, the accuracy and precision seem to not be impacted with a duration of 20 s, as errors did not vary across the different phases of the test.
A limitation of this study is that temporal properties (e.g., impact phase, mid-stance or toe off) of the running, change of direction or hopping cycles were not examined. Errors are likely to vary at different times, which was also highlighted by the reference trail comparison as most IMC (except rotation) measures examined were within the benchmark. This is in line with previous studies [
25,
28].