1. Introduction
Load carriage is an important component of typical occupational activities for military soldiers. When in the battlefield and during training, soldiers wear bulky clothing, stiff armour, and carry heavy backpacks. The items a soldier carries are lifesaving and operationally-relevant. However, they may also lead to musculoskeletal injuries over time [
1,
2,
3] and impair overall performance (e.g., movement speed and exposure time) [
4,
5]. To better understand the demands placed on the body by typical military loads, researchers have studied the effects of load carriage on joint kinematics using optical motion capture. Operationally-relevant loads increase trunk flexion, lower limb flexion range of motion (ROM), and walking speed during gait [
6,
7]. However, optical motion capture is generally confined to use in a laboratory setting and measuring motion in a more natural working environment is not typically feasible. Furthermore, optical motion capture requires line of sight between the cameras and body-mounted markers, which makes it challenging to study the effects of wearing military equipment as the markers must not be covered.
Inertial measurement units (IMUs) are becoming increasingly popular as a method of measuring human movement that overcomes some of the barriers of optical motion capture. IMUs are small devices that contain sensors to make inertia-based measurements of motion. These may consist of an accelerometer, gyroscope, and magnetometer that measure linear acceleration, angular velocity, and variations in the magnetic field, respectively. A range of kinematic and kinetic variables, including joint angles can be calculated by fusing data from these sensors and incorporating constraints from a biomechanical model [
8,
9]. IMUs can be placed directly on the skin, can be covered with clothing, and are highly transportable, which directly addresses the aforementioned limitations of optical systems. These benefits of IMU technology allow researchers to study individuals in their natural environment, while wearing the appropriate clothing and equipment needed to perform a given task.
Growing interest in the potential of IMUs has led to a number of validation studies that aim to assess the accuracy of joint kinematics calculated using these wearable sensors. Typically, accuracy has been quantified by comparing joint angles calculated using IMU systems to optical motion capture through root mean squared error (RMSE) [
10,
11], correlation coefficients [
12,
13], and/or Bland-Altman limits of agreement [
14,
15]. Efforts have largely been focused on IMU system validation for lower limb angles during gait [
16,
17,
18]. However, other activities, including stair climbing, kicking, materials handling, and skiing [
14,
19,
20,
21] and upper body angles [
22,
23] have been examined as well. Lower limb sagittal plane angles have been found to have relatively low RMSE ranging from 2–11° [
24,
25,
26,
27] and good correlation coefficients from 0.9–1.0 [
11,
12,
28] with optically-derived angles. Errors tend to be greater and correlations worse for frontal and axial plane angles [
12,
19,
25]. Studies reporting full-body angles have demonstrated larger errors for upper body angles, compared to lower limb angles [
14,
29]. These previous IMU system validation studies were aimed at investigating movements that target the lower limbs and manual materials handling tasks where there was little translation by the participant. These are not representative of the prone positions or rapid changes in height that are typically performed by soldiers. As a result, the validity of using an IMU system to measure military-based movements remains unclear.
Our larger, overarching goal is to generate morphable models of movement using principal component analysis (PCA) and linear discriminant analysis. That is, we intend to incrementally alter and represent military movement patterns between two body-borne load conditions without actually collecting an intermediate load. This will allow us to simulate a spectrum of body-borne load conditions that are infeasible to collect. Before this can be accomplished, we must establish an accurate method of measuring full-body movement during typical military activities in the field while wearing operationally-relevant loads. The purpose of this investigation was to compare an IMU system (MVN BIOMECH Link, Xsens, Enschede, the Netherlands) with an optical (OPT) motion capture system (Vantage V5, Vicon, Oxford, UK) for a variety of operationally-relevant military movements. To be comparable to previous research, we compared the RMSE of the joint angles calculated using the IMU system to the OPT system. We calculated joint angles using the IMU data in multiple ways to determine the differences between systems that could be expected in practice, and to examine any difference due to the biomechanical models used by each system and the technology itself. We also examined the agreement of the systems in terms of full-body movement variability as captured using PCA as this pattern recognition technique will be used in the future to generate morphable models of movement.
2. Materials and Methods
2.1. Participants
Ten male and 10 female civilian participants were recruited for this study. However, due to technical difficulties caused by both systems, which made the data unusable, only nine males and nine females were analyzed. Mean age, height, and mass of all participants were 23.7 years (standard deviation (SD) = 3.44), 175 cm (SD = 7.93), and 71.9 kg (SD = 13.2), respectively; all participants’ demographics are reported in
Table 1. All participants read and signed the participant information letter, which was approved by the University of Ottawa ethics board (approval number H-06-18-721), before data collection began.
2.2. Participant Preparation and Equipment
Participants were asked to change into athletic shorts and a Lycra® T-shirt that is specifically designed for the Xsens system. The T-shirt has Velcro® locations on the shoulders to place sensors, zippers on the back to contain wires, and pockets in the back to host the battery and onboard computer.
Participants were first outfitted with 17 IMU sensors, which were placed on the back of the head, sacrum, sternum, and bilaterally on the upper arms, forearms, hands, shoulders, thighs, shanks, and feet. The pelvis, upper limb, and lower limb sensors were affixed to the skin using neoprene bands, specialty gloves were used for the hand sensors, an elastic headband was used for the head sensor, the trunk sensors (sternum and shoulders) were placed in the T-shirt described above, and the feet sensors were placed inside participants’ shoes using a neoprene insert.
A cluster-based optical marker set was then placed atop the IMU system. Passive four-marker clusters were affixed to the body with Velcro® atop the IMU system’s neoprene bands to be in a similar locality as the IMU system for each segment. The four-marker cluster placed approximately over the T10–T12 vertebrae was placed using a neoprene band that wrapped around the abdomen and over the IMU system’s battery pack and onboard computer; the cluster was placed between the hardware. A three-marker cluster was taped onto the dorsal aspect of the toe bed of the participants’ shoes and four individual markers were taped onto the headband. Individual markers were also taped to specific anatomical landmarks (bilaterally on the medial and lateral malleoli, medial and lateral condyles of the humerus and femur, greater trochanter of the femur, radial and ulnar styloid processes, PSIS, ASIS, iliac crests, acromions, and dorsum of the hands and on C7) so a whole-body biomechanical model could be created.
Participants’ kinematic data were captured simultaneously from both the IMU (MVN BIOMECH, Xsens, Enschede, the Netherlands) and OPT (Vantage 5, Vicon, Oxford, UK) motion capture systems at 240 Hz. A common event of hand clapping/floor smacking was used to synchronize both motion capture systems. A visual representation of the participant setup is displayed in
Figure 1.
2.3. Movement Protocol
Participants were asked to perform eight operationally-relevant military movements: running, walking, kneel-to-prone (KTP), prone-to-kneel (PTK), kneel-to-run (KTR), prone-to-run (PTR), run-to-kneel (RTK), and run-to-prone (RTP). All tasks were performed three times in a straight line with the exception of walking, where participants were asked to walk in a criss-cross pattern measuring 3.5 m × 2 m twice (i.e., two repetitions of walking forward 3.5 m, turning 135° clockwise to walk on a diagonal 4 m, turning 135° counter clockwise to walk forward 3.5 m, and turning 135° counter clockwise to walk on a 4 m diagonal to return to the start position).
Due to the limited field of view of the OPT system, participants started ~3.5 m outside of the capture volume for the RTK and RTP movements and were instructed to transition into kneeling or prone in the middle of the capture volume. Conversely, during the KTR and PTR movements, participants started in the middle of the capture volume and were instructed to run ~3.5 m outside the capture volume. For the running movement, participants started ~3.5 m outside the capture volume and were instructed to run through the capture volume, beginning their deceleration ~3 m past the capture volume. For movements that began outside the capture volume, participants performed the hand clap in the middle of the OPT capture volume, jogged to the start position, performed the movement, and then hand clapped/floor smacked at the end of the movement. For those movements ending outside of the OPT capture volume, participants completed the movement then jogged back to the middle of the capture volume to perform a hand clap.
2.4. Data Processing
Three-dimensional (3D) time series marker trajectories from all trials and repetitions were exported from the OPT motion capture system to coordinate 3D file format (C3D). Prior to exporting, OPT marker trajectories were gap filled and filtered using a 4th order Butterworth filter in Nexus 2.5 (Vicon, Oxford, UK). MVN Analyze (Xsens, Enschede, The Netherlands) provides 64 3D time series marker trajectories of virtual markers for the IMU data that are generated based on the pose of each segment and their underlying biomechanical model. These were exported in C3D format as well. For PCA, only those markers that were similar between systems (N = 30;
Supplementary Material Table S1) were imported into Matlab (2018b The MathWorks, Natick, MA, USA). For the joint angle RMSE analysis, C3D data from both systems (
Supplementary Material Table S2) were imported into Visual3D (C-motion, Germantown, MD, USA) to calculate joint angles using the Visual3D six degrees of freedom pose computation algorithm (
Figure 2), which were subsequently imported into Matlab for further analysis (V
OPT and V
IMU). Joint angles were calculated a second time from the Visual3D IMU data following the application of a method to better align the segment anatomical coordinate systems to those in the optical data model (V
IMU-CAL). In this procedure, for each body segment the angular velocities of adjacent segments were calculated with respect to that body segment for both V
OPT and V
IMU models throughout all motion trials. This was used to calculate a transformation matrix between the anatomical coordinate systems of the two models for each body segment. The transformation matrix was applied to the coordinate systems in the V
IMU model to calculate V
IMU-CAL [
30]. This enabled a closer comparison of the two measurement technologies by mitigating error due to coordinate system alignment [
14]. Joint angles were also calculated directly from the IMU data in the MVN Analyze software and imported into Matlab for comparison (X
IMU). All data were synchronized using the peak acceleration of the hand markers (OPT)/hand segments (X
IMU) during the hand clap/floor smack events.
2.4.1. Principal Component Analysis
For both systems, 3D marker trajectories were synchronized then cropped. For the walk and run trials, each gait cycle was cropped based on heel strike, while the remaining movement trials were cropped based on visually-identified start and end points. All trials were then normalized to 101 data samples using a piecewise cubic Hermite interpolating polynomial (PCHIP). These data were reshaped from the original 101 × 90 matrix (101 data points × (30 markers * 3 axes)) to a 1 × 9090 vector. For each movement type, the newly shaped vectors were horizontally concatenated to create a N × 9090 matrix, where the first
rows were OPT data and the last
rows were IMU data. PCA was applied to each N × 9090 matrix.
PCA was applied separately for each movement based on the way the participants performed the movement. The KTP, PTK, PTR, KTR, RTP, RTK, and Run tasks were separated as left or right movers. For example, for the KTP task, the participants who kneeled with their left leg were considered left movers versus participants who kneeled with their right leg were right movers, and, therefore, were analyzed through PCA separately. This right versus left kneeling criteria was used for the prone transitional tasks as well, as a kneel is a functional movement in the transition into and out of prone. Run was separated based on either a left or a right gait cycle. Additionally, for the walking task, gait cycles were separated by left and right sides as well as the four directions that made up the crisscross pattern (i.e., left gait and right cycles were analyzed separately for the initial forward progression, both diagonals, and the second forward progression for a total of 8 PCA models). Movements were analyzed separately based on left and right sides because PCA identifies the greatest modes of variance within a dataset. Therefore, if both left and right movers were included in the same analysis, PCA would identify left versus right as the greatest difference, rather than the variance within the time series marker trajectories, which is of much greater interest.
For each movement type, the number of principal components (PCs) retained was the minimum amount to describe >90% of the variance within the marker trajectories. The PC scores for each retained PC were compared between the OPT and IMU systems using Pearson correlations; Spearman rank correlations were used for those data that violated parametric assumptions. Correlation coefficients from left and right sides as well as each walking direction were averaged together to represent one correlation coefficient for each retained PC for each movement. Correlation coefficients were interpreted as negligible (0.00 to ± 0.30), low (± 0.30 to ± 0.50), moderate (± 0.50 to ± 0.70), high (± 0.70 to ± 0.90), and very high (± 0.90 to ± 1.00); direction of the correlation was interpreted as either positive or negative. Variance explained by each PC from left and right sides were averaged as well.
2.4.2. Root Mean Squared Error
Joint angles calculated through Visual3D and MVN Analyze were imported into Matlab 2018b where a bias for each joint angle was removed by subtracting the joint angles from a standing calibration trial that was shared by both systems. Data were synchronized then cropped and normalized to 101 data samples using PCHIP, as described above. Following normalization, the value of the first frame was subtracted throughout the entire joint angle so that all systems began at 0°. To ensure synchronization, flexion-extension data were compared using cross-correlation for all joints between VOPT and VIMU, VOPT and VIMU-CAL, and VOPT and XIMU. Those trials that had an optimal lag greater than 10 frames or less than −10 frames were excluded. For each joint, RMSE was calculated four times: (1) OPT calculated though Visual3D and IMU calculated through MVN Analyze (VOPT vs. XIMU), (2) both systems calculated through Visual3D (VOPT vs. VIMU), (3) both systems calculated through Visual3D with the segment coordinate systems alignment procedure for the IMU data (VOPT vs. VIMU-CAL), and (4) IMU data calculated by both MVN Analyze and Visual3D with coordinate system alignment (XIMU vs. VIMU-CAL) for each movement trial. VOPT versus XIMU was included to assess the overall difference between systems. VOPT vs. VIMU compares differences between the systems when the IMU joint angles are calculated in a way that is more similar to the method used for the OPT data while not requiring any information from the OPT system. VOPT vs. VIMU-CAL was included to examine the difference due to technology while mitigating errors arising from differences in segment coordinate system alignment. XIMU vs. VIMUCAL was included to examine differences arising between models while controlling for segment coordinate system alignment between MVN Analyze and Visual 3D with no effect of technology.
4. Discussion
The purpose of this investigation was to validate an IMU system against a gold standard OPT system for military-based movements using PCA and joint angle RMSE. Overall, the OPT and IMU systems produced PC scores that were highly positively correlated; the average correlation coefficient was 0.81 (SD = 0.14) across all 48 retained PCs (
Table 2). This result instills confidence that both the OPT and IMU systems are reconstructing whole-body movement patterns similarly. Lower limb RMSEs for joint angles calculated using the IMU data for all methods compared to the OPT system were less than 10 degrees for all axes (
Table 3,
Table 4,
Table 5 and
Table 6), while differences were greater between the systems for the shoulder and elbow angles. The angles measured using the OPT and IMU systems were most similar for walking trials, and RMSE tended to be greater for more rapid motions involving larger ranges of motion. RMSE was generally smaller for lower limb joint angles compared to upper limb angles.
Previous validation research of the Xsens IMU system against a gold standard OPT system during over ground walking [
11,
19,
25] and stair ascent [
19] reported lower limb joint angle RMSE values less than 5° [
25], 4° [
19], and 6° [
11] in the flexion axis and less than 8° [
19,
25] and 10° [
11] for the other axes. In the current study, for over ground walking, we observed joint angle RMSE values of less than 5° for the lower limbs across all axes when comparing V
OPT vs. X
IMU. Across all tasks studied, we observed V
OPT vs. X
IMU RMSE values of less than 8° for the lower limbs across all axes. Our results are also comparable to [
14] who reported RMSE values less than 8° for the lower limbs and shoulder axial rotation RMSE up to 40° during a manual materials handling task versus the 31.5° reported here. Robert-Lachaine et al. [
14] did report smaller differences for the knee (flexion RMSE 3.2° vs. 7.3° presented here) and elbow (axial rotation RMSE 12.2° vs. 39.9° presented here) compared to the present study. However, the higher RMSE values presented here may be attributed in part to the nature of the movements studied (i.e., material handling vs. rapid changes in height and large ranges of motion).
We used multiple methods in calculating joint angles from the IMU data to provide some insight into the sources of differences between the IMU and OPT systems. The V
OPT vs. X
IMU comparison is what would be expected during a typical analysis session, where overall differences are a result of coordinate system alignment, biomechanical model constraints, and the measurement technology used between the two systems. We found that the overall RMSEs between V
OPT vs. X
IMU were less than 9° for the lower limbs but up to 40.5° for the upper limbs. The V
OPT vs. V
IMU comparison was included to investigate the differences between the systems using a similar underlying biomechanical model for calculating joint angles, which resulted in an average reduction in RMSE values for the upper limb angles of 9.3° (21.6° vs. 12.3°) across all axes compared to the V
OPT vs. X
IMU comparison, while slightly increasing the error in the lower limb angles (<2.5° for knee flexion and <0.5° for all others). To further mitigate the error associated with coordinate system definition and focus on differences due to technology alone, we used an alignment procedure based on segment angular velocities [
14,
30]. The V
OPT vs. V
IMU-CAL comparison further reduced the RMSE between the OPT and IMU systems, especially in the upper limbs where RMSE values decreased on average by 13.0° (21.6° vs. 8.65°) across all axes compared to V
OPT vs. X
IMU. Robert-Lachaine et al. [
14] used this method previously and found differences between OPT and IMU systems due to measurement technology alone to be less than 4° for most angles. While, we observed RMSEs of similar magnitude during walking, across all tasks, RMSEs were close to 10° for many of the angles studied. Finally, we examined RMSE arising from model differences, while controlling for coordinate system alignment between systems (X
IMU vs. V
IMU-CAL). We observed that model differences contributed to a significant portion of the error, with RMSE values for the elbows up to 33.2°, while lower limb RMSE values were lower, indicating the lower limb model constraints and structure were more closely aligned between the two systems. However, model differences even when a coordinate system alignment procedure is applied made a large contribution to the overall error between the systems. Overall, the differences in the model and the constraints therein appear to influence RMSE values to a greater extent than the technology itself as whole-body ab/adduction and axial rotation, as well as upper limb flexion RMSE values were considerably lower in the V
OPT vs. V
IMU-CAL (system error with coordinate system alignment) than in the X
IMU vs. V
IMU-CAL comparison (model difference while controlling for coordinate system alignment).
Typical use of the Xsens system (VOPT vs. XIMU) results in differences in joint angles and time series marker trajectories compared to an OPT system. However, these differences are relatively low for the lower limbs (RMSE of less than 8° across all axes) and whole-body time series marker trajectories have similar modes of variance between systems (average correlation coefficient of 0.81 across the 48 retained PCs). Upper limb joint angle RMSE values are quite high (RMSE of 21.6° across all axes). However, if upper limb joint angles are of great interest, these values can be decreased by mitigating differences in biomechanical models with (VOPT vs. VIMU-CAL; RMSE of 8.7° across all axes) or without (VOPT vs. VIMU; RMSE of 12.3° across all axes) a coordinate alignment procedure. The coordinate system alignment procedure requires simultaneous collection of optical motion capture, which may negate some of the advantages of using an IMU system. However, the results of this comparison demonstrate that, particularly in the upper limbs, a large portion of the differences between the systems (VOPT vs. XIMU) is a result of the biomechanical model, as opposed to differences in the actual measured position and orientation of body segments. Strictly looking at time series marker trajectories, PCA identified similar modes of variance between OPT data and virtual markers projected by the IMU system, with high or very high (≥0.70) PC score correlation coefficients in 38 of the 48 comparisons. This shows that time series marker trajectories can be reconstructed similarly between data derived from the Xsens IMU and the Vicon OPT systems. The RMSE values (average of 8.7° across all joints, axes, and tasks for VOPT vs. VIMU) provide an indication of the differences, compared to an OPT system that could be expected if the reconstructed markers are to be used for joint angle calculations. Whether this is acceptable will depend on the application. However, it indicates that joint angles similar to that of an OPT system can be calculated without the inherent line of sight and laboratory constrictions. The presented RMSE values in addition to the highly correlated PC score values show that the Xsens system is appropriate for our future purposes of creating morphable movement patterns. Users interested in other applications should be aware of the presented errors and make an informed decision if they are within an appropriate range for their intended purposes.
The present study has limitations. Although we have used the OPT system as a ‘gold standard’ for comparison, there are also errors associated with these systems. Therefore, the RMSEs we have calculated represent the observed differences between the two systems, as opposed to the difference between the IMU system and the ‘true’ movement. Due to the limited capture volume of the OPT system, for some movements participants were instructed to start/end their movement outside of the calibrated capture volume. Entering and exiting the calibrated space may cause higher sources of measurement error in the OPT system. Additionally, all OPT systems are limited by skin artefact, which would alter joint angle calculations, especially for more dynamic movements. IMU systems, when collecting for long periods of time, experience positional drift. It was also noted that upper arm segments would become misaligned over time (i.e., when the participants touched their hands together one hand of the Xsens avatar would go through the other). Although, the Xsens avatar was realigned between movement trails, positional and segmental drift could contribute to lower correlation coefficients for some PCs and contribute to the higher RMSE values for the upper arm joints. Additionally, we realized some large anecdotal improvements with the Xsens tracking after discussing with their technology team who suggested: (1) a hand clap during the walking calibration process to improve upper limb tracking, and (2) to start and end all collections with the participant in standing. Unfortunately, this information was learned after this data collection ceased so it could not be implemented. However, we believe that these would improve the PC score correlation coefficients and reduce RMSE values in future work.