Validity and Reliability of the Leomo Motion-Tracking Device Based on Inertial Measurement Unit with an Optoelectronic Camera System for Cycling Pedaling Evaluation

Background: The use of inertial measurement sensors (IMUs), in the search for a more ecological measure, is spreading among sports professionals with the aim of improving the sports performance of cyclists. The kinematic evaluation using the Leomo system (TYPE-R, Leomo, Boulder, CO, USA) has become popular. Purpose: The present study aimed to evaluate the reliability and validity of the Leomo system by measuring the angular kinematics of the lower extremities in the sagittal plane during pedaling at different intensities compared to a gold-standard motion capture camera system (OptiTrack, Natural Point, Inc., Corvallis, OR, USA). Methods: Twenty-four elite cyclists recruited from national and international cycling teams performed two 6-min cycles of cycling on a cycle ergometer at two different intensities (first ventilatory threshold (VT1) and second ventilatory threshold (VT2)) in random order, with a 5 min rest between intensity conditions. The reliability and validity of the Leomo system versus the motion capture system were evaluated. Results: Both systems showed high validity and were consistently excellent in foot angular range Q1 (FAR (Q1)) and foot angular range (FAR) (ICC-VT1 between 0.91 and 0.95 and ICC-VT2 between 0.88 and 0.97), while the variables leg angular range (LAR) and pelvic angle showed a modest validity (ICC-VT1 from 0.52 to 0.71 and ICC-VT2 between 0.61 and 0.67). Compared with Optitrack, Leomo overestimated all the variables, especially the LAR and pelvic angle values, in a range between 12 and 15°. Conclusions: Leomo is a reliable and valid tool for analyzing the ranges of motion of the cyclist’s lower limbs in the sagittal plane, especially for the variables FAR (Q1) and FAR. However, its systematic error for FAR and Pelvic Angle values must be considered in sports performance analysis.


Introduction
Biomechanics applied to cycling has undergone great evolution in recent decades, both in the analysis techniques and in the technological systems used. This evolution has allowed the passage of traditional techniques for different bicycle adjustments such as the inseam leg length measurement multiplied by a correction factor [1], the use of static goniometers to measure knee angle [2], or 2D and 3D motion capture systems as a gold standard [3]. All these techniques mentioned above start from an important limitation, isolating the cyclist from the ecological context of their sports practice and leading them to develop their capabilities under laboratory conditions.

•
We checked the validity and reliability of a well-known and commonly used IMU in a cycling performance context in the search of a more ecological situation.

•
We compared the agreement of the IMU with a gold-standard motion capture system. • We described the bias of the IMU system, and we give several recommendations to mitigate the lack of agreement between systems to allow coaches to interpret the results in a performance context.
Next, in Section 2, the materials and methods used in this study are presented, while, in Section 3, the results are presented through tables and figures to facilitate their understanding. Next, in Section 4, the results of the present study are discussed with previous studies, and the manuscript ends with a section in which the main results are synthesized in the form of conclusions of the study.

Participants
The Human Research Ethics Committee of the University of Valencia approved the study (registry number: 20390). Twenty-four male elite cyclists were recruited from national and international cyclist club teams in December 2020. Inclusion criteria were: to be selected by the National Cyclist Federation, to be injuries-free in the last half year; not to be taking medication that alters normal cycling; not to suffer musculoskeletal disorders, neurological disorders, or heart failure that could affect normal pedaling. Cyclists were excluded if they have had significant injury, illness, or surgery within the previous six months. Before their inclusion in the study, all participants provided informed consent. A sample size calculation was performed based on the related sample comparison design, using the G-Power 3 software (version 3.1.9.7, Düsseldorf, Germany). This analysis indicated that at least a sample of 16 cyclists was required to detect significant differences in the different variables analyzed with a minimum detectable effect size of f = 1 (large) (α = 0.05, β = 0.05, and power = 0.96).

Experimental Designs
Before testing sessions, the anthropometric characteristics of the cyclists were taken by a bioelectrical impedance analysis (BIA) system, dual-frequency (50 kHz and 6.25 kHz) (Tanita BC-545, Tanita Europe BV, Amsterdam, The Netherlands). Cyclists were required to stand barefoot on metal electrodes while grasping two electrodes fixed on a handle with the arms extended. The percentage of body fat was calculated using all scales. Anthropometric measurements were performed by the same trained researcher who followed the device manufacturers' recommendations. The anthropometric characteristics of the sample are shown in Table 1. Lower-limb angular kinematic data during cycling were recorded using an optoelectronic three-dimensional (3D) camera system (Optitrack V120:Trio, NaturalPoint, Inc., Oregon, OR, USA), at a sampling frequency of 120 Hz, which was placed perpendicularly 7 m away from the sagittal plane of the cyclist. Simultaneously, a system consisting of a set of five inertial-measurement-unit (IMU) motion-tracking lightweight sensors (12 g) (Type-S Motion Sensor, Leomo, Inc., Boulder, Colorado, CO, USA), placed on the skin of cyclists and sampling at 100 Hz, was used.
Validity and reliability were evaluated at two common training intensities. The power, measured in watts (W), developed at aerobic ventilatory thresholds (first ventilatory threshold (VT1) and second ventilatory threshold (VT2)) were calculated by an incremental ramp test [20] 48 h before testing. Then, cyclists performed two bouts of 6 min pedaling at different intensities (VT1 and VT2) in a randomized order, with a 5 min rest between intensity conditions to avoid fatigue interference. In each intensity condition, two 30 s independent datasets were taken, separately by at least 2 min during the last four minutes at a controlled cadence of ±90 rpm and with their hands on top of the handlebars throughout the testing session ( Figure 1). The first repetition intended to assess the validity of the IMU sensor system (Leomo) for angular kinematics assessment versus a gold-standard for angular kinematics assessment, an optoelectronic 3D camera system (Optitrack); the second repetition intended to test the reliability. Both cycling kinematics' measurement repetitions were undertaken in the biomechanics lab under the same environmental conditions and at the same time of the day. Rate of Perceived Exertion (RPE) [21] was also registered after each cycling intensity condition (VT1 and VT2) and each repetition (REP 1 and 2).
Lower-limb angular kinematic data during cycling were recorded using an optoelectronic three-dimensional (3D) camera system (Optitrack V120:Trio, NaturalPoint, Inc., Oregon, OR, USA), at a sampling frequency of 120 Hz, which was placed perpendicularly 7 m away from the sagittal plane of the cyclist. Simultaneously, a system consisting of a set of five inertial-measurement-unit (IMU) motion-tracking lightweight sensors (12 g) (Type-S Motion Sensor, Leomo, Inc., Boulder, Colorado, CO, USA), placed on the skin of cyclists and sampling at 100 Hz, was used.
Validity and reliability were evaluated at two common training intensities. The power, measured in watts (W), developed at aerobic ventilatory thresholds (first ventilatory threshold (VT1) and second ventilatory threshold (VT2)) were calculated by an incremental ramp test [20] 48 h before testing. Then, cyclists performed two bouts of 6 min pedaling at different intensities (VT1 and VT2) in a randomized order, with a 5 min rest between intensity conditions to avoid fatigue interference. In each intensity condition, two 30 s independent datasets were taken, separately by at least 2 min during the last four minutes at a controlled cadence of ±90 rpm and with their hands on top of the handlebars throughout the testing session ( Figure 1). The first repetition intended to assess the validity of the IMU sensor system (Leomo) for angular kinematics assessment versus a goldstandard for angular kinematics assessment, an optoelectronic 3D camera system (Optitrack); the second repetition intended to test the reliability. Both cycling kinematics' measurement repetitions were undertaken in the biomechanics lab under the same environmental conditions and at the same time of the day. Rate of Perceived Exertion (RPE) [21] was also registered after each cycling intensity condition (VT1 and VT2) and each repetition (REP 1 and 2). A randomized design protocol was used to determine the intensity condition order. For this purpose, opaque envelopes were used for allocation concealment [22]. As many envelopes as possible orders of the study conditions were prepared with the sequence of study conditions (two possible order conditions). These envelopes were placed, unmarked and unidentified, on the table in the laboratory and the cyclists randomly selected an envelope, determining the sequence of study conditions.
All tests were carried out using the same cycle ergometer (Wattbike Pro, Wattbike LTD, Nottingham, UK). Before testing, the geometric measurements of cyclists' own bicycles were taken with the use of an anthropometric tape [23] to fit the cycle ergometer with the same bicycle configuration. A randomized design protocol was used to determine the intensity condition order. For this purpose, opaque envelopes were used for allocation concealment [22]. As many envelopes as possible orders of the study conditions were prepared with the sequence of study conditions (two possible order conditions). These envelopes were placed, unmarked and unidentified, on the table in the laboratory and the cyclists randomly selected an envelope, determining the sequence of study conditions.
All tests were carried out using the same cycle ergometer (Wattbike Pro, Wattbike LTD, Nottingham, UK). Before testing, the geometric measurements of cyclists' own bicycles were taken with the use of an anthropometric tape [23] to fit the cycle ergometer with the same bicycle configuration.
Before the warm-up, cyclists were instrumented with six retroreflective markers to the Optitrack system and with the five Leomo sensors. To avoid marker loss or Leomo sensors loss due to the cyclist's sweat during pedaling tests, a complete fixation protocol was followed [24] and the Leomo sensors were protected under the cyclist's clothes.
Via previous sensor synchronization via Bluetooth, Leomo sensors were fixed, following the brand recommendations, at the sacrum, right above the tail bone; right and left thigh, centered and 10 to 15 cm from the top of the patella; right and left foot, on the cycling shoes' laces closest to the tip of each foot using a sensor clip ( Figure 2).
Before the warm-up, cyclists were instrumented with six retroreflective markers to the Optitrack system and with the five Leomo sensors. To avoid marker loss or Leomo sensors loss due to the cyclist's sweat during pedaling tests, a complete fixation protocol was followed [24] and the Leomo sensors were protected under the cyclist's clothes.
Via previous sensor synchronization via Bluetooth, Leomo sensors were fixed, following the brand recommendations, at the sacrum, right above the tail bone; right and left thigh, centered and 10 to 15 cm from the top of the patella; right and left foot, on the cycling shoes' laces closest to the tip of each foot using a sensor clip ( Figure 2).

Data Analysis
The XYZ Cardan sequence of rotations was used to calculate angular kinematics, and the 3D reconstruction accuracy was calculated by the root-mean-square error (RMSE). Results showed a systematic error for X (mediolateral), Y (anteroposterior), and Z (vertical) axes of 0.005, 0.012, and 0.037 mm, respectively. Conventions used for lower-extremity angles. White circle: retroreflective markers; yellow squares: Leomo sensors; 1: posterior superior iliac spine marker (PSIS); 2: anterior superior iliac spine marker (ASIS); 3: greater trochanter marker; 4: femoral condyle marker; 5: lateral malleolus marker; 6: 5th meta-tarsal head marker; LAR: Leg Angular Range; FAR: Foot Angular Range. Blue shadow represents the angular range measured by Leomo system; red shadow represents the angular range measured by Optitrack system. Before the kinematic assessment, a familiarization with the testing condition and a self-selected speed warm-up (10 min) were carried out [25].

Data Analysis
The XYZ Cardan sequence of rotations was used to calculate angular kinematics, and the 3D reconstruction accuracy was calculated by the root-mean-square error (RMSE). Results showed a systematic error for X (mediolateral), Y (anteroposterior), and Z (vertical) axes of 0.005, 0.012, and 0.037 mm, respectively.
To process the data, Motive software (NaturalPoint, Inc., Oregon, OR, USA) was used. In each repetition, a minimum of 45 complete pedaling cycles were registered approximately during the 30 s of recording. A fourth-order low-pass Butterworth filter with a cut-off frequency of 6 Hz was used to filter marker data [23].
Angular kinematics parameters were calculated using a custom routine performed with the MatLab R2020b program (Mathworks Inc., Natick, MA, USA).
To obtain the pelvic angle, the leg angular range (LAR), and the foot angular range during the first 90 • of the pedaling cycle (FAR(Q1)) and during the whole cycle (FAR), the angle convention shown in Figure 2 was used. Therefore, the absolute angle of the sacrum with respect to the horizontal represented the Pelvic Angle. LAR represented the range of movement of the absolute angle of the thigh, calculated with the greater trochanter marker and the femoral condyle marker, during the whole cycle (LAR). The FAR represented the range of movement of the absolute angle of the foot, calculated with the lateral malleolus marker and the 5th meta-tarsal head marker, during the first 90 • of the cycle (FAR(Q1)) or during the whole cycle (FAR). To define the anatomical position, the angular positions of body segments in a standing calibration trial were analyzed.

Statistical Analysis
Bland-Altman plots were used to check the agreement between systems for each variable. Differences in each variable between Leomo and Optitrack were plotted against the mean results [26]. Differences and the association between systems were assessed by the mean of a t-test and Pearson's correlation coefficient, respectively. A two-way, randomeffects, single-measure intraclass correlation coefficients model was performed to check reliability. According to previous studies [27], the standard error of measurement (SEM) and minimum detectable change values (MDCs), in combination with the ICC values, were calculated to assess the simultaneous validity between the Optitrack and Leomo systems, in addition to the within-systems test-retest reliability and measurement error over the two testing repetitions for all outcome measures [28]. ICCs were interpreted as: excellent (0.75-1), modest (0.4-0.74), or poor (0-0.39) [29]. SPSS Statistics (SPSS Inc. Version 26.0, Chicago, IL, USA) was used to carry out the statistical analyses. The equations reported previously by Jacobson and Truax [30] were used to calculate the MDC, which is also known as the reliable change index score, which is expressed as the percentage test-retest change in the angular kinematics parameter required to find a significant difference at an alpha level of 0.05 based on the Repetition 1 mean value.

Bland-Altman Plots
All participants successfully completed the two intensities and the two repetition measurements. The Bland-Altman plots for the Foot Angular Range (Q1), Foot Angular Range, Leg Angular Range, and Pelvic Angle during the VT1 intensity condition are provided in Figure 3. During the VT1 intensity condition, there was a small relationship between the difference and the mean for all the variables, except for pelvic angle parameter, which showed a low to moderate relationship (R 2 = 0.4199). Likewise, all variables were overestimated (p = 0.000) with the Leomo system by a mean of 2.1 • ± 1.8 • for FAR(Q1), 3.6 • ± 1.3 • for FAR, 14.8 • ± 5.1 • for LAR, and, finally, 13.5 • ± 9.8 • for Pelvic Angle. provided in Figure 3. During the VT1 intensity condition, there was a small relationsh between the difference and the mean for all the variables, except for pelvic angle param ter, which showed a low to moderate relationship (R 2 = 0.4199). Likewise, all variabl were overestimated (p = 0.000) with the Leomo system by a mean of 2.1° ± 1.8° f FAR(Q1), 3.6° ± 1.3° for FAR, 14.8° ± 5.1° for LAR, and, finally, 13.5° ± 9.8° for Pelvic Ang The Bland-Altman plots during the VT2 intensity condition are provided in Figu 4. The behavior observed during the VT2 intensity condition was very similar to that o served in VT1, and all variables were overestimated (p = 0.000) with the Leomo system a mean of 1.7° ± 2.1° for FAR (Q1), 3.7° ± 1.3° for FAR, 15.1° ± 4.5° for LAR, and, final 12.8° ± 10.1° for Pelvic Angle, respectively. The Bland-Altman plots during the VT2 intensity condition are provided in Figure 4. The behavior observed during the VT2 intensity condition was very similar to that observed in VT1, and all variables were overestimated (p = 0.000) with the Leomo system by a mean of 1.7 • ± 2.1 • for FAR (Q1), 3.7 • ± 1.3 • for FAR, 15.1 • ± 4.5 • for LAR, and, finally, 12.8 • ± 10.1 • for Pelvic Angle, respectively.

Validity and Reliability
The results for the FAR (Q1), FAR, LAR, and Pelvic Angle during VT1 and VT2 intensity conditions are provided in Tables 3 and 4, respectively. The results demonstrated that all variables measured with the Leomo system showed a bias toward higher values compared with the Optitrack system at any of the intensities (VT1 and VT2).

Validity and Reliability
The results for the FAR (Q1), FAR, LAR, and Pelvic Angle during VT1 and VT2 intensity conditions are provided in Tables 3 and 4, respectively. The results demonstrated that all variables measured with the Leomo system showed a bias toward higher values compared with the Optitrack system at any of the intensities (VT1 and VT2).  At VT1 and VT2, both systems showed excellent test-retest reliability (Tables 2 and 3), with only the Leg Angular Range values being measured with the Optitrack system (VT1: ICC = 0.88 and VT2: ICC = 0.80, respectively), reducing slightly the level but reaching an ICC value over 0.75, considered as an excellent value. Concurrent validity was shown to be consistently excellent across FAR (Q1) and FAR variables and repetitions for VT1 (ICC = 0.91-0.95) and for VT2 (ICC = 0.88-0.97). Modest validity (VT1: ICC = 0.52-0.71 and VT2: ICC = 0.61-0.67) was shown on the LAR and Pelvic Angle, reaching the highest mean differences between systems, ranging between 12.0 and 15.0 • difference. The SEM for all variables ranged from 1.46 to 3.92% in the Optitrack system, from 0.57 to 2.10% in the Leomo system at VT1, from 1.63 to 4.49% in the Optitrack system, and from 1.15 to 2.56% in the Leomo system at VT2, showing a good reliability for both systems. FAR (Q1) and FAR for both systems showed the best results related to SEM values, ranging from 1.29 to 1.86% at VT1.
The MDC in all variables ranged from 1.44 to 5.3% for the Optitrack system and from 0.89 to 3.6% for the Leomo system at VT1, whereas, at VT2, it ranged from 1.54 to 6.2% for the Optitrack system and from 1.79 to 2.8% for the Leomo system. The MDCs were slightly higher for both systems in the Pelvic Angle variable (5.3% at Optitrack and 3.6% at Leomo), and only in the LAR for the Optitrack System (4.1%). At VT2, MDC values were slightly higher at Pelvic Angle and LAR in the Optitrack system (6.2 and 4.9%, respectively). With respect to the other variables (FAR (Q1) and FAR), the MDCs were lower and similar in between for both systems (lower than 2%) at VT1, and between 1.7 and 2.4% (at VT2 on average) at Optitrack and Leomo systems, respectively.

Discussion
Thanks to the advancement of technology, cycling biomechanics has evolved notably in recent years, enabling the development of portable technology such as IMU devices, which allow us to measure the movement of the cyclist in their daily sports practice environment. These new devices are promising and open a wide range of possibilities, and although there are already several studies that have demonstrated the strength of IMU technology to measure joint kinematics [8,31], incorporating prior information and assumptions is necessary before drawing clinical decisions [14], and the validity and reliability of each device must be individually contrasted.
For this purpose, we carried out this study using five IMUs placed in the shoes, thighs, and sacrum of twenty-four cyclists with the objective of evaluating the reliability, validity and sensitivity of the angle measurements reported by the LEOMO system in comparison with a 3D photogrammetry motion capture system considered as the gold standard.
Leomo is a novel device, based on IMUs technology, that aims to provide real-time information on the position of the cyclist's body segments during pedaling, and the fact that, for the first time, this information can be obtained in real time and in an ecological context highlights the importance of knowing the validity and reliability of the Leomo device. This fact will open a very important window to the knowledge of the angular kinematics in relation to the variables related to the internal and external load (i.e., heart rate, power, and torque) that can affect the performance of cyclists. Our results show that Leomo is a valid, sensitive, and reliable device when compared to a gold standard (Optitrack) to measure the lower-limbs angular kinematics in the sagittal plane while pedaling. Despite this, it should be considered that Leomo overestimates all the variables, especially the values for LAR and Pelvic Angle, in a range between 12 and 15 • , respectively, at both intensities analyzed. These discrepancies were also reported by Dahl et al. [32], although they indicated a 5 • smaller difference for the sagittal plane by the IMU device used in their study. These differences could be slightly corrected if a previous calibration protocol is applied [33]. However, this is not possible with Leomo, so, knowing this overestimation, the coaches and biomechanics can "correct" it when interpreting the results in their field interventions.
Regarding validity, our results indicate that both systems have high validity and are consistently excellent across FAR (Q1) and FAR, with both variables having an ICC ranging between 0.91 and 0.95 for the VT1 intensity condition and between 0.88 and 0.97 for the VT2 intensity condition. Conversely, the LAR and Pelvic Angle variables showed a modest validity, with the ICC ranging from 0.52 to 0.71 for the VT1 intensity condition and between 0.61 and 0.67 for the VT2 intensity condition. Our results partially agree with those achieved by Cho et al. [33] who reported ICC values greater than 0.938 for the ankle, knee, and hip flexion extension variables during walking.
The SEM ranged from 1.46 to 3.92% at VT1 and from 1.63 to 4.49% at VT2 in the Optitrack system, and from 0.57 to 2.10% at VT1 and from 1.15 to 2.56% at VT2 in the Leomo system, showing a good reliability for both systems. FAR (Q1) and FAR for both systems showed the best results related to SEM values, ranging from 1.29 to 1.86% at VT1. Furthermore, the data show MDC from 1.44 to 6.2% for the Optitrack system and from 0.89 to 3.6% for the Leomo system. In the case of Leomo, this presents a very high reliability in the variables FAR (Q1) and FAR, with a very low MDC (lower than 2.4%), which indicates that they are the best variables to use in the pedaling analysis, which is of great interest considering that the ankle joint must optimize its stiffness and maximize the effective transmission of mechanical energy to the crank for optimal pedaling [34].
As the main limitations of our study, we must be aware that the reliability and validity of this device outside the sagittal plane has not been explored, and other studies should delve into it. In other types of exercise different from pedaling, they have demon-strated reliability and validity [32]; however, there are other works such as the review of Poitras et al. [31] in which they speak of a lower validity for the measurement of abduction/adduction movements with respect to flexion movements. Chia et al. [35], despite supporting their reliability and validity, suggest the need for continuous validation and improvement of IMU systems, as well as a methodological improvement to further reduce measurement errors. Therefore, future research needs to explore the validity and reliability of measurements collected by Leomo outside the sagittal plane. Another aspect that is crucial in the quality of data is related with synchronization of the sensors with the central unit. Improving the protocol not only reduces the delay between sensor information, but also reduces energy expenditure [36,37]. It has been demonstrated that all systems had a delay depending on the synchronization protocol followed by each system [37], but it has been demonstrated that several delays do not represent a problem if all the sensors have the same delay, and it is recommended that systems have a common time reference [37]. Regardless, Leomo sensor's internal data protocols are confidential, but the authors believe that they follow a time-based synchronization protocol. Future studies should analyze the effect of the possible delay with the kinematic response obtained using the analyzed system.

Conclusions
In summary, according to our results, Leomo is a reliable and valid tool to analyze the ranges of motion of the cyclist's lower limbs in the sagittal plane, especially for the variables FAR (Q1) and FAR. However, even though its error is systematic, it must be considered that for the LAR and pelvic angle values, Leomo overestimates them between 12 • and 15 • . Therefore, it is necessary to improve the algorithms that these devices use to extract the data, and the results of these variables should be interpreted with caution.