1. Introduction
Locomotor diseases associated with lameness are among the most frequent disorders affecting horses. Hence, they represent a major part of the working time and incomes of equine veterinarians [
1,
2]. Even for the most experienced veterinarians, the evaluation of lameness remains subjective and the diagnosis complicated [
3,
4,
5]. Subjectivity increases when the lameness is slight [
6], and some bias in the evaluation makes the repeatability of the visual examination questionable, as with the expectation of improvement after nerve block [
7].
The will to move towards a more objective evaluation allowed the rise of several tools for gait analysis. Stationary force plate and Motion Capture are the most precise and accurate methods, but they cannot be used in current practice, in the field [
8]. The needs of veterinary practitioners are for a portable, trustworthy, fast and easy tool to use in the field. Inertial Measurement Units (IMUs) appear then as the most suitable alternative [
9].
For instance, EQUISYM® is a tool designed to assist veterinarians in the daily clinical diagnosis of locomotor disorders in horses. It provides quantified locomotion data and guides the diagnosis of lameness with objective indicators based on the calculation of asymmetries between left and right limbs during locomotion at the trot. In this context, the system can be used by different veterinarians at different times. In order to be able to compare results with each other, to ensure long-term locomotor monitoring or to carry out multi-centre studies, it is necessary to verify that the system is capable of giving an equivalent measurement regardless of the operator using it, including novices.
For this purpose, the repeatability and reproducibility of the system, among other factors, must be assessed. The repeatability condition of measurement is defined as “condition of measurement, out of a set of conditions that includes the same measurement procedure, same operators, same measuring system, same operating conditions and same location, and replicate measurements on the same or similar objects over a short period of time” and the reproducibility condition of measurement as “condition of measurement, out of a set of conditions that includes different locations, operators, measuring systems, and replicate measurements on the same or similar objects” [
10].
IMUs-based systems for quantifying lameness in horses have been assessed by showing the correlation or the concordance with Motion Capture systems used as a gold standard [
11,
12,
13,
14,
15]. However, few studies aimed to assay the repeatability of the measurements. For instance, Keegan and al. studied an IMUs-based system repeatability in an uncontrolled environment on 236 horses [
16]. After instrumentation of each horse, they were trotted twice at five minutes’ intervals in a straight line, in various places and surface conditions. Another study aimed to determine repeatability of gait variables such as the symmetry of several segments (metacarpal and metatarsal region, hind limb asymmetry…) and joint range of motion in the sagittal and coronal planes, during trotting under a controlled treadmill exercise [
17]. In this study, ten non-lame horses were instrumented and then trotted three times a day, three days a week, over a three-week period. The repeatability was considered as high for most of the variables. A third study evaluated the influence of biological variations over time by trotting fourteen thoroughbred horses in a straight line once a day for five consecutive days, and then once a week for five consecutive weeks while horses were in race training [
18]. Daily and weekly repeatability was lower than in the previous study [
17]. Mainly, horses were of varying training and movement asymmetry levels and it was not possible to dissociate the effects of the sensors placement from the biological variability. None of these studies have evaluated the inter-operator reproducibility. To the best of our knowledge, it has not been studied yet.
In this context, the goal of our study was to focus on the effect of IMUs placement by different operators in order to assess reproducibility and to verify that the system can be used reliably regardless of the user’s level of expertise, under all the conditions of the locomotor examination (straight line and circles).
The objective of our study was then to evaluate the repeatability and the inter-operator reproducibility of a gait analysis system, EQUISYM®, in a standardised environment, on a straight line and in circles, with different users. It was hypothesised that the variance attributed to the operator is negligible compared to the total variance.
2. Materials and Methods
2.1. Horses
Three trotters from the Centre d’Imagerie et de Recherche sur les Affections Locomotrices Equines (CIRALE) were included: a 13 years old gelding, a 14 years old mare and a 27 years old gelding. These three horses were not on any training beyond the experiment and no subjective gait change was noticed over the three days of experiment.
2.2. Instrumentation
Horses were equipped with EQUISYM
®, composed of 7 wireless IMUs (tri-axial accelerometer ±16 × gravity, tri-axial gyroscope ±2000 deg/s) placed on the head, the withers, the pelvis, and the 4 cannon bones (
Figure 1). They recorded at 200 Hz during a minimum of 20 trot strides on straight line and in circles.
2.3. Protocol
The facilities were composed of a straight line of 30 m and a circle of 14 m in diameter. Four different operators were selected: two of them had never placed the sensors before, whereas the other two operators were used to doing it. A demonstration was given for the operators who had never placed it before the experiment. Each operator placed the sensors four times on each horse along three consecutive days, which led to a total of 48 repetitions randomly assigned. To avoid the habituation effect, each operator waited at least one hour between two consecutive positionings. When the measurement system was placed, the horse was trotted in-hand on a hard surface, firstly in a left rein circle, then in a right rein circle and finally in a straight line. The protocol was intended to mimic the standardised conditions of a locomotor examination. The mean +/− standard deviation (SD) of the recorded strides was distributed as following: 29.5 +/− 5.5 strides on straight line, 32.1 +/− 9.5 on right rein circle, 36.5 +/− 9.4 on left rein circle.
The handler was the same person during the three days of the experiment. Each horse performed a maximum of three successive repetitions in order to limit the influence of fatigue. Between two sessions of three repetitions, each horse had a minimum break of one hour.
2.4. Data Processing
Data were analysed using custom-written Matlab2020a (The MathWorks, Natick, MA, USA) scripts. Stance sequences were selected based on sequences of the cannon bones gyroscopic signal. Foot-on and foot-off were determined from cannon bones IMUs’ signals thanks to the method developed by Hatrisse and al. [
19]. The dorso-ventral acceleration signal was integrated twice to obtain displacement values, and was high-pass filtered using a fourth-order Butterworth filter with a cut-off frequency set to 1 Hz. The choice of this cut-off frequency was made after checking that the high-pass filter suppresses the slow drift but does not distort the shape of the signal, following the recommendations of Serra Bragança et al. [
20]. Vertical displacements of the head, withers and pelvis were segmented into strides. One stride was defined as the time between two consecutive foot-on of the left forelimb.
From the vertical displacement of the head (-H), withers (-W) and pelvis (-P) occurring along a stride, 4 variables were calculated for each sensor location. The following Asymmetry Indexes (AI), expressed as percentage of the maximal range of motion within a stride, were used to compare left vs. right part of the stride [
21]: AI-Min was the left-right difference of the lowest altitude point; AI-Max was the left-right difference of the highest altitude point, AI-up was the left-right difference of the upward range of motion during the propulsion phase, AI-down was the left-right difference of the downward range of motion during the damping phase, and AI-stance-duration was the left-right difference of the duration of the stance phase. Positive AI value indicated a weaker movement amplitude or a shorter stance phase duration (AI-stance-duration) during the right stance than during the left stance, and negative AI value indicated the opposite.
2.5. Statistical Analysis
To evaluate the repeatability and the inter-operator reproducibility, for each AI presented above and each condition (i.e., straight line, left and right rein circle), the total variance, as well as the corresponding SD, and the variance attributable to the operator were calculated. To do this, a model of analysis of variance was computed with operators and horses as random effects [
22]. As the rank of repetition and the experience of the operator could also be a source of variation, the model was adjusted on these two factors as fixed effects. The statistical analysis was performed with the RStudio software 4.0.3 (RStudio Inc., Boston, MA, USA).
3. Results
Straight line results are shown in
Table 1 and
Figure S1. In a straight line, the total variance was the highest for the index AI-Up-H (723.3) and the lowest for AI-Stance-Duration-HIND (11.1). Total variance and SD were higher for head-related AIs than for the withers or pelvis. The SD ranged between 18.6% and 26.9% for head-related AIs, and between 3.3% and 14.2% for pelvis and withers-related AIs. For all indexes, the percentage of variance attributable to the operator was close to 0, with a maximum value of 2.1% for AI-Stance-Duration-FRONT.
Left rein circle results and right rein circle results are presented in
Table 2 and
Figures S2 and S3. In both left and right rein circles, total variance was maximum for AI-Up-H (respectively 1448.2 and 2089.4), and minimum for AI-Stance-Duration-FRONT (respectively 18.1 and 36.7). As in a straight line, the total variance and SD were higher for head-related AIs. The SD varied in a similar range of values, respectively between 15.0% and 38.1% in the left rein circle and between 13.5% and 45.7% in the right rein circle for head-related AIs, and between 4.2% and 26.2% in the left rein circle and between 6.3% and 31.3% in the right rein circle for pelvis and withers-related AIs. AI-Down was systematically associated with a larger variance than the other AIs calculated for the withers and the pelvis.
Similar to the straight line, the variance attributable to the operator in the circle was close to zero for most of the AIs and reached a maximum of 4.6% of the total variance for AI-Stance-Duration-HIND in the left rein circle, and a maximum of 5.3% of the total variance for AI-Max-H in the right rein circle.
4. Discussion
The purpose of this study was to evaluate repeatability and reproducibility between operators of an IMU-based gait analysis system used to quantify asymmetry indexes in horses trotting in a straight line and circles under the same circumstances as a standard locomotor clinical exam. In our study, the variance attributable to the operator was very low, which tends to validate our starting hypothesis: the asymmetry indexes are repeatable when one or several operators set up the sensors several times, when the horse trots on a hard surface, in a straight line or in a circle.
In this study, the variance attributable to the operators was slightly higher on a right rein circle, for head-related indexes (up to 5.3% of total variance). No obvious explanation can be attributed to this observation except than that the effect of a small misplacement of the sensors by the operator may have been slightly higher on the circle than for the straight line.
As slight biological variations may be clinically relevant and many operators may be required to use the system, including those with little training, assessing reproducibility is essential to enable comparison of results obtained at different times, by different operators. Our results show that on the 42 asymmetry indexes measured (14 in a straight line, 14 in a left hand circle and 14 in a right hand circle), the variance attributable to the operator counted for less than 1% of the total variance in 81% of cases. For each exercise, the head-related indexes showed the highest SD (between 13.5% and 45.7%). In contrast, the withers-related variables showed the lowest SD (between 4.2% and 23.4%). The index AI-Down was associated with a larger variance than the other AIs for the withers and for the pelvis, indicating a higher variability of this index and thus a more complex interpretation.
Observed values for SD for the straight line were close to the values calculated from a group of 49 sound horses with the same system [
21]. SD values for the left and right rein circles were slightly higher. It has been shown that moving in circles induces a movement asymmetry which depends on speed and circle radius [
23]. In our experiment none of these factors were strictly controlled, knowing that we wanted to remain in the same conditions as the standard clinical examination. This could explain the increased SD in circles, with a higher level of freedom, enabling the horse to change its body direction and speed [
23,
24]. It is also known that asymmetry on a circle is physiologically higher than in a straight line.
Total variance was higher for head related indexes than for the withers and pelvis, for the straight line and circles. Similar results have been found in previous experiments, in which head variability was higher than pelvis and withers variability [
18,
24]. This is probably due to a wider degree of freedom for the head, resulting in multiple head movements caused by external stimuli such as other horses passing by, or noises. These uncontrolled head movements during some of our experiments are likely to explain the increase in total variance for indexes related to the head. This result supports the use of a sensor on the withers, which presents a much lower variability when exploring forelimb lameness.
After adjustment of our statistical model on the level of expertise of the operator and on the rank of repetition, nothing indicates that one of them had a significant effect. Hence, no confounding could have been made between the level of expertise of the operator and the variability induced by the operators.
The area of the experiment was fixed in our study, and our hypotheses were tested only on a hard surface. A similar study should be done on a soft surface to extrapolate our findings. The reproducibility should also be studied in different places. Furthermore, asymmetry of the included horses has been considered steady during the three days of the experiment but it is obvious that the total variance measured depends not only on the measurement system but also on the inevitable biological variation of the locomotor asymmetry, whether physiological or pathological.
To date, no study had been conducted to measure the inter-operator reproducibility of IMU’s to evaluate gait asymmetry in horses. Our study provides initial evidence that this type of system can be used by different operators, including novices, without major disruption to the measurement collection. It also confirms that the variability of the measurements is higher for the head-related indexes than for the withers and pelvis related indexes. The sample size (number of horses and operators) remains a limitation of this study. Ideally, future studies should be conducted to verify that inter-operator reproducibility remains valid in larger groups and on horses with different degrees of locomotor asymmetry.
Supplementary Materials
The following supporting information can be downloaded at:
https://www.mdpi.com/article/10.3390/s22239533/s1, Figure S1: Asymmetry indexes (mean and SD) measured on the head (-H), the withers (-W) and the pelvis (-P) of three horses trotting
on a straight line by four operators (operator A: red dot, operator B: blue dot, operator C: green dot and operator D: magenta dot) repeating 12 times the measurements for each horse, after systematically repositioning the sensors at each recording. Figure S2: Asymmetry indexes (mean and SD) measured on the head (-H), the withers (-W) and the pelvis (-P) of three horses trotting
on a left rein circle by four operators (operator A: red dot, operator B: blue dot, operator C: green dot and operator D: magenta dot) repeating 12 times the measurements for each horse, after systematically repositioning the sensors at each recording. Figure S3: Asymmetry indexes (mean and SD) measured on the head (-H), the withers (-W) and the pelvis (-P) of three horses trotting
on a right rein circle by four operators (operator A: red dot, operator B: blue dot, operator C: green dot and operator D: magenta dot) repeating 12 times the measurements for each horse, after systematically repositioning the sensors at each recording.
Author Contributions
Conceptualization, I.T., C.M., S.H.-P. and H.C.; methodology, I.T., C.M., S.H.-P. and H.C.; software, C.M. and S.H.-P.; validation, L.B., P.M. and F.M.; formal analysis, S.H.-P.; investigation, I.T., C.M.; resources, L.B.; data curation, I.T., C.M.; writing—original draft preparation, I.T.; writing—review and editing, C.M., S.H.-P. and H.C.; supervision, S.H.-P., H.C., P.M. and F.M.; project administration, H.C.; funding acquisition, H.C., P.M. All authors have read and agreed to the published version of the manuscript.
Funding
The authors thank the Agence Nationale de la Recherche for the funding of the LabCom “CWD-Vetlab” (Contract ANR 16-LCV2-0002-01) and the European Union (FEDER) and the Region Normandie for funding of the project “EQUISYM” (Contract 20E01636).
Institutional Review Board Statement
The animal study protocol was approved by the Ethics Committee of COMITE D’ETHIQUE EN RECHERCHE CLINIQUE—ComERC (n◦2022-01-19).
Informed Consent Statement
Not applicable.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors thank the Centre d’Imagerie et de Recherche sur les Affections Locomotrices Equines (CIRALE).
Conflicts of Interest
The project was supported by the Agence Nationale de la Recherche (ANR) which sponsors collaborative projects between a company (LIM France) and the Ecole Nationale Vétérinaire d’Alfort (ENVA). LIM France owns and markets the EQUISYMND brand.
References
- Loomans, J.B.A. The Equine Veterinarian: Past, Present and Prospects of a Profession; Utrecht University: Utrecht, The Netherlands, 2008; ISBN 978-90-393-4927-4. [Google Scholar]
- Nielsen, T.D.; Dean, R.S.; Robinson, N.J.; Massey, A.; Brennan, M.L. Survey of the UK Veterinary Profession: Common Species and Conditions Nominated by Veterinarians in Practice. Vet. Rec. 2014, 174, 324. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Fuller, C.J.; Bladon, B.M.; Driver, A.J.; Barr, A.R.S. The Intra- and Inter-Assessor Reliability of Measurement of Functional Outcome by Lameness Scoring in Horses. Vet. J. 2006, 171, 281–286. [Google Scholar] [CrossRef] [PubMed]
- Mitchell, J.S. Lameness and Performance Evaluation in Ambulatory Practice. Vet. Clin. N. Am. Equine Pract. 2012, 28, 101–115. [Google Scholar] [CrossRef] [PubMed]
- Hammarberg, M.; Egenvall, A.; Pfau, T.; Rhodin, M. Rater Agreement of Visual Lameness Assessment in Horses during Lungeing. Equine. Vet. J. 2016, 48, 78–82. [Google Scholar] [CrossRef] [Green Version]
- Keegan, K.G.; Dent, E.V.; Wilson, D.A.; Janicek, J.; Kramer, J.; Lacarrubba, A.; Walsh, D.M.; Cassells, M.W.; Esther, T.M.; Schiltz, P.; et al. Repeatability of Subjective Evaluation of Lameness in Horses: Repeatability of Subjective Evaluation of Lameness in Horses. Equine Vet. J. 2010, 42, 92–97. [Google Scholar] [CrossRef]
- Arkell, M.; Archer, R.M.; Guitian, F.J.; May, S.A. Evidence of Bias Affecting the Interpretation of the Results of Local Anaesthetic Nerve Blocks When Assessing Lameness in Horses. Vet. Rec. 2006, 159, 346–348. [Google Scholar] [CrossRef]
- Serra Bragança, F.M.; Rhodin, M.; van Weeren, P.R. On the Brink of Daily Clinical Application of Objective Gait Analysis: What Evidence Do We Have So Far from Studies Using an Induced Lameness Model? Vet. J. 2018, 234, 11–23. [Google Scholar] [CrossRef] [Green Version]
- McCracken, M.J.; Kramer, J.; Keegan, K.G.; Lopes, M.; Wilson, D.A.; Reed, S.K.; LaCarrubba, A.; Rasch, M. Comparison of an Inertial Sensor System of Lameness Quantification with Subjective Lameness Evaluation: Comparison of Inertial System with Subjective Lameness Evaluation. Equine Vet. J. 2012, 44, 652–656. [Google Scholar] [CrossRef]
- Balazs, A. International Vocabulary of Metrology–Basic and General Concepts and Associated Terms. Chem. Int.-Newsmag. IUPAC 2008, 30, 21–22. [Google Scholar] [CrossRef]
- Keegan, K.G.; Yonezawa, Y.; Pai, P.F.; Wilson, D.A.; Kramer, J. Evaluation of a Sensor-Based System of Motion Analysis for Detection and Quantification of Forelimb and Hind Limb Lameness in Horses. Am. J. Vet. Res. 2004, 65, 665–670. [Google Scholar] [CrossRef]
- Pfau, T.; Witte, T.H.; Wilson, A.M. A Method for Deriving Displacement Data during Cyclical Movement Using an Inertial Sensor. J. Exp. Biol. 2005, 208, 2503–2514. [Google Scholar] [CrossRef] [Green Version]
- Martin, P.; Chateau, H.; Pourcelot, P.; Duray, L.; Cheze, L. Comparison Between Inertial Sensors and Motion Capture System to Quantify Flexion-Extension Motion in the Back of a Horse. Equine Vet. J. 2014, 46, 43. [Google Scholar] [CrossRef] [Green Version]
- Bosch, S.; Serra Bragança, F.; Marin-Perianu, M.; Marin-Perianu, R.; van der Zwaag, B.; Voskamp, J.; Back, W.; van Weeren, R.; Havinga, P. EquiMoves: A Wireless Networked Inertial Measurement System for Objective Examination of Horse Gait. Sensors 2018, 18, 850. [Google Scholar] [CrossRef] [Green Version]
- Sapone, M.; Martin, P.; Ben Mansour, K.; Chateau, H.; Marin, F. Comparison of Trotting Stance Detection Methods from an Inertial Measurement Unit Mounted on the Horse’s Limb. Sensors 2020, 20, 2983. [Google Scholar] [CrossRef]
- Keegan, K.G.; Kramer, J.; Yonezawa, Y.; Maki, H.; Pai, P.F.; Dent, E.V.; Kellerman, T.E.; Wilson, D.A.; Reed, S.K. Assessment of Repeatability of a Wireless, Inertial Sensor–Based Lameness Evaluation System for Horses. Am. J. Vet. Res. 2011, 72, 1156–1163. [Google Scholar] [CrossRef]
- Cruz, A.M.; Maninchedda, U.E.; Burger, D.; Wanda, S.; Vidondo, B. Repeatability of Gait Pattern Variables Measured by Use of Extremity-Mounted Inertial Measurement Units in Nonlame Horses during Trotting. Am. J. Vet. Res. 2017, 78, 1011–1018. [Google Scholar] [CrossRef]
- Caviedes, M.F.S.; Forbes, B.S.; Pfau, T. Repeatability of Gait Analysis Measurements in Thoroughbreds in Training. Equine Vet. J. 2018, 50, 513–518. [Google Scholar] [CrossRef] [Green Version]
- Hatrisse, C.; Macaire, C.; Sapone, M.; Hebert, C.; Hanne-Poujade, S.; De Azevedo, E.; Marin, F.; Martin, P.; Chateau, H. Stance Phase Detection by Inertial Measurement Unit Placed on the Metacarpus of Horses Trotting on Hard and Soft Straight Lines and Circles. Sensors 2022, 22, 703. [Google Scholar] [CrossRef]
- Bragança, F.S.; Roepstorff, C.; Rhodin, M.; Pfau, T.; van Weeren, P.R.; Roepstorff, L. Quantitative lameness assessment in the horse based on upper body movement symmetry: The effect of different filtering techniques on the quantification of motion symmetry. Biomed. Signal Process. Control. 2020, 57, 101674. [Google Scholar] [CrossRef]
- Macaire, C.; Hanne-Poujade, S.; De Azevedo, E.; Hatrisse, C.; Hébert, C.; Martin, P.; Marin, F.; Chateau, H. Movement Asymmetry Measured in Sound Horses Trotting on a Straight Line. in Abstracts 47th congress of the Society of Biomechanics. Comput. Methods Biomech. Biomed. Eng. 2022, 25, S194–S195. [Google Scholar] [CrossRef]
- Shrout, P.E.; Fleiss, J.L. Intraclass correlations: Uses in assessing rater reliability. Psychol. Bull. 1979, 86, 420–428. [Google Scholar] [CrossRef] [PubMed]
- Pfau, T.; Stubbs, N.C.; Kaiser, L.J.; Brown, L.E.A.; Clayton, H.M. Effect of Trotting Speed and Circle Radius on Movement Symmetry in Horses during Lunging on a Soft Surface. Am. J. Vet. Res. 2012, 73, 1890–1899. [Google Scholar] [CrossRef] [PubMed]
- Hardeman, A.M.; Serra Bragança, F.M.; Swagemakers, J.H.; Weeren, P.R.; Roepstorff, L. Variation in Gait Parameters Used for Objective Lameness Assessment in Sound Horses at the Trot on the Straight Line and the Lunge. Equine Vet. J. 2019, 51, 831–839. [Google Scholar] [CrossRef]
| Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).