Reliability and Validity of the Articulation Motion Assessment System Using a Rotary Encoder

Abstract

This study aimed to validate the effectiveness of the Articulation Motion Assessment System (AMAS), a joint kinematic evaluation system, for clinical applications. AMAS enables synchronised measurement using neurophysiological indicators, overcoming laboratory setting limitations. We compared AMAS-based ankle joint kinematic evaluations, particularly the sagittal and frontal plane angles, with two-dimensional (2D) motion analysis to determine the validity and reliability of AMAS. Both AMAS and 2D motion analysis reliably detected significant differences in angles within the sagittal and frontal planes. Correlation analysis revealed a significant moderate-to-strong correlation between the AMAS and the conventional method of 2D motion analysis, proving the measurement validity of the AMAS (ρ = 0.53–0.77 for sagittal plane angles; ρ = 0.46–0.72 for frontal plane angles). The average root mean squared error (RMSE) was significantly lower in AMAS (10.90 ± 2.93° for sagittal plane angles; 13.44 ± 1.09° for frontal plane angles) than in the inertial sensor-based three-dimensional (3D) motion analysis. Reliability analysis revealed high reliability of measurements (intraclass correlation coefficients (ICC) ≥ 0.76). However, the Bland–Altman analysis identified a slightly lower fixed bias, which was observed as a characteristic of each measurement system. The AMAS accurately detects ankle joint angles without being constrained by measurement environment limitations. Synchronised measurements using neurophysiological indicators potentially contribute to understanding ankle joint control mechanisms and developing rehabilitation strategies.

1. Introduction

Human walking is a uniquely acquired upright bipedal gait used to complete daily activities [1]. Walking maintains functional independence [2] and quality of life [3]. Typical clinical evaluation indexes of gait performance include gait speed, which is affected by gait pattern [4,5]. The control of the ankle joint is essential in regulating changes in walking speed by altering the gait pattern; however, age-related neurodegeneration interferes with this motor control, causing motor impairments and other neurological symptoms [6,7,8]. Therefore, assessing ankle joint control is crucial for rehabilitation, including attaining or restoring walking ability. The ankle joint control coordinates the relative timing of ankle joint torque exertion and angular displacement during changes in walking velocity [9]. It encompasses the multifaceted elements of the so-called movement in kinematics and its underlying neural mechanisms in neurophysiology. This results in numerous degrees of freedom due to anatomical, biomechanical, and neurophysiological redundancies within the interaction between the body and environment [10,11]. However, evaluating ankle joint control has been limited to one aspect of biomechanics [12,13] and neurophysiology [14] using electroencephalography and electromyography. Therefore, analysing the motor control of the ankle joint should include correlating neural activity with motor behaviour to understand the precise objectives of the central nervous system and the mechanisms by which it modulates the final output of motor actions [15,16]. Therefore, motor control should be comprehensively understood based on a unified theoretical framework [17].

Muscle synergies [18] involve the cooperative activities of multiple muscles and degrading and controlling movement. Additionally, neuromuscular control of the lower extremity by the upper motor centre (hereafter referred to as ‘lower-extremity muscle control’) [19] interacts with the body and the environment and influences muscle synergies. Understanding that lower-extremity muscle is being controlled by a nervous system-based motor control mechanism [20,21] will help clarify the neural mechanisms of movement disorders to capture the control mechanisms of various motor components [22,23,24]. Cortical–muscular coherence, a neurophysiological index for capturing neural mechanisms in lower-extremity muscle control, correlates brain activity with muscle activity [19]. It is a biomarker for movement disorders used in rehabilitation therapy [25]. However, in clinical application, cortical–muscular coherence on the affected side must be classified accurately by determining its relationship to movements hampered by ageing or disability to better understand its impact on movement disorders [26]. Therefore, neurorehabilitation for ankle joint control warrants elucidating human walking ability using the neurophysiological indices of lower-extremity muscle control. Due to its anatomical structure, ankle motion is a simultaneous and complex multi-axis movement [27], with significant individual differences and some even result in altered motion on the forefoot plane [28,29]. Therefore, we hypothesised that a new lower-extremity muscle control evaluation method combining kinematic and neurophysiological indices may capture ankle joint control, which is essential for gait.

This hypothesis requires measurements of synchronous joint motion, muscular activity, and brain activity. Therefore, a new system is necessary to measure ankle joint control kinematic and conventional neurophysiological indices and determine the relationship between each index. Electroencephalography (EEG) and electromyography (EMG) use non-invasive scalp or skin electrodes for neurophysiological analysis, which can be monitored remotely [30]. A 3D motion analysis using optical motion capture with markers is the gold standard for the kinematic analysis of walking and lower limb motion [31], despite the 3D motion analysis being expensive, requiring dedicated space [32], being time-consuming, and highly dependent on the evaluator’s skill and experience. Since it involves manually identifying specific anatomical indicators captured by multiple videos [33], the measurement technique, environment, specifications, and number of cameras affect the accuracy of joint angle measurements [34]. A 2D motion analysis, which shows moderate correlation and high concordance with a 3D motion analysis [35], can help identify kinematics [36] and is used in foot kinematic analysis [37]. However, capturing optical motions using these markers requires video cameras and a restricted measurement environment.

Inertial sensor-based 3D motion analysers offer a simple method for outdoor measurement without a video camera. However, they have measurement errors due to magnetic field effects [38,39]. A measurement method without magnetic sensors has been proposed; however, the magnetic sensor information is required during initial calibration [40] with an environment free from magnetic field effects. Therefore, establishing an evaluation method integrating kinematic and neurophysiological indices remains challenging in terms of convenience and versatility in clinical applications.

We developed a rotary encoder-based ankle joint angle measurement system and an articulation motion assessment system (AMAS) to address the aforementioned clinical problems. This rotary encoder converts the mechanical displacement of rotation into an electrical signal and processes it to detect position, speed, and angle. It measures motor displacement in patients with Parkinson’s [41] and in wearable exoskeleton robots supporting human movement [42]. Without video cameras or magnetic sensors, this system analyses the kinematics of dorsiflexion, plantar flexion, inversion, and eversion of the ankle. Additionally, AMAS synchronises with existing bio-signal processing devices that capture neurophysiological indicators, such as EEG and EMG. Synchronised ankle kinematic and neurophysiological analysis facilitates exploring the indices of these functional couplings for motor control. This enables elucidating mechanisms of ankle joint control, which is essential for walking.

This study aimed to establish the ankle motion task as a simple method of motion analysis using the newly developed AMAS and confirmed the validity and reliability of this system via a comparative validation with existing evaluations.

2. Materials and Methods

2.1. Participants

This study included 14 participants (12 men and two women; age 22 ± 1 years; height 1.70 ± 0.06 m; weight 65.07 ± 14.81 kg) enrolled between December 2022 and February 2023. The inclusion criteria were an absence of cognitive problems, motor sensory dysfunction in the lower extremities, and visual impairment. Fourteen healthy adults (twelve males and two females) participated in and completed the entire study. This study was explained verbally and in writing, and the participants provided written informed consent before participating. This study was approved by the Research Ethics Committee of Kyoto Tachibana University (Approval No. 22-13) and conducted in accordance with the Declaration of Helsinki.

2.2. Protocol

Each measurement device was attached to the participant before any measurements were collected (Figure 1a). This study used AMAS, Inertial Measurement Unit (IMU), and a 2D motion analysis to measure voluntary movements in the sagittal and frontal plane of the ankle joint. Voluntary movements often face redundancy issues [11], making it challenging to capture motor control due to the many degrees of freedom involved, especially in free and coarse movements [43,44]. Analytical methods have been employed to minimise these degrees of freedom and constrain the analysis to a single plane in two dimensions [45,46,47]. A 2D motion analysis is a widely used technique in clinical practice as an alternative to a 3D motion analysis, a costly and environmentally restricted technique [35,36,37]. It is also suitable for validation because it increases the measurement’s simplicity and reproducibility by limiting it to a single plane [48]. The present study was designed to compare the developed AMAS with the IMU, which has been used in recent years, using a 2D motion analysis as a reference and an alternative method in biomechanics and rehabilitation.

All the tasks were performed with the participant sitting on a chair with a backrest one step above the floor. The measurement tasks consisted of automatic voluntary movements of dorsiflexion, plantar flexion, inversion, and eversion of the ankle on one side. They were performed in two ranges of motion, the maximum range of motion and the mild range of motion, according to the participant’s subjective judgement. For each automatic voluntary movement, the following conditions were assigned according to the participant’s subjective movement speed: ‘movement at a comfortable speed’ and ‘movement at a slow speed’. These conditions were set to verify whether it was possible to measure a wide range of participants with mild to severe disabilities since the range of joint motion was assumed to be smaller and patients with motor dysfunction exhibited slow movement. Each participant was required to complete five consecutive measurement tasks. The measurement task sequence was randomised using a random number table. Each measuring device was calibrated simultaneously in the midfoot position. Only the heel is in contact with the floor, while the area distal to the heel is raised off the ground to maintain freedom of ankle motion. Because biomechanical and neurophysiological studies have shown that ankle motion was not lateralised [49,50], the kicking foot was determined to be the dominant foot and was used as the measurement limb. All the participants’ dominant feet were determined to be the right foot. Measurements were taken using a video camera. The centre of the video camera lens was aligned to the centre of the foot; therefore, the height was adjusted using a tripod, and the camera was positioned 1 cm from the front and rear and 1 m from the right side.

The synchro light at the start of the IMU measurement and sensor mat light input to the AMAS were used to synchronise the data. The data’s starting point was then temporally synchronised by the sensor mat signal input after initiating the AMAS measurement, and the video camera captured the sensor mat light. Synchronously measured data were corrected to align the data points, where time-series data of joint angles were recorded over five consecutive trials. All data acquired were used in this analysis, corrected for the 50 Hz sampling rate.

2.3. Instrumentation

An AMAS, reflective marker, and IMU sensor were attached to one of the participant’s legs (Figure 1b).

2.3.1. AMAS

This joint angle measurement device simultaneously measures the plantarflexion/dorsiflexion and inversion/eversion angles during ankle joint motion and consists of an ankle joint orthosis, controller, and operation application (Figure 2a).

The ankle joint unit (Figure 2b) comprises two units with a built-in rotary encoder (manufactured by Supertech Electronic, Rixin Street, Chiayi City, Taiwan). One unit was attached to the outer lower leg, which served as the fundamental axis, and the other unit was attached to the dorsal foot (metatarsal head), which served as the moving axis.

A pipe connects both units, and plantar dorsiflexion movement is transmitted to the basic axis-side unit via the connecting pipe, causing the rotary encoder to rotate. Similarly, the basic shaft-side unit supported the connecting pipe, and the inversion/eversion motion of the ankle was transmitted to the rotary encoder of the mobile shaft-side unit. The controller transmits the signal from the rotary encoder, resulting in ankle motion, to the application (tablet). The application converts the signals into angles and displays the changes in plantarflexion/dorsiflexion and inversion/eversion angles in real time. The angle data were saved as a CSV file. The system simultaneously extracts the dorsiflexion/plantarflexion and inversion/eversion angles (degrees) of the ankle. The sampling rate was 50 Hz.

2.3.2. A 2D Motion Analysis

The reference motion capture was performed using the free 2D motion analysis software Image J Version 1.54h (NIH, Bethesda, MD, USA) for a 2D motion analysis with reflective markers. The following nine anatomical landmarks were selected according to previous studies: fibular head; tibial tuberosity; posterior medial surface of the lower leg; medial anterior surface of the medial and lateral phalanges; lateral malleolus; first and fifth metatarsal heads; posterior calcaneus (CAL); and, directly above that, CAL [51]. Reflective markers were attached to the skin using double-sided tape at selected landmarks. Two 30 bits per second digital high-definition video cameras (Panasonic, Tokyo, Japan) were used to capture images from the anterior and posterior sides for the frontal plane task and the right lateral side for the sagittal plane task during the measurement task. The video captured by the video camera was clipped at 30 Hz using the free video playback software GOM Player 2.3.104.5374 (Gretech, Soul, Korea). Each data point was then extracted and corrected every 0.14 s. The captured data were used to calculate joint angles from 2D coordinates using the free 2D motion analysis software ImageJ. The dorsiflexion/plantarflexion angle of the ankle in the sagittal plane is the angle between the straight line connecting the fibular head and external capsule and the straight line connecting the fifth metatarsal head and external capsule. The inversion/eversion angle of the ankle viewed from the anterior is the angle between the lower leg axis consisting of the centre of the medial and lateral phalanges and the tibial coarse surface and the plantar axis consisting of the first and fifth metatarsal heads, based on the ‘Methods for Indication and Measurement of Range of Motion of Joints’. The angle of inversion/eversion of the ankle viewed from the posterior is the angle between the line passing from the centre of the lower leg through the centre of the Achilles tendon and the line connecting the two markers at the rear of the heel.

The joint angles (degrees) were extracted from the 2D motion analysis of sagittal plane videos, with the dorsiflexion direction being positive and the plantar flexion direction being negative. The joint angles (degrees) were extracted from the 2D motion analysis of the frontal plane video, with the inversion direction being positive and the eversion direction being negative.

2.3.3. Inertial Measurement Unit (IMU)

Two MyoMotion sensors (Noraxon USA Inc., Scottsdale, AZ, USA) were used as IMU. The tibial sensor was wrapped around the lower leg using an attached belt, and the dorsal foot sensor was attached to double-sided tape. The sampling rate was 100 Hz.

From the IMU, dorsiflexion/plantarflexion and the ankle’s internal/external return angles (degrees) were extracted using sensors placed across the ankle joint.

2.4. Data Analysis

The statistical software SPSS (version 27.0; IBM Corporation, Armonk, NY, USA) was used for statistical analysis, where the p-value was set at 5%. Corresponding t-tests were used to determine the time (s) difference between comfortable and slow conditions in the measurement task. All data were subjected to the Kolmogorov–Smirnov test of normality. The feature extraction of the AMAS, the regression model’s performance, and the measurement method’s reliability and agreement were statistically validated [52]. For assessing validity, comparisons between conditions were performed using the mean angles measured by each system and analysed with paired t-tests. In addition, the correlation coefficients between the AMAS and IMU were calculated from the time-series data of joint angles of the 2D motion analysis. Root mean squared error (RMSE) (1) was calculated as the performance index of the regression model using Equation (1) [53]:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{\sum {\left(\mathrm{X}\left(\mathrm{t}\right)-\mathrm{Y}\left(\mathrm{t}\right)\right)}^2}{\mathrm{n}}}}$$

(1)

X is the joint angle measured by the 2D motion analysis as the reference value, and Y is calculated by substituting the measured joint angle by AMAS or IMU, squaring the difference for n data and taking the square root.

The maximum value of each of the five repeatedly extracted angular data was used to evaluate reliability. A two-way mixed model by a single ICC examiner was used [54]. Bland–Altman analysis was performed to confirm the presence of systematic errors in the overall data to evaluate the agreement between the measurement methods and the characteristics of the measurement methods using the margin of error (LOA) (2) [55].

$$\mathrm{LOA}={\displaystyle \frac{1}{n}}{{\displaystyle \sum }}_{𝒾=1}^{n}d𝒾\pm 1.96\times \sqrt{{\displaystyle \frac{1}{n-1}}{{\displaystyle \sum }}_{𝒾=1}^n{\left(d𝒾-\overline{d}\right)}^2}$$

(2)

3. Results

The time taken for each condition in the measurement task was 9.22 ± 2.99 s (AVE ± SD) in the comfortable condition and 13.64 ± 6.60 s (AVE ± SD) in the slow condition, with a significant difference (t = −8.28, p < 0.001) and 95% confidence interval (CI): ([−5.49) to [−3.35]). It was confirmed that the joint angle data per measurement task extracted by each measuring device were non-normal using a normality test (p < 0.01).

3.1. Validity

3.1.1. Comparisons Between Conditions

In the sagittal plane angles, the comparison between the maximum and mild conditions revealed significant differences across all systems: a 2D motion analysis (t(13) = 9.523, p < 0.001); AMAS (t(13) = 10.498, p < 0.001); and IMU (t(13) = 2.708, p = 0.018). In contrast, for subjective motor speed, significant differences were found in a 2D motion analysis (t(13) = 4.142, p = 0.001) and AMAS (t(13) = 5.287, p < 0.001), but not in IMU (t(13) = 1.758, p = 0.102).

In the frontal plane angles, the comparison between the maximum and mild conditions showed significant differences in the 2D motion analysis performed from the front (t(13) = 6.597, p < 0.001), the 2D motion analysis performed from the back (t(13) = 4.661, p < 0.001), and AMAS (t(13) = 6.166, p < 0.001). However, no significant differences were observed in IMU (t(13) = 2.076, p = 0.058). In contrast, for subjective motor speed, significant differences were observed only in the 2D motion analysis performed from the back (t(13) = 2.816, p = 0.015). No significant differences were observed in the 2D motion analysis performed from the front (t(13) = 0.189, p = 0.853), AMAS (t(13) = −1.139, p = 0.275), or IMU (t(13) = 1.440, p = 0.173).

3.1.2. Criterion Validity

Spearman’s correlation analysis was used to compute the correlation coefficients between the 2D motion analysis, AMAS, and IMU. In the sagittal plane angle, the correlation coefficient for the AMAS was ρ = 0.45 to 0.72 (

Table 1. Spearman’s correlation coefficients and RMSE values between systems for sagittal.

					Subjective Motor Speed
					Comfortable		Slow
Activity	Conditions	System	ρ	RMSE	ρ	RMSE	ρ	RMSE
Dorsiflexion	maximum	AMAS	0.69 **	9.22	0.65 **	9.47	0.72 **	9.04
		IMU	0.44 **	17.3	0.41 **	22.23	0.47 **	12.73
	mild	AMAS	0.54 **	7.29	0.50 **	7.35	0.58 **	7.25
		IMU	0.35 **	10.19	0.28 **	10.02	0.40 **	10.31
Plantarflexion	maximum	AMAS	0.53 **	16.23	0.66 **	14.32	0.45 **	17.33
		IMU	0.47 **	21.26	0.38 **	22.34	0.54 **	20.55
	mild	AMAS	0.53 **	9.95	0.59 **	9.97	0.48 **	9.94
		IMU	0.16 **	19.66	0.17 **	17.12	0.16 **	21.23

** p < 0.01.

). In the frontal plane angle, the correlation coefficient with the 2D motion analysis performed from the front was ρ = 0.39 to 0.72 (

Table 2. Spearman’s correlation coefficients and RMSE values between systems for 2D motion analysis viewed from the front.

					Subjective Motor Speed
					Comfortable		Slow
Activity	Conditions	System	ρ	RMSE	ρ	RMSE	ρ	RMSE
Inversion	maximum	AMAS	0.69 **	14.49	0.72 **	14.64	0.68 **	14.38
		IMU	0.02	28.82	−0.08 *	27.94	0.08 **	29.42
	mild	AMAS	0.55 **	13.11	0.63 **	12.29	0.49 **	13.63
		IMU	0.03	28.43	−0.07 *	29.78	0.10 **	27.49
Eversion	maximum	AMAS	0.46 **	14.74	0.39 **	15.29	0.52 **	14.31
		IMU	0.01	35.7	0.01	38.23	0	33.72
	mild	AMAS	0.50 **	11.19	0.55 **	11.15	0.46 **	11.22
		IMU	−0.06 *	34.96	−0.02	41.86	−0.07 *	28.77

* p < 0.05, ** p < 0.01.

), and that with the 2D motion analysis performed from behind was ρ = −0.05 to 0.52 (

Table 3. Spearman’s correlation coefficients and RMSE values between systems for 2D motion analysis viewed from behind.

					Subjective Motor Speed
					Comfortable		Slow
Activity	Conditions	System	ρ	RMSE	ρ	RMSE	ρ	RMSE
Inversion	maximum	AMAS	0.48 **	17.46	0.50 **	17.8	0.48 **	17.22
		IMU	0.18 **	24.69	0.19 **	22.44	0.18 **	26.16
	mild	AMAS	0.49 **	12.89	0.52 **	12.75	0.49 **	12.98
		IMU	0.31 **	26.32	0.23 **	27	0.35 **	25.85
Eversion	maximum	AMAS	0.07 **	12.02	0.04	12.5	0.11 **	11.65
		IMU	0.02	27.49	0.13 **	28.7	−0.09 **	26.55
	mild	AMAS	−0.05 *	11.83	−0.07 *	12.49	−0.04	11.32
		IMU	0.04	28.17	0.11 **	34.04	−0.03	22.84

* p < 0.05, ** p < 0.01.

). The correlation coefficient with the 2D motion analyses performed from the sagittal and front indicated a more than moderate correlation, while those with the 2D motion analysis from behind indicated a low correlation.

In the sagittal plane angle, the correlation coefficient for the IMU was ρ = 0.16 to 0.54 (p < 0.01). In the frontal plane angle, the correlation coefficient with the 2D motion analysis performed from the front was ρ = −0.08 to 0.10, and that with the 2D motion analysis performed from behind was ρ = −0.09 to 0.35 (Table 3). The correlation coefficient with the 2D motion analysis performed from the sagittal indicated a more than low correlation, while those with the 2D motion analyses from the front and behind indicated low correlations.

3.1.3. Root Mean Squared Error (RMSE)

The mean and standard deviation of RMSEs with each measurement device (AMAS and IMU) for the 2D motion analysis were calculated. The RMSE of the AMAS in the sagittal and frontal plane angle was lower than that of the IMU (Table 1, Table 2, Table 3 and Table S1). The RMSEs of the AMAS and IMU significantly differed in the t-test (p < 0.01).

3.2. Reliability

The ICC [56] was used to evaluate reliability. Five repetitions were treated as a fixed effect, and the ICC (3,k) measured how consistently the same evaluator could measure the participant’s ankle movement angle under the same conditions. The angle measured per ankle movement task by the AMAS was above the ICC (3,1) of 0.76 under all conditions (

Table 4. Reliability results ICC (3,1).

		Comfortable				Slow
		ICC		95% CI		ICC		95% CI
				Lower	Upper			Lower	Upper
Dorsiflexion	maximum	0.83	**	0.68	0.93	0.98	**	0.95	0.99
	mild	0.95	**	0.89	0.98	0.93	**	0.87	0.98
Plantarflexion	maximum	0.93	**	0.85	0.97	0.87	**	0.75	0.95
	mild	0.86	**	0.73	0.94	0.76	**	0.58	0.9
Inversion	maximum	0.96	**	0.91	0.98	0.98	**	0.96	0.99
	mild	0.97	**	0.94	0.99	0.97	**	0.94	0.99
Eversion	maximum	0.76	**	0.58	0.9	0.86	**	0.73	0.95
	mild	0.9	**	0.8	0.96	0.94	**	0.88	0.98

** p < 0.01.

).

Furthermore, ICC(3,5) was more significant than 0.91 under all conditions (

Table 5. Reliability results ICC (3,5).

		Comfortable				Slow
		ICC		95% CI		ICC		95% CI
				Lower	Upper			Lower	Upper
Dorsiflexion	maximum	0.96	**	0.91	0.99	1	**	0.99	1
	mild	0.99	**	0.98	1	0.99	**	0.97	0.99
Plantarflexion	maximum	0.98	**	0.97	0.99	0.97	**	0.94	0.99
	mild	0.97	**	0.93	0.99	0.94	**	0.87	0.98
Inversion	maximum	0.99	**	0.98	1	1	**	0.99	1
	mild	0.99	**	0.99	1	0.99	**	0.99	1
Eversion	maximum	0.94	**	0.87	0.98	0.97	**	0.93	0.99
	mild	0.98	**	0.95	0.99	0.99	**	0.97	1

** p < 0.01.

).

3.3. Consistency

Bland–Altman plots were constructed between the AMAS and IMU against 2D motion analysis performed from the front with high values for the lateral and correlation coefficients in the ankle dorsiflexion/plantarflexion and inversion/eversion tasks. The vertical axis (DIFF) is the difference between the 2D motion analysis and AMAS or IMU, and the horizontal axis (MMEAN) is the mean between the 2D motion analysis and AMAS or IMU. LOA represents the range within which the difference between two measurements is acceptable as an error. Bland–Altman plots of sagittal plane angles in the dorsiflexion/plantarflexion task did not converge within the LOA for both instruments (Figure 3a).

The AMAS had a mean value of 1.26, with an LOA of 23.58 to −21.07. However, the IMU had a mean of −1.82 and an LOA of 30.00 to −36.54. The Bland–Altman plots of the frontal plane angle in the inversion/eversion task did not converge within the LOA for either instrument (Figure 3b).

The mean difference between the two systems for the AMAS was 0.18, and the LOA was from 26.70 to −26.33. However, the IMU had a mean of −9.54 for the difference between the two systems, and the LOA was from 50.48 to −69.56. T-tests were used to confirm the presence of fixed errors. For each task, the AMAS and IMU for the 2D motion analysis differed significantly, with a two-tailed test and a probability of significance of <1% and 95% CI: (1.02–1.49) for the AMAS and ([−2.19] to [−1.46]) for IMU for the sagittal plane angle and ([−0.09] to [0.46]) for the AMAS and ([−10.16] to [−8.91]) for IMU for the frontal plane angle. A fixed error was suggested by the absence of 0 in the 95% CI, except for the angle of the frontal plane by the AMAS. Regression analysis was performed to confirm the presence of proportional errors. The probability of significance of the regression equation, with DIFF as the dependent variable and MMEAN as the independent variable, was <1% for all tasks for both measurement methods, indicating no proportional error.

4. Discussion

We compared the reliability and validity of the AMAS in kinematic analysis with existing kinematic analysis assessment systems.

Comparisons between conditions demonstrated that the 2D Motion Analysis (both front and back) and AMAS consistently detected significant differences between most conditions. This indicates that both systems are generally reliable for evaluating sagittal and frontal plane angles. In particular, the 2D Motion Analysis and AMAS proved effective in detecting differences during maximum vs. mild conditions for both sagittal and frontal plane angles. In contrast, the IMU showed limitations in detecting differences, particularly in subjective motor speed and frontal plane angles. These limitations may stem from higher variability, lower resolution, or differences in data processing methods. As a result, the IMU appears less suited for identifying smaller differences or subtle changes, especially in subjective motor speed and frontal plane angle conditions. Criterion validation showed a significantly greater than moderate correlation between the AMAS and 2D motion analysis from the sagittal and anterior planes in all motor tasks and conditions without affecting joint angles and movement speed. The correlation coefficient values of 0–0.3, 0.3–0.7, and 0.7–1.0 represent a weak, moderate, and strong correlation, respectively [57]. Therefore, the present results revealed a more than moderate correlation. However, the IMU had no or a significantly low correlation with the 2D motion analysis. This suggests that the AMAS is more valid than the IMU for measuring the forefoot’s sagittal and forehead angles. Outliers and fixation errors may have caused the lower validity of the IMU over the AMAS, which may be due to the magnetic effects. Biases exist in measurement systems, such as IMU, with an improved agreement with marker-based optical motion capture in the sagittal plane. Still, there are more significant errors in non-sagittal planes [58], and the differences vary among participants [59]. Standard wearable sensors’ position, calibration method, and measurement algorithm affect accuracy [60]. Electromagnetic interference makes IMUs unreliable indoors [61]. Therefore, measurements should be collected in a magnetically neutral environment [62]. This study was conducted in a laboratory, and the results are consistent with previous studies. Our findings show that AMAS is a highly versatile system for clinical use.

Regarding the regression model performance index validation results, the mean RMSE of the AMAS based on the 2D motion analysis was significantly lower than that of the IMU. However, the mean RMSE of the AMAS was 10.90° ± 2.93° for the sagittal angle and 13.44 ± 1.09° for the frontal angle. Each participant’s RMSE was 4.37–15.88° for the sagittal angle and 13.07–20.97° for the frontal angle, suggesting that significant differences in measurement angles were observed and errors were introduced. IMU kinematics should be interpreted cautiously [58,59] as the IMU must improve consistency and accuracy to replace marker-based optical 3D motion analysers, considered the gold standard. Our results confirm the limitations of joint kinematics analysis using IMUs owing to the magnetic effects.

Regarding the results of the reliability validation, the AMAS’ reliability of sagittal and forehead angles had an ICC value of ≥0.76 for all motor tasks and conditions. This suggests that the reliability of measurements by the AMAS is higher than that by the conventional method [56]. Because obtaining kinematic data from individuals using the 3D motion analysis with markers, the gold standard method, is costly and complex, there is growing interest in a simple, inexpensive system with no local limitations [63]. The joint angles of ankle motion measured by this system are comparable to those obtained using previous measurement methods. This system has been verified to be a highly convenient measurement device because it solves the problem of environmental restrictions. The system’s rotary encoder is a reliable and accurate sensor for measuring limb angles [64,65]; however, it is often attached to a rigid mechanical attachment on the exoskeleton and is, therefore, unsuitable as a wearable device [66]. Therefore, the external dimensions of this system were designed to be compact enough to be worn on the foot, increasing its versatility as a wearable device. However, regarding the results of the agreement validation, the Bland–Altman analysis showed that the differences between the 2D motion analysis and each of the measurement devices did not converge within the LOA, with no proportional error but a fixed error being found. The differences between the 2D motion analysis and AMAS were plotted upward in the sagittal and forehead planes, indicating that the 2D motion analysis tended to measure at lower values than those associated with the AMAS. However, the difference between 2D motion analysis and IMU was plotted downward in the sagittal and forehead planes, suggesting that IMU measurements tended to be higher than the 2D motion analysis measurements. The clinically acceptable joint angle error in gait motion analysis is <5° RMSE [67]. In the present study, the RMSE exceeded the clinically significant limit, although it differed from the gait motion. This raises the concern that while ankle joint angle measurement by AMAS can be used as a primary indicator, there may be some errors when comparing the ankle motion analysis data from other motion analysis devices.

A multidisciplinary team in collaboration with clinicians in the field is necessary to provide a diverse perspective [68]. The AMAS developed in our study measures ankle motion from two aspects, the sagittal plane and anterior frontal plane, and its high validity and high reliability have been verified using the measurement results from the 2D motion analysis. Additionally, the velocity of the ankle motion did not affect the system’s angular measurement. Poor joint angle changes, decreased velocity in various movements, and abnormal compensatory movements accompany movement in post-stroke patients. Kinematic movement analysis results are essential for distinguishing between motor recovery and compensatory movement patterns [69,70]. Therefore, validation is needed in the rehabilitation field in patients presenting with various motor impairments, ranging from mild to severe.

In summary, AMAS is an evaluation system specialising in ankle joint motion control. Further validation can be applied to other joints and daily life activities. The development of AMAS is anticipated to contribute to the realisation of a multidimensional assessment of motor control in biomechanics and rehabilitation. It needs to be validated in other joints to demonstrate its usefulness as a wearable system.

The present study has several limitations. One limitation is that the results of this study measure non-weight-bearing motion that may not be valid or reliable during walking. Therefore, further analysis of actual walking motion measurements may be necessary before concluding that AMAS can measure motion under load, such as walking motion. Another limitation is that the subjects of this study were all healthy. The task of this study was to set up a condition of slow movement and minimum range of motion, assuming a patient, but we did not measure an actual patient. Therefore, further analysis of patients presenting with various movement disorders may be warranted in the future.

5. Conclusions

Because of convenience, versatility, and the ability to synchronously measure various biological signals, the AMAS was developed to assess ankle joint control, which is essential for walking in rehabilitation settings and real-life environments. The occurrence of a fixation error suggests room for improvement in this system. With future enhancements against errors, clinical evaluation using this system may help elucidate the mechanism of ankle joint control and develop individualised and optimised rehabilitation.

6. Practical Applications

The importance of capturing such quantitative analysis of human movement in free daily activities has been highlighted [71], and the AMAS has the potential to be applied to the evaluation of ankle joint motor control in various activities of daily living (ADLs). For example, it could assess movements in which ankle joint control is crucial, such as ascending and descending stairs, rising from a chair, and crossing an obstacle. However, it is necessary to optimise the measurement protocols and consider precautions for data interpretation according to the characteristics of each movement.

7. Patents

Japanese Patent Application No. 2023-67115.