Machine Learning Model to Estimate Net Joint Moments during Lifting Task Using Wearable Sensors: A Preliminary Study for Design of Exoskeleton Control System

Abstract

Accurately measuring the lower extremities and L5/S1 moments is important since L5/S1 moments are the principal parameters that measure the risk of musculoskeletal diseases during lifting. In this study, protocol that predicts lower extremities and L5/S1 moments with an insole sensor was proposed to replace the prior methods that have spatial constraints. The protocol is hierarchically composed of a classification model and a regression model to predict joint moments. Additionally, a single LSTM model was developed to compare with proposed protocol. To optimize hyperparameters of the machine learning model and input feature, Bayesian optimization method was adopted. As a result, the proposed protocol showed a relative root mean square error (rRMSE) of 8.06~13.88% while the single LSTM showed 9.30~18.66% rRMSE. This protocol in this research is expected to be a starting point for developing a system for estimating the lower extremity and L5/S1 moment during lifting that can replace the complex prior method and adopted to workplace environments. This novel study has the potential to precisely design a feedback iterative control system of an exoskeleton for the appropriate generation of an actuator torque.

1. Introduction

More than 40% of workers suffer from work-related musculoskeletal disorders (WMSD) annually due to manual materials handling (MMH) tasks despite efforts to improve the work environment [1]. It has been reported that 52% of WMSDs are caused by lifting tasks, of which 62% are low-back musculoskeletal disorders [2]. The high mechanical load applied to the lumbar spine during lifting is considered a major risk factor for back pain. Studies have been conducted to measure the risk factor by calculating low-back loading [3]. Internal moments to a body segment are caused by a combined action of muscle strength within the joint or tensile force of the skin, joint capsule, and ligaments [4]. By calculating external joint moments, internal moments acting within each joint can be estimated. The L5/S1 moment tends to increase according to the weight of the object. Repetitive moment is one of the risk factors that can increase the incidence of low-back musculoskeletal disorders [5]. The lower extremity and L5/S1 moment are also used as evaluation parameters for lifting posture [6]. Therefore, the moment is being studied as a major parameter to determine the risk of musculoskeletal disorders during lifting [3,6,7,8].

Optical 3D motion tracking system (OTS) is a method of calculating the lower extremity and L5/S1 moments [9]. The OTS acquires optical marker trajectory and ground reaction force based on optical camera, force plate, and the subject’s anthropometric information. It then calculates the moment of each joint using the acquired optical marker trajectory, ground reaction force, and inverse dynamic method [10]. Calculating moments using an OTS is considered as the golden standard method. Many studies have been conducted to measure the lower extremity and L5/S1 moment during symmetric and asymmetric lifting [1,6,8,11,12]. However, the OTS has a disadvantage in that it is impossible to be utilized in a working environment due to spatial constraints [13,14]. To overcome a spatial limitation, methods using wearable sensors have been reported [15]. Wearable sensors such as inertial measurement unit (IMU), insole sensor, or portable force plate can be used to calculate joint moments [16,17]. In the case of an IMU sensor, the center of joint can be obtained by attaching it onto the body segment, with an insole sensor or portable force plate that can be worn on shoes to obtain ground reaction force data [18]. Previous studies suggest L5/S1 moment measurement system using 12–17 IMU sensors and 2 force shoes or insole sensors [18,19]. According to recent studies mentioned above, studies using human body modeling [16,17,18] based on wearable sensor data have shown the possibility of overcoming limitations of OTS using multiple wearable sensors. However, using multiple sensors showed a complicated application method, inducing unnatural movement and taking a longer time for detachment in a real working environment. In addition, a robust estimation of movement is difficult due to substantial movement variability and irregular data [20].

To overcome these limitations mentioned above, previous studies have been conducted to predict biomechanical parameters such as ground reaction force, joint torque, and joint moment by replacing it with artificial intelligence techniques [21,22]. Mundt et al. (2020) have conducted a study to estimate the angle and moment of lower extremity joints during gait using IMU sensors and machine learning techniques. Their research suggested a predictive model using long short-term memory (LSTM). The predicted moment and joint angle showed a normalized root mean square error (RMSE) of about 10% [23]. Additionally, Choi et al. (2019) have developed an LSTM model that predicts the inclination angle between the center of gravity and the center of pressure when walking using time-series data obtained from a single IMU sensor. Compared to an artificial neural network (ANN), the relative RMSE result of the LSTM model showed 6% reduction, indicating improved accuracy [24]. Previous studies have reported that the LSTM shows particularly high performance when predicting biomechanical parameters using motion sequenced sensor data while developing a machine learning model in the biomechanical field.

Therefore, in this study, we evaluated the performance of LSTM for predicting a joint moment based on plantar pressure data collected from a wearable insole sensor during a lifting task.

The purpose of this study is to propose a system that predicts lower extremities and L5/S1 joint moment using machine learning model and plantar pressures from an insole sensor system. In detail, we developed a new deep learning structure by combining classifier and regressor as one network to estimate the lower extremities and L5/S1 moments. Additionally, we evaluated the usefulness of the proposed machine learning model by applying to various lifting conditions.

2. Materials and Methods

2.1. Subjects, Apparatus, and Lifting Experiments

This experiment was conducted on 9 adult males (age: 23.1 ± 1.4 years, height: 176.6 ± 4.9 cm, weight: 69.3 ± 6.7 kg). The subjects were selected based on the questionnaires of the subjects considering general musculoskeletal diseases such as rheumatoid arthritis, osteoarthritis, and mechanical pain in the lower extremities and lumbar joints [25]. This experiment was approved by the local ethics committee. It was carried out following the guidelines of Sungkyunkwan University. In addition, written informed consent was obtained from all participants before the experiment.

As experimental equipment, six MCam2 cameras (VICON, Oxford Metrics, Oxford, UK) and 2 OR6-6-2000 ground reaction forces (AMTI Inc., Newton, MA, USA) were used. Each system was operated at sampling rates of 120 Hz and 1080 Hz, respectively [24,26,27]. Participants wore a Pedar-X insole system (Pedar Mobile, Novel Electronics Inc., GmbH, Munich, Germany). Data were acquired at 100 Hz [21,28]. The camera and ground reaction force data were time-series synchronized with a Vicon 460 system. The motion analysis system and the Pedar-X insole system were manually synchronized based on the instance when the subject lifted a box from the ground during a lifting task.

Optical markers were attached to 16 anatomical landmarks of the lumbar spine and lower extremity joints of each subject based on the modified Helen Hayes marker protocol [11,26]. In addition, one optical marker was attached to the box to distinguish the event of the lifting task. Each subject performed warm-up exercises before participating in this experiment. Preliminary movements were performed to adapt to the experimental environment. For the lifting motion, two postures were performed: squat and stoop. Each lifting motion was performed with boxes (Size: 32 × 40 × 25 cm³) having 3 different weights (4, 8, and 12 kg). The speed of the lifting task was set as normal, fast, or slow. Each speed indicated the normal, fast, and slow speed considered by the subject, respectively. Each subject performed the 54 times of lifting motions (3 repetitions for 18 types of lifting tasks: 2 postures × 3 weights × 3 speeds). Subjects are asked to lift boxes from ground to the knuckle height [5,29].

2.2. Data Processing

Optical marker trajectories obtained by OTS were filtered by 4th-order, low-pass Butterworth filter and cut-off frequency was set at 5 Hz [12]. Kinematic data of the lower extremities and L5/S1 moments were calculated using Euler angle and numerical differentiation methods. Based on the kinematic data, the ground reaction force data of 6 axes, and the subject’s anthropometric information, the lower extremities and L5/S1 moments of sagittal plane were calculated. These calculations were performed using kinematic and kinetic models developed in the Biomedical Engineering Laboratory of Sungkyunkwan University [8,30]. During the lifting task, pressure data obtained from the insole sensor were filtered with 3rd Butterworth, zero-lag low-pass filter and the frequency was set at 7 Hz [31]. The event of the lifting task was segmented based on the optical marker attached to the box. The time when the box was moved from the ground was marked as the starting point and the time when the box reached the highest point was marked as end of task. The amplitude of moment obtained from the OTS and 99 pressure data collected from the insole system were normalized by dividing the subject’s body weight (BW). In addition, the length of moment and pressure data were normalized to 100% for each trial [27]. In the proposed model, each posture was labeled for motion classification where the squat posture was labeled as 0 and the stoop posture was labeled as 1. The total insole pressure, lower extremities, and L5/L1 moments dataset was 480.

2.3. Neural Network Architecture

In this study, an architecture hierarchically composed of a classification model and a regression model was proposed to predict lower extremities and L5/S1 moments during symmetric lifting using plantar pressure obtained from the insole sensor. In order to classify the lifting motion, the following three machine learning techniques commonly used as classification models in previous studies [32,33] were used: ANN, Support Vector Machine (SVM), Random Forest (RF). The LSTM model showing relatively high performance on a time-series data set [34,35,36,37] was used as a regression model for predicting joint moment. Figure 1 presents the structure of the proposed architecture and the overall flow chart to predict lower extremities and L5/S1 moments on sagittal plane.

The proposed architecture consisted of 1 classification model that could classify the lifting posture, and 2 regression models for the lower extremities and L5/S1 moments. For each classification model, pre-processed pressure data acquired from the left insole of the Pedar-X were used as input values, and the labeled index for lifting posture was used as an output value [6]. Based on results of the classification model, dataset was classified into squat data and stoop data. Pressure values and lower extremities and L5/S1 moments were used as input and output for each regression model, respectively.

To evaluate the performance of the proposed model, a single machine learning model composed of single LSTM was constructed, and the input and output values were as follows. The pressure data obtained from the left insole of Pedar-X were used as input data, and the predicted values were the sagittal moment of the left ankle, left knee, left hip, and L5/S1 was used.

In this study, performance was evaluated through a 10-fold cross-validation. The ratio of training, validation, and testing data was 80:10:10 (the number of training data set: 384, the number of validation data set: 48, the number of test data set: 48). All models were trained and tested using MATLAB R2020a version (The Mathworks, Inc., Natick, MA, USA) and RTX 2080 Ti GPU.

2.4. Hyperparameter Optimization

Hyperparameters of all models used in this study were optimized using the Bayesian optimization method, as an efficient method to find the optimal solution for a function with a large amount of computation based on the Gaussian probability [38]. As an optimization objective function of the proposed architecture, it is constructed based on the error rate of the classification model and the rRMSE of the regression model, which is shown in Equations (1) and (2). In addition, the single LSTM model used rRMSE as the objective function:

$$f\left(x\right)=\min \left(error\right)$$

(1)

$$error=w_{1}x+w_{2}y$$

(2)

where f(x) is the optimization objective function and error is value calculated based on the error rate of the classification model and the rRMSE of the regression model. The x is the error rate of the classification model and y is the rRMSE of the regression model. In addition, $w_1$ and $w_2$ presented as the weighted values of the error rate of the classification model and the rRMSE value of the regression model, respectively. Weights are set as 0.7 and 0.3, respectively [39]. The maximum optimization iteration was set to 100.

Table 1. The classification model’s optimized hyperparameter and their ranges by the Bayesian optimization method.

ANN Hyperparameter	Neuron	Momentum	Learning Rate	Training Function
Range	20–200	10−5–10−2	10−6–10−2	trainscg, trainlm, traingdx
Selected Parameter	35	0.0036	4.45 × 10−5	trainscg
RF Hyperparameter	Min Leaf Size	Max Number Splits	Split Criterion	Variables’ Sample
Range	1–240	1–479	gdi, deviance, twoing	1–99
Selected Parameter	5	177	Deviance	99
SVM Hyperparameter	Box Constraint	Kernel Scale	Kernel Function	Polynomial Order
Range	10−5–10−3	10−5–103	Gaussian, linear, polynomial	2–4
Selected Parameter	9.75 × 102	10−3	polynomial	2

presents the classification model’s hyperparameter and their ranges optimized by the Bayesian optimization method.

Table 2. LSTM model’s hyperparameters and their structure selected by the Bayesian optimization method.

Phase	Hyperparameter	Range
Structure	Bi-LSTM layer1	99–990
	Dropout layer	0.5–0.95
	Bi-LSTM layer2	99–990
	Dropout layer	0.5–0.95
	Bi-LSTM layer3	99–990
	Fully connected layer1	10–50
	Dropout layer	0.5–0.95
	Fully connected layer2	Fully connected layer1/2
Training	Momentum	0.5–0.95
	L2 regularization factor	10−5–10−2
	Initial learning rate	10−5–10−2
	Input weights initializer	Glorot, He, Narrow-normal
	Gradient threshold method	Global-l2norm, l2norm
	Gradient threshold	1–6
	Number of layers	1–3
	Number of sensors selected	3–99

shows the LSTM model’s hyperparameters and ranges. Optimized results of the LSTM models are shown in

Table 3. LSTM model’s structure and hyperparameters optimized by the Bayesian optimization method.

Phase	Hyperparameter	LSTM	ANN-LSTM		RF-LSTM		SVM-LSTM
			Squat	Stoop	Squat	Stoop	Squat	Stoop
Regression Structure	Bi-LSTM layer1 Node	103	610	88	625	211	104	168
	Dropout layer	0.172	0.120	-	0.101	-	-	0.267
	Bi-LSTM layer2 Node	103	439	-	387	-	-	167
	Dropout layer	0.172	0.1203	-	0.101	-	-	0.267
	Bi-LSTM layer3 node	31	316	-	240	-	-	166
	Fully connected layer1	43	20	50	26	36	17	31
	Dropout layer	0.172	0.120	0.386	0.101	0.136	0.149	0.267
	Fully connected layer2	22	10	25	13	18	9	16
	Fully connected layer3	4	4	4	4	4	4	4
Training	Momentum	0.920	0.503	0.812	0.530	0.701	0.643	0.709
	L2 regularization factor	8.49 × 10−4	6.28 × 10−3	8.41 × 10−3	7.16 × 10−3	9.07 × 10−3	8.45 × 10−3	4.84 × 10−3
	Initial learning rate	2.67 × 10−3	1.87 × 10−3	8.41 × 10−3	9.27 × 10−4	4.89 × 10−3	6.75 × 10−3	2.78 × 10−3
	Input weights initializer	He	Glorot	Glorot	Glorot	Narrow-normal	Glorot	Glorot
	Gradient threshold method	l2norm	Global-l2norm	Global-l2norm	Global-l2norm	Global-l2norm	l2norm	l2norm
	Gradient threshold	2	1	5	1	3	1	4
	Number of layers	3	3	1	3	1	1	3

.

2.5. Data Analysis

The classification model of symmetric lifting posture was evaluated for classification accuracy rate. The performance of the regression model was evaluated with RMSE, Pearson correlation coefficient (R), and rRMSE using moments of the lower extremities and L5/S1 obtained from the OTS as a reference system. RMSE, rRMSE, and R are calculated with Equations (3)–(5), respectively:

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{\displaystyle \sum }_{i=1}^N{\left(y_i-y_i^{*}\right)}^2}$$

(3)

$$\mathrm{R}=\frac{{{\displaystyle \sum }}_{i=1}^N\left(y_i-\overline{y_i}\right)\left(y_i^{*}-\overline{y_i^{*}}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^N{\left(y_i-\overline{y_i}\right)}^2{\left(y^{*}{}_i-\overline{y_i^{*}}\right)}^2}}$$

(4)

$$\mathrm{rRMSE}=\frac{\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{\left(y_i-y_i^{*}\right)}^2}}{\max \left(y_i\right)-\min \left(y_i\right)}\times 100\%$$

(5)

where $y_i$ represents the actual value, $y_i{}^{*}$ represents the predicted value from the artificial intelligence model, and N represents the length of 1 trial. In Equation (2), R represents the value of the Pearson correlation coefficient. $\overline{y_i}$ is the average of the actual values, $\overline{y_i^{*}}$ is the average of the values predicted by the artificial intelligence model, $\max \left(y_i\right)$ is the maximum value in 1 trial, and $\min \left(y_i\right)=$ is the minimum value in 1 trial. One-way analysis of variance (ANOVA) was performed to compare the accuracy of the single LSTM and the proposed artificial intelligence model. After the analysis, post hoc Tukey’s method was used. The significance level was set at p < 0.01. All statistics analyses were performed using PASW Statistics 18 (Ver. 18, SPSS Inc., Chicago, IL, USA).

For moments obtained from the proposed model and the reference system, the maximum value among absolute values of moments was extracted for each trial. The error rate between the extracted peak moment value of the proposed model and the reference system was calculated using the following Equation (6):

$$Error rate(\%)=\frac{ Peak{ }_{Model}-Peak{ }_{ref}}{Peak{ }_{ref}} \ast 100\%$$

(6)

where $Peak{ }_{Model}$ was the peak value among absolute values of prediction values of the artificial intelligence model, and $Peak{ }_{ref}$ was the peak value among absolute values of the reference moment. $Error rate(\%)$ meant the error rate between peak moment.

3. Results

Figure 2 presents results of feature selection on the number of sensors as input data using the Bayesian optimization method. For single LSTM, ANN-LSTM, RF-LSTM and SVM-LSTM models, 96, 96, 98 and 99 sensors were selected, respectively. Results of the objective function at the time when the sensor was selected were 14.49, 15.38, 6.53 and 4.94, respectively.

Table 4. Symmetric lifting task posture classification accuracy for three classification models.

	1. ANN	2. RF	3. SVM	ANOVA	Post Hoc Test
Squat accuracy (%)	87.06 ± 9.11	91.36 ± 7.16	92.84 ± 6.00	-	-
Stoop accuracy (%)	80.26 ± 5.67	92.15 ± 5.57	94.59 ± 2.93	-	-
Total accuracy (%)	83.54 ± 4.48	91.67 ± 3.40	94.00 ± 2.43	F = 16.1 (p < 0.01)	2, 3 > 1

shows classification accuracies of ANN, RF, and SVM as motion classification models of the proposed algorithm. Based on confusion matrix results of 10-fold cross-validation, average accuracies of ANN, RF, and SVM were 83.54 ± 4.48, 91.67 ± 3.40, and 94.00 ± 2.43%, respectively. There was a significant difference (p < 0.01) between the accuracy of ANN, RF, and SVM models.

In

Table 5. RMSE, rRMSE and R (correlation coefficients) values between calculated joint moment from optical tracking system and predicted joint moment with a hierarchical deep learning model.

		Ankle			Knee			Hip			L5S1
		r	RMSE (N∙m/BW)	rRMSE (%)	r	RMSE (N∙m/BW)	rRMSE (%)	r	RMSE (N∙m/BW)	rRMSE (%)	r	RMSE (N∙m/BW)	rRMSE (%)
Squat	LSTM	0.85	0.056 ± 0.007	12.19 ± 3.21	0.83	0.150 ± 0.010	18.64 ± 2.18	0.94	0.127 ± 0.017	9.38 ± 1.40	0.96	0.189 ± 0.021	9.95 ± 0.97
	NN-LSTM	0.87	0.052 ± 0.005	11.72 ± 2.04	0.79	0.167 ± 0.033	20.35 ± 5.80	0.95	0.121 ± 0.016	8.81 ± 3.28	0.94	0.174 ± 0.024	9.23 ± 1.44
	RF-LSTM	0.92	0.050 ± 0.004	10.69 ± 0.86	0.87	0.126 ± 0.032	16.33 ± 2.97	0.96	0.108 ± 0.024	8.06 ± 1.51	0.95	0.167 ± 0.033	8.50 ± 1.87
	SVM-LSTM	0.93	0.048 ± 0.007	10.48 ± 1.65	0.91	0.121 ± 0.023	14.03 ± 3.23	0.96	0.112 ± 0.030	8.08 ± 1.49	0.95	0.167 ± 0.026	8.44 ± 1.30
Stoop	LSTM	0.94	0.052 ± 0.005	11.58 ± 1.75	0.88	0.155 ± 0.032	18.70 ± 4.01	0.94	0.128 ± 0.017	9.26 ± 1.40	0.94	0.194 ± 0.029	10.04 ± 0.75
	NN-LSTM	0.86	0.059 ± 0.009	12.66 ± 3.21	0.83	0.153 ± 0.042	18.36 ± 4.32	0.95	0.136 ± 0.043	9.59 ± 1.67	0.93	0.206 ± 0.036	11.20 ± 2.14
	RF-LSTM	0.93	0.049 ± 0.007	10.12 ± 2.36	0.90	0.120 ± 0.034	15.36 ± 3.11	0.96	0.102 ± 0.023	7.91 ± 1.90	0.95	0.176 ± 0.027	9.64 ± 1.82
	SVM-LSTM	0.94	0.048 ± 0.009	10.22 ± 3.17	0.92	0.117 ± 0.031	13.73 ± 2.51	0.96	0.113 ± 0.028	8.02 ± 2.47	0.96	0.173 ± 0.021	8.81 ± 1.17

, RMSE, rRMSE, and R of each model were calculated to evaluate the performance of single LSTM, ANN-LSTM, RF-LSTM, and SVM-LSTM models. With the squat posture, rRMSE (%) results of single LSTM, ANN-LSTM, RF-LSTM, and SVM-LSTM were 9.38~18.64, 8.81~20.35, 8.06~16.33 and 8.44~14.03, respectively. With the stoop posture, rRMSE (%) values were 9.26~18.70, 9.59~18.36, 7.91~15.36 and 8.02~13.73, respectively. In both squat and stoop lifting tasks, the knee showed the highest rRMSE and a low correlation coefficient while the hip joint showed the lowest rRMSE with a high correlation coefficient.

Figure 3 shows performances of single LSTM, ANN-LSTM, RF-LSTM, and SVM-LSTM models by rRMSE. The SVM-LSTM model showed significant differences (p < 0.01) in ankle and L5S1 joints compared with single LSTM and ANN-LSTM models. In addition, for the knee joint, the SVM-LSTM model showed a significant difference (p < 0.01) compared with all other models. Compared to the single LSTM model, the SVM-LSTM model showed approximately 2.19%, 4.68%, 1.23% and 1.35% rRMSE reductions for the ankle, knee, hip, and L5/S1, respectively. In particular, the knee joint moment showed the largest decrease in rRMSE compared to other joints.

Figure 4 shows the moment predicted by the single LSTM model, the SVM-LSTM model and representative values of the reference moment classified according to posture. The x-axis is the time normalized to 100% during the lifting task and the y-axis is the moment normalized by the weight of each subject. (−) on the y-axis means flexion moment and (+) refers to the extension moment. During the lifting task, changes in the moment of the knee joint showed a different tendency from those of the ankle, hip joint, and lumbar joint. In the squat posture, the peak extension moment at the knee joint occurred at the beginning of the lifting motion and showed a pattern of decreasing close to zero. In stoop posture, the peak flexion moment occurred initially, and the flexion moment tended to decrease as the lifting task progressed.

Table 6. Reference peak moment calculated from OTS, peak moment predicted from SVM-LSTM model, and error rate between these values.

	Ankle	Knee	Hip	L5S1
Reference peak moment (N∙m/BW)	0.706 ± 0.034	0.768 ± 0.041	1.399 ± 0.037	2.504 ± 0.047
Predicted peak moment (N∙m/BW)	0.738 ± 0.036	0.735 ± 0.067	1.414 ± 0.031	2.377 ± 0.087
Error rate (%)	5.20 ± 4.85	8.77 ± 6.45	3.20 ± 2.33	5.19 ± 3.18

shows the reference peak moment, the predicted peak value of the SVM-LSTM model, and the error rate of the peak moment of the lower extremity and L5/S1. Error rates were 5.20 ± 4.85 (%), 8.77 ± 6.45 (%), 3.20 ± 2.33 (%), and 5.19 ± 3.18 (%) for the ankle, knee, hip and L5/S1, respectively.

4. Discussion

In this study, a novel machine learning-based protocol was proposed to predict lower extremity and L5/S1 moments during symmetrical lifting using plantar pressure as an input. This study designed a structure that combines SVM, a classifier, and LSTM model, one of the time-series prediction deep learning models. The performance of the SVM-LSTM model showed rRMSE (%) of 8.06~13.88, and peak moment error rate (%) of 3.20~8.77 (Table 3 and Table 5).

Recently, cases of estimating the moment of the L5/S1 during lifting using wearable sensor data and human body modeling showed rRMSE of 8~10% and peak moment error rate of 5~10% by wearing 6~17 IMU sensors and two portable force plate or insole sensors [19,38]. These could be judged to have similar accuracy to results of this study, which estimated the lower extremity and L5/S1 moments during symmetrical lifting using 2 insole sensors and the SVM-LSTM hierarchical machine learning model. According to Matijevich (2020), the architecture presented in this study is a system that wears two insoles [17]. It is expected to show higher applicability to the working environment than a system that wears 6–19 wearable sensors (e.g., IMU sensors) [17]. Kipp et al. (2018) developed a model that predicts the lower extremity moment during weightlifting operation using artificial intelligence and optical marker trajectories. As a result, the error rate of peak moment was about 5 to 16% [40]. Considering the similarity between weightlifting and symmetric lifting, the deep learning-based model proposed in this study showed similar results with a small number of sensors, indicating that the model proposed in this study could be more applicable to the field in terms of convenience than models proposed in previous studies (Table 3).

The proposed deep learning-based moment prediction model consists of a hierarchical structure, which is a step of classifying posture (stop and square) and a step of estimating the moment. Collected data were analyzed by dividing the lifting operation into three types of conditions (speed, weight, and posture). As a result, the foot pressure which was the input of the proposed model showed a clear difference according to posture in the cumulative pressure distribution of 1 trial based on the result classification model composed the posture among three types of conditions as the output of the classification model [41]. According to results of this study, the knee joint showed a high rRMSE (4.68%) reduction compared to other joints for the predicted moment between the SVM-LSTM model and the single LSTM model. Khashei et al. (2012) have reported that the hierarchical architecture is highly likely to improve the performance due to reduced data complexity [42]. However, in the case of the ANN-LSTM model compared with the single LSTM model, rRMSE and correlation coefficient showed no significant performance improvement. This was because classification accuracy of ANN was 83.5 ± 4.5%, which was low compared to accuracies of other RF and SVM classification models. This might be due to the accumulation of errors due to misclassification of the ANN classification model. Hierarchical architecture can be applied as a method to improve performance when a single model does not show high performance due to complexity of input/output data. However, errors can accumulate depending on the performance of the classification model. Thus, it is necessary to consider the performance of the classification model.

In this study, Bayesian optimization was used to obtain the optimal sensor location information from the acquired 99 pressure sensor data. According to a previous study, the main change direction of the pressure center of the foot during symmetric lifting was the anterior/posterior direction, and a constraint was given to select the number of at least 1 sensor in each area to select the sensor position for the entire range of the foot [43]. When the subject wears an insole sensor, the position and absolute value at which the sensor is pressed are different depending on the position of the sensor and the size of the foot. Thus, a large number of sensors was selected. Additionally, in this study, the area of the foot was divided into 3 zones (forefoot, midfoot, rearfoot) and the sensor position was extracted for each area. In the case of a different task, for example, an asymmetric lifting, it is necessary to consider a method of selecting an optimal position by setting the region differently according to the position of the pressed sensor. As an objective function, the error rate of the classification model and the rRMSE of the regression model were calculated by giving 7 and 3 weights, respectively. This is because, in the case of a model using a hierarchically structured protocol such as in this study, the error in the classification model has a structure in which the error in the subsequent regression model can be accumulated. To minimize the misclassification rate, the ratio of 7 to 3 was chosen. However, if weights of the objective function for the classification model and the regression model are different, the selection of features might be different.

The purpose of this study is to propose a system that can predict the joint moments of the lower extremities and L5/S1 in the workplace. The proposed system is meaningful in overcoming the spatial constraints of existing studies. It is considered that this system can be applied to workers who are engaged in an irregular working environment. For example, paramedics show a 13 times higher risk of musculoskeletal disorders than other medical workers, mainly due to frequent lifting of patients [44]. In addition, agricultural workers are also exposed to musculoskeletal disorders caused by lifting work, carrying heavy objects, or improper posture [45,46]. Based on the system proposed in this study, a prediction of the lower extremities and L5/S1 joint moment might provide a continuous health notification for workers who do not have an irregular working environment. Therefore, the risk of musculoskeletal disorders can be reduced by applying the health notification which includes the system proposed in this research.

This study has several limitations. First, the data used for training the model consisted of only the data of young adult males. However, to the best of our knowledge, this is the first study to predict moments of the lower extremities and lumbar spine during work-related lifting motions using insole sensors and machine learning. In future research, it is necessary to include the various age groups in order to increase the usefulness of the model. Second, a model was constructed for limiting the lifting task. In this study, models were constructed for the squat and stoop posture during symmetric lifting. In future research, it is necessary to construct a generalized model for lifting tasks by adding data to symmetric lifting posture such as free lifting, semi-squat, and asymmetric lifting tasks. Lastly, a limitation of this study is that it was conducted with limited number of healthy adults. The subjects’ musculoskeletal disorders were defined by their opinion without any medical history confirmation. Consequently, laterality or underlying physical condition was not considered. In future study, obtaining more data while considering laterality and medical history is necessary to construct a more robust machine learning model.

In conclusion, we proposed an architecture that could predict the lower extremity and L5/S1 moments as plantar pressure during symmetric lifting. In order to improve the performance of the machine learning model, a hierarchical architecture was presented and the performance was evaluated by comparing with the reference value. The accuracy between the single LSTM model and each proposed architecture was then compared. The hierarchical architecture was divided into two stages. The first stage consists of classifying the posture by constructing a motion classification model and the second stage predicted the lower extremity and L5/S1 moment using the data classified by posture. Among classification models of the proposed architecture, SVM showed the highest classification accuracy and the SVM-LSTM model showed an rRMSE of 8.06~13.88%, and an error rate of a peak moment of 3.20~8.77%. This is similar to the case of estimating the moment using a wearable system. Therefore, the protocol for estimating the lower extremity and L5/S1 moment during lifting using the insole system and artificial intelligence proposed in this study provides a starting point for developing a system for estimating the lower extremity and L5/S1 moment during lifting that can be applied to future workplace environments. This novel study has the potential to precisely design feedback iterative control system of an exoskeleton for the appropriate generation of an actuator torque.